Skip to content

Replace container-stats with golang client #2188

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open

Replace container-stats with golang client #2188

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
0518786
Replace container-stats with golang client
erthalion Jun 20, 2025
6d7e570
No need to access executor directly
erthalion Jun 23, 2025
87c5799
Avoid separate variable for the context
erthalion Jun 27, 2025
2108ff7
Avoid separate variable for the context
erthalion Jun 27, 2025
724f0d8
Share stats context between the routine and API
erthalion Jun 27, 2025
046ab03
Rename the method to represent new functionality
erthalion Jun 27, 2025
c5279ca
Shuffle stats variable around
erthalion Jun 27, 2025
b966ef2
Comment on one-shot
erthalion Jun 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 8 additions & 5 deletions integration-tests/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,19 @@ toolchain go1.23.6
require (
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc
github.com/docker/cli v26.1.3+incompatible
github.com/docker/docker v26.1.3+incompatible
github.com/docker/docker v28.2.2+incompatible
github.com/gonum/stat v0.0.0-20181125101827-41a0da705a5b
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510
github.com/hashicorp/go-multierror v1.1.1
github.com/pkg/errors v0.9.1
github.com/prometheus/common v0.55.0
github.com/stackrox/rox v0.0.0-20210914215712-9ac265932e28
github.com/stretchr/testify v1.9.0
github.com/thoas/go-funk v0.9.3
go.opentelemetry.io/otel/trace v1.28.0
golang.org/x/exp v0.0.0-20240506185415-9bf2ced13842
golang.org/x/sys v0.26.0
google.golang.org/grpc v1.65.0
google.golang.org/grpc v1.67.0
gopkg.in/yaml.v3 v3.0.1
k8s.io/api v0.32.2
k8s.io/apimachinery v0.32.2
Expand All @@ -34,6 +35,8 @@ require (
github.com/blang/semver/v4 v4.0.0 // indirect
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/containerd/errdefs v1.0.0 // indirect
github.com/containerd/errdefs/pkg v0.3.0 // indirect
github.com/distribution/reference v0.6.0 // indirect
github.com/docker/docker-credential-helpers v0.8.1 // indirect
github.com/docker/go-connections v0.5.0 // indirect
Expand Down Expand Up @@ -71,18 +74,18 @@ require (
github.com/mitchellh/go-wordwrap v1.0.1 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/moby/spdystream v0.5.0 // indirect
github.com/moby/sys/atomicwriter v0.1.0 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/morikuni/aec v1.0.0 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.0 // indirect
github.com/planetscale/vtprotobuf v0.6.0 // indirect
github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/prometheus/client_golang v1.19.1 // indirect
github.com/prometheus/client_model v0.6.1 // indirect
github.com/prometheus/common v0.55.0 // indirect
github.com/prometheus/procfs v0.15.1 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/spf13/cobra v1.8.1 // indirect
Expand All @@ -107,7 +110,7 @@ require (
golang.org/x/text v0.19.0 // indirect
golang.org/x/time v0.7.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20240826202546-f6391c0de4c7 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240826202546-f6391c0de4c7 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240903143218-8af14fe29dc1 // indirect
google.golang.org/protobuf v1.35.1 // indirect
gopkg.in/evanphx/json-patch.v4 v4.12.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
Expand Down
24 changes: 16 additions & 8 deletions integration-tests/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@ github.com/cenkalti/backoff/v4 v4.3.0 h1:MyRJ/UdXutAwSAT+s3wNd7MfTIcy71VQueUuFK3
github.com/cenkalti/backoff/v4 v4.3.0/go.mod h1:Y3VNntkOUPxTVeUxJ/G5vcM//AlwfmyYozVcomhLiZE=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/containerd/errdefs v1.0.0 h1:tg5yIfIlQIrxYtu9ajqY42W3lpS19XqdxRQeEwYG8PI=
github.com/containerd/errdefs v1.0.0/go.mod h1:+YBYIdtsnF4Iw6nWZhJcqGSg/dwvV7tyJ/kCkyJ2k+M=
github.com/containerd/errdefs/pkg v0.3.0 h1:9IKJ06FvyNlexW690DXuQNx2KA2cUJXx151Xdx3ZPPE=
github.com/containerd/errdefs/pkg v0.3.0/go.mod h1:NJw6s9HwNuRhnjJhM7pylWwMyAkmCQvQ4GpJHEqRLVk=
github.com/containerd/log v0.1.0 h1:TCJt7ioM2cr/tfR8GPbGf9/VRAX8D2B4PjzCpfX540I=
github.com/containerd/log v0.1.0/go.mod h1:VRRf09a7mHDIRezVKTRCrOq78v577GXq3bSa3EhrzVo=
github.com/cpuguy83/go-md2man/v2 v2.0.4/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
Expand All @@ -26,8 +30,8 @@ github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5Qvfr
github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
github.com/docker/cli v26.1.3+incompatible h1:bUpXT/N0kDE3VUHI2r5VMsYQgi38kYuoC0oL9yt3lqc=
github.com/docker/cli v26.1.3+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/docker v26.1.3+incompatible h1:lLCzRbrVZrljpVNobJu1J2FHk8V0s4BawoZippkc+xo=
github.com/docker/docker v26.1.3+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker v28.2.2+incompatible h1:CjwRSksz8Yo4+RmQ339Dp/D2tGO5JxwYeqtMOEe0LDw=
github.com/docker/docker v28.2.2+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker-credential-helpers v0.8.1 h1:j/eKUktUltBtMzKqmfLB0PAgqYyMHOp5vfsD1807oKo=
github.com/docker/docker-credential-helpers v0.8.1/go.mod h1:P3ci7E3lwkZg6XiHdRKft1KckHiO9a2rNtyFbZ/ry9M=
github.com/docker/go-connections v0.5.0 h1:USnMq7hx7gwdVZq1L49hLXaFtUdTADjXGp+uj1Br63c=
Expand Down Expand Up @@ -126,6 +130,10 @@ github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3N
github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo=
github.com/moby/spdystream v0.5.0 h1:7r0J1Si3QO/kjRitvSLVVFUjxMEb/YLj6S9FF62JBCU=
github.com/moby/spdystream v0.5.0/go.mod h1:xBAYlnt/ay+11ShkdFKNAG7LsyK/tmNBVvVOwrfMgdI=
github.com/moby/sys/atomicwriter v0.1.0 h1:kw5D/EqkBwsBFi0ss9v1VG3wIkVhzGvLklJ+w3A14Sw=
github.com/moby/sys/atomicwriter v0.1.0/go.mod h1:Ul8oqv2ZMNHOceF643P6FKPXeCmYtlQMvpizfsSoaWs=
github.com/moby/sys/sequential v0.6.0 h1:qrx7XFUd/5DxtqcoH1h438hF5TmOvzC/lspjy7zgvCU=
github.com/moby/sys/sequential v0.6.0/go.mod h1:uyv8EUTrca5PnDsdMGXhZe6CCe8U/UiTWd+lL+7b/Ko=
github.com/moby/term v0.5.0 h1:xt8Q1nalod/v7BqbG21f8mQPqH+xAaC9C3N3wfWbVP0=
github.com/moby/term v0.5.0/go.mod h1:8FzsFHVUBGZdbDsJw/ot+X+d5HLUbvklYLJ9uGfcI3Y=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
Expand All @@ -151,8 +159,8 @@ github.com/opentracing/opentracing-go v1.2.0/go.mod h1:GxEUsuufX4nBwe+T+Wl9TAgYr
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/planetscale/vtprotobuf v0.6.0 h1:nBeETjudeJ5ZgBHUz1fVHvbqUKnYOXNhsIEabROxmNA=
github.com/planetscale/vtprotobuf v0.6.0/go.mod h1:t/avpk3KcrXxUnYOhZhMXJlSEyie6gQbtLq5NM3loB8=
github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 h1:GFCKgmp0tecUJ0sJuv4pzYCqS9+RGSn52M3FUwPs+uo=
github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10/go.mod h1:t/avpk3KcrXxUnYOhZhMXJlSEyie6gQbtLq5NM3loB8=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
Expand Down Expand Up @@ -286,10 +294,10 @@ golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8T
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/genproto/googleapis/api v0.0.0-20240826202546-f6391c0de4c7 h1:YcyjlL1PRr2Q17/I0dPk2JmYS5CDXfcdb2Z3YRioEbw=
google.golang.org/genproto/googleapis/api v0.0.0-20240826202546-f6391c0de4c7/go.mod h1:OCdP9MfskevB/rbYvHTsXTtKC+3bHWajPdoKgjcYkfo=
google.golang.org/genproto/googleapis/rpc v0.0.0-20240826202546-f6391c0de4c7 h1:2035KHhUv+EpyB+hWgJnaWKJOdX1E95w2S8Rr4uWKTs=
google.golang.org/genproto/googleapis/rpc v0.0.0-20240826202546-f6391c0de4c7/go.mod h1:UqMtugtsSgubUsoxbuAoiCXvqvErP7Gf0so0mK9tHxU=
google.golang.org/grpc v1.65.0 h1:bs/cUb4lp1G5iImFFd3u5ixQzweKizoZJAwBNLR42lc=
google.golang.org/grpc v1.65.0/go.mod h1:WgYC2ypjlB0EiQi6wdKixMqukr6lBc0Vo+oOgjrM5ZQ=
google.golang.org/genproto/googleapis/rpc v0.0.0-20240903143218-8af14fe29dc1 h1:pPJltXNxVzT4pK9yD8vR9X75DaWYYmLGMsEvBfFQZzQ=
google.golang.org/genproto/googleapis/rpc v0.0.0-20240903143218-8af14fe29dc1/go.mod h1:UqMtugtsSgubUsoxbuAoiCXvqvErP7Gf0so0mK9tHxU=
google.golang.org/grpc v1.67.0 h1:IdH9y6PF5MPSdAntIcpjQ+tXO41pcQsfZV2RxtQgVcw=
google.golang.org/grpc v1.67.0/go.mod h1:1gLDyUQU7CTLJI90u3nXZ9ekeghjeM7pTDZlqFNg2AA=
google.golang.org/protobuf v1.35.1 h1:m3LfL6/Ca+fqnjnlqQXNpFPABW1UD7mjh8KO2mKFytA=
google.golang.org/protobuf v1.35.1/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
Expand Down
11 changes: 11 additions & 0 deletions integration-tests/pkg/executor/executor.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
package executor

import (
"context"
"os/exec"
"time"

"github.com/stackrox/collector/integration-tests/pkg/config"
)
Expand All @@ -16,6 +18,14 @@ type ContainerLogs struct {
Stderr string
}

type ContainerStat struct {
Timestamp time.Time
Id string
Name string
Mem uint64
Cpu uint64
}

func (c *ContainerLogs) Empty() bool {
return *c == (ContainerLogs{})
}
Expand Down Expand Up @@ -47,6 +57,7 @@ type Executor interface {
StopContainer(name string) (string, error)
StartContainer(config config.ContainerStartConfig) (string, error)
GetContainerHealthCheck(containerID string) (string, error)
GetContainerStats(ctx context.Context, containerID string) (*ContainerStat, error)
GetContainerIP(containerID string) (string, error)
GetContainerLogs(containerID string) (ContainerLogs, error)
CaptureLogs(testName, containerName string) (string, error)
Expand Down
17 changes: 17 additions & 0 deletions integration-tests/pkg/executor/executor_cri.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -289,6 +289,23 @@ func (c *criExecutor) GetContainerHealthCheck(containerID string) (string, error
return "", fmt.Errorf("Unsupported")
}

func (c *criExecutor) GetContainerStats(ctx context.Context, containerID string) (
*ContainerStat, error) {

stats, err := c.runtimeService.ContainerStats(ctx, containerID)
if err != nil {
return nil, fmt.Errorf("error getting container stats: %w", err)
}

return &ContainerStat{
Timestamp: time.Unix(stats.Memory.Timestamp, 0),
Id: stats.Attributes.Id,
Name: stats.Attributes.Metadata.Name,
Mem: stats.Memory.UsageBytes.GetValue(),
Cpu: stats.Cpu.UsageCoreNanoSeconds.GetValue(),
}, nil
}

func (c *criExecutor) GetContainerIP(name string) (string, error) {
container, err := c.getContainer(name)
if err != nil {
Expand Down
33 changes: 31 additions & 2 deletions integration-tests/pkg/executor/executor_docker_api.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package executor
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"strings"
Expand Down Expand Up @@ -91,6 +92,34 @@ func (d *dockerAPIExecutor) GetContainerHealthCheck(containerID string) (string,
return strings.Join(inspectResp.Config.Healthcheck.Test, " "), nil
}

func (d *dockerAPIExecutor) GetContainerStats(ctx context.Context, containerID string) (
*ContainerStat, error) {

// ContainerStatsOneShot add "one-shot" parameter to the API query, which
// instructs the server to get a single stat, rather than waiting for more.
statsResp, err := d.client.ContainerStatsOneShot(ctx, containerID)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this ContainerStatsOneShot return a snapshot of the usage at the time of the call or does it do some sort of aggregation over time? Is the behavior consistent with the CRI counterpart?

I tried to figure it out from reading the docs but I couldn't find an answer, I can dig a little more later.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation is sparse, but I guess it's the same one-shot as in the API, which just eliminates waiting and returns empty per_cpu:
https://docs.docker.com/reference/api/engine/version/v1.45/#tag/Container/operation/ContainerStats

No aggregation is happening on the client either:
https://github.com/moby/moby/blob/master/client/container_stats.go

if err != nil {
return nil, fmt.Errorf("error getting container stats: %w", err)
}

var stats container.StatsResponse
decoder := json.NewDecoder(statsResp.Body)

if err := decoder.Decode(&stats); err == io.EOF {
return nil, nil
Comment on lines +108 to +109
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this situation that the response was empty when we queried for the stats? Do we know what could cause this? Should we return some sort of error?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will bubble up an warning "no stats" later. A reason could be e.g. an empty response body, if the stats api was called before Collector container was started.

} else if err != nil {
return nil, fmt.Errorf("error decoding container stats: %w", err)
}

return &ContainerStat{
Timestamp: stats.Read,
Id: stats.ID,
Name: stats.Name,
Mem: stats.MemoryStats.Usage,
Cpu: stats.CPUStats.CPUUsage.TotalUsage,
}, nil
}

func (d *dockerAPIExecutor) StartContainer(startConfig config.ContainerStartConfig) (string, error) {
ctx := context.Background()
var binds []string
Expand Down Expand Up @@ -159,7 +188,7 @@ func (d *dockerAPIExecutor) StartContainer(startConfig config.ContainerStartConf

func (d *dockerAPIExecutor) ExecContainer(opts *ExecOptions) (string, error) {
ctx := context.Background()
execConfig := types.ExecConfig{
execConfig := container.ExecOptions{
AttachStdout: true,
AttachStderr: true,
Cmd: opts.Command,
Expand All @@ -170,7 +199,7 @@ func (d *dockerAPIExecutor) ExecContainer(opts *ExecOptions) (string, error) {
return "", fmt.Errorf("error creating Exec: %w", err)
}

execStartCheck := types.ExecStartCheck{Detach: false, Tty: false}
execStartCheck := container.ExecAttachOptions{Detach: false, Tty: false}
attachResp, err := d.client.ContainerExecAttach(ctx, resp.ID, execStartCheck)
if err != nil {
return "", fmt.Errorf("error attaching to Exec: %w", err)
Expand Down
1 change: 0 additions & 1 deletion integration-tests/suites/async_connections.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@ type AsyncConnectionTestSuite struct {
*/
func (s *AsyncConnectionTestSuite) SetupSuite() {
s.RegisterCleanup("server", "client")
s.StartContainerStats()

collectorOptions := collector.StartupOptions{
Env: map[string]string{
Expand Down
Loading
Loading