Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PMM-12375 add database version to services #2438

Closed
wants to merge 42 commits into from
Closed
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
bd793d1
PMM-12375 address a deprecation warning
Aug 25, 2023
3e167ad
PMM-12375 add a Version field to CheckConnectionResponse.Stats
Aug 25, 2023
246f815
PMM-12375 extract version from different DBs
Aug 25, 2023
7eb66f3
Merge branch 'main' into PMM-12375-add-database-version-to-services
Aug 25, 2023
296d5cf
PMM-12375 revert Makefile
Aug 25, 2023
57ec9aa
Bump github.com/prometheus/alertmanager from 0.25.0 to 0.26.0 (#2436)
dependabot[bot] Aug 28, 2023
c45cc58
PMM-12375 add a column to store the version
Aug 28, 2023
f98ce11
Merge branch 'main' into PMM-12375-add-database-version-to-services
Aug 28, 2023
99b0532
PMM-12375 save version to DB
Aug 29, 2023
9d1ebec
PMM-12375 add version detection for ProxySQL
Aug 30, 2023
7348e67
PMM-12375 choose the dabase first (mysql)
Aug 30, 2023
f58949e
PMM-12375 fix syntax
Aug 30, 2023
7865feb
PMM-12375 use Exec to select a database
Aug 30, 2023
fb745dd
PMM-12375 use a different database
Aug 30, 2023
911b14d
PMM-12375 use performance_schema for higher versions
Aug 30, 2023
64715ae
PMM-12375 use a different syntax for checking the version
Aug 30, 2023
7c4b4b8
PMM-12375 quote the dashed variable
Aug 30, 2023
d8f36f4
PMM-12375 remopve --short as deprecated
Aug 31, 2023
9d7ed72
Merge branch 'main' into PMM-12375-add-database-version-to-services
Sep 7, 2023
ccb01e4
Merge branch 'main' into PMM-12375-add-database-version-to-services
Sep 11, 2023
5de5333
PMM-12375 add a dedicated message
Sep 4, 2023
ebfbb9c
PMM-12375 refactor to use a separate method for version and tableCount
Sep 12, 2023
f3bf65c
Bump github.com/grafana/grafana-api-golang-client from 0.23.0 to 0.24…
dependabot[bot] Sep 12, 2023
18806c9
PMM-12375 re-generate protobufs
Sep 12, 2023
8a7b12f
Bump @typescript-eslint/parser from 6.6.0 to 6.7.0 in /cli-tests (#2476)
dependabot[bot] Sep 12, 2023
3163ec5
Bump @typescript-eslint/eslint-plugin from 6.6.0 to 6.7.0 in /cli-tes…
dependabot[bot] Sep 12, 2023
800d582
Bump docker/login-action from 2 to 3 (#2480)
dependabot[bot] Sep 13, 2023
f2b8b95
Bump docker/setup-buildx-action from 2 to 3 (#2479)
dependabot[bot] Sep 13, 2023
fe495cc
PMM-11670 Set the cluster label if empty (#2459)
Sep 13, 2023
8008009
PMM-12376 single inventory add endpoint (#2422)
Sep 13, 2023
3e453b1
PMM-12448 Fix bug (#2437)
artemgavrilov Sep 13, 2023
002fdbb
Bump docker/build-push-action from 4 to 5 (#2481)
dependabot[bot] Sep 13, 2023
0697b04
PMM-12457 add node_name to agent metrics (#2474)
Sep 13, 2023
b2e10bc
PMM-12488 update VictoriaMetrics to v1.93.4 (#2483)
Sep 15, 2023
0b52509
PMM-9769 update all components (#2478)
talhabinrizwan Sep 15, 2023
06d99a8
PMM-12375 add version prop to ProxySQL service
Sep 15, 2023
f1a259a
PMM-12375 move GetInfoFromService to serviceInfoBroker
Sep 16, 2023
d9cc8e4
Merge branch 'main' into PMM-12375-add-database-version-to-services
Sep 16, 2023
8595ae6
PMM-12375 fix service_test.go by adding params
Sep 16, 2023
e621aca
PMM-12375 remove Stats from agent.proto
Sep 16, 2023
f5ad29e
PMM-12375 fix the linter error
Sep 16, 2023
c89d862
PMM-12375 clean up serviceInfoBroker component
Sep 16, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .golangci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,8 @@ linters:
- nlreturn # too annoying
- nosnakecase # deprecated
- scopelint # too many false positives
- structcheck # replaced by unused
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will silence the linter's deprecation warnings.

- varcheck # replaced by unused
- varnamelen # useless
- wrapcheck # we do not use wrapping everywhere
- wsl # too annoying
Expand Down
2 changes: 1 addition & 1 deletion admin/commands/management/add.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ import (

// AddCommand is used by Kong for CLI flags and commands.
type AddCommand struct {
External AddExternalCommand `cmd:"" help:"Add External source of data (like a custom exporter running on a port) to the monitoring"`
External AddExternalCommand `cmd:"" help:"Add External source of data (like a custom exporter running on a port) to monitoring"`
ExternalServerless AddExternalServerlessCommand `cmd:"" help:"Add External Service on Remote node to monitoring"`
HAProxy AddHAProxyCommand `cmd:"" name:"haproxy" help:"Add HAProxy to monitoring"`
MongoDB AddMongoDBCommand `cmd:"" name:"mongodb" help:"Add MongoDB to monitoring"`
Expand Down
7 changes: 6 additions & 1 deletion agent/client/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ type Client struct {
supervisor supervisor
connectionChecker connectionChecker
softwareVersioner softwareVersioner
serviceInfoBroker serviceInfoBroker

l *logrus.Entry
backoff *backoff.Backoff
Expand All @@ -89,12 +90,13 @@ type Client struct {
// New creates new client.
//
// Caller should call Run.
func New(cfg configGetter, supervisor supervisor, r *runner.Runner, connectionChecker connectionChecker, sv softwareVersioner, cus *connectionuptime.Service, logStore *tailog.Store) *Client { //nolint:lll
func New(cfg configGetter, supervisor supervisor, r *runner.Runner, connectionChecker connectionChecker, sv softwareVersioner, sib serviceInfoBroker, cus *connectionuptime.Service, logStore *tailog.Store) *Client { //nolint:lll
return &Client{
cfg: cfg,
supervisor: supervisor,
connectionChecker: connectionChecker,
softwareVersioner: sv,
serviceInfoBroker: sib,
l: logrus.WithField("component", "client"),
backoff: backoff.New(backoffMinDelay, backoffMaxDelay),
dialTimeout: dialTimeout,
Expand Down Expand Up @@ -376,6 +378,9 @@ loop:
case *agentpb.CheckConnectionRequest:
responsePayload = c.connectionChecker.Check(ctx, p, req.ID)

case *agentpb.ServiceInfoRequest:
responsePayload = c.serviceInfoBroker.GetInfoFromService(ctx, p, req.ID)

case *agentpb.StartJobRequest:
var resp agentpb.StartJobResponse
if err := c.handleStartJobRequest(p); err != nil {
Expand Down
12 changes: 6 additions & 6 deletions agent/client/client_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ func TestClient(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())

cfgStorage := config.NewStorage(&config.Config{})
client := New(cfgStorage, nil, nil, nil, nil, nil, nil)
client := New(cfgStorage, nil, nil, nil, nil, nil, nil, nil)
cancel()
err := client.Run(ctx)
assert.EqualError(t, err, "missing PMM Server address: context canceled")
Expand All @@ -98,7 +98,7 @@ func TestClient(t *testing.T) {
Address: "127.0.0.1:1",
},
})
client := New(cfgStorage, nil, nil, nil, nil, nil, nil)
client := New(cfgStorage, nil, nil, nil, nil, nil, nil, nil)
cancel()
err := client.Run(ctx)
assert.EqualError(t, err, "missing Agent ID: context canceled")
Expand All @@ -115,7 +115,7 @@ func TestClient(t *testing.T) {
Address: "127.0.0.1:1",
},
})
client := New(cfgStorage, nil, nil, nil, nil, connectionuptime.NewService(time.Hour), nil)
client := New(cfgStorage, nil, nil, nil, nil, nil, connectionuptime.NewService(time.Hour), nil)
err := client.Run(ctx)
assert.EqualError(t, err, "failed to dial: context deadline exceeded")
})
Expand Down Expand Up @@ -164,7 +164,7 @@ func TestClient(t *testing.T) {
s.On("ClearChangesChannel").Return()

r := runner.New(cfgStorage.Get().RunnerCapacity)
client := New(cfgStorage, &s, r, nil, nil, connectionuptime.NewService(time.Hour), nil)
client := New(cfgStorage, &s, r, nil, nil, nil, connectionuptime.NewService(time.Hour), nil)
err := client.Run(context.Background())
assert.NoError(t, err)
assert.Equal(t, serverMD, client.GetServerConnectMetadata())
Expand Down Expand Up @@ -192,7 +192,7 @@ func TestClient(t *testing.T) {
},
})

client := New(cfgStorage, nil, nil, nil, nil, connectionuptime.NewService(time.Hour), nil)
client := New(cfgStorage, nil, nil, nil, nil, nil, connectionuptime.NewService(time.Hour), nil)
client.dialTimeout = 100 * time.Millisecond
err := client.Run(ctx)
assert.EqualError(t, err, "failed to get server metadata: rpc error: code = Canceled desc = context canceled", "%+v", err)
Expand Down Expand Up @@ -282,7 +282,7 @@ func TestUnexpectedActionType(t *testing.T) {
s.On("ClearChangesChannel").Return()

r := runner.New(cfgStorage.Get().RunnerCapacity)
client := New(cfgStorage, s, r, nil, nil, connectionuptime.NewService(time.Hour), nil)
client := New(cfgStorage, s, r, nil, nil, nil, connectionuptime.NewService(time.Hour), nil)
err := client.Run(context.Background())
assert.NoError(t, err)
assert.Equal(t, serverMD, client.GetServerConnectMetadata())
Expand Down
6 changes: 6 additions & 0 deletions agent/client/deps.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import (
)

//go:generate ../../bin/mockery --name=connectionChecker --case=snake --inpackage --testonly
//go:generate ../../bin/mockery --name=serviceInfoBroker --case=snake --inpackage --testonly
//go:generate ../../bin/mockery --name=supervisor --case=snake --inpackage --testonly

// connectionChecker is a subset of methods of connectionchecker.ConnectionChecker used by this package.
Expand All @@ -32,6 +33,11 @@ type connectionChecker interface {
Check(ctx context.Context, req *agentpb.CheckConnectionRequest, id uint32) *agentpb.CheckConnectionResponse
}

// serviceInfoBroker is a subset of methods of serviceinfobroker.ServiceInfoBroker used by this package.
type serviceInfoBroker interface {
GetInfoFromService(ctx context.Context, req *agentpb.ServiceInfoRequest, id uint32) *agentpb.ServiceInfoResponse
}

// softwareVersioner is a subset of methods of version.Versioner used by this package.
type softwareVersioner interface {
MySQLdVersion() (string, error)
Expand Down
47 changes: 47 additions & 0 deletions agent/client/mock_service_info_broker_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 3 additions & 1 deletion agent/commands/run.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ import (
"github.com/percona/pmm/agent/connectionchecker"
"github.com/percona/pmm/agent/connectionuptime"
"github.com/percona/pmm/agent/runner"
"github.com/percona/pmm/agent/serviceinfobroker"
"github.com/percona/pmm/agent/tailog"
"github.com/percona/pmm/agent/versioner"
"github.com/percona/pmm/api/inventorypb"
Expand Down Expand Up @@ -76,8 +77,9 @@ func Run() {
v := versioner.New(&versioner.RealExecFunctions{})
supervisor := supervisor.NewSupervisor(ctx, v, configStorage)
connectionChecker := connectionchecker.New(configStorage)
serviceInfoBroker := serviceinfobroker.New(configStorage)
r := runner.New(cfg.RunnerCapacity)
client := client.New(configStorage, supervisor, r, connectionChecker, v, connectionUptimeService, logStore)
client := client.New(configStorage, supervisor, r, connectionChecker, v, serviceInfoBroker, connectionUptimeService, logStore)
localServer := agentlocal.NewServer(configStorage, supervisor, client, configFilepath, logStore)

var wg sync.WaitGroup
Expand Down
17 changes: 0 additions & 17 deletions agent/connectionchecker/connection_checker.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ import (
"database/sql"
"fmt"
"io"
"math"
"net/http"
"path/filepath"
"strconv"
Expand Down Expand Up @@ -140,22 +139,6 @@ func (cc *ConnectionChecker) checkMySQLConnection(ctx context.Context, dsn strin
} else {
res.Error = err.Error()
}
return &res
}

var count uint64
if err = db.QueryRowContext(ctx, "SELECT /* agent='connectionchecker' */ COUNT(*) FROM information_schema.tables").Scan(&count); err != nil {
res.Error = err.Error()
return &res
}

tableCount := int32(count)
if count > math.MaxInt32 {
tableCount = math.MaxInt32
}

res.Stats = &agentpb.CheckConnectionResponse_Stats{
TableCount: tableCount,
}

return &res
Expand Down
Loading
Loading