Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PMM-11423: support for updating a service #1590

Merged
merged 27 commits into from
Aug 28, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
367690c
PMM-11423: support for updating a service
michal-kralik Jan 19, 2023
7b5f0cb
Refactor update of labels to a model
michal-kralik Jan 23, 2023
27b2d32
Merge branch 'main' into PMM-11423-update-standard-labels
michal-kralik Jan 23, 2023
3b9d97b
Merge branch 'main' into PMM-11423-update-standard-labels
denisok Jan 30, 2023
7d1c3c0
Rename UpdateService to ChangeService
michal-kralik Jan 30, 2023
bb07ef5
Change API comments
michal-kralik Jan 31, 2023
f9d0806
Split business and model logic
michal-kralik Jan 31, 2023
9b512f6
Remove unused code
michal-kralik Feb 6, 2023
43040dc
Merge branch 'main' into PMM-11423-update-standard-labels
michal-kralik Feb 6, 2023
a9716c8
Merge branch 'main' into PMM-11423-update-standard-labels
PavelKhripkov Feb 6, 2023
4335370
Add tests for changing standard labels
michal-kralik Feb 7, 2023
c4bcfbd
Merge branch 'main' into PMM-11423-update-standard-labels
michal-kralik Feb 7, 2023
8b52e00
PMM-11546 remove scheduled backups when cluster changed (#1662)
PavelKhripkov Feb 7, 2023
1419049
Merge branch 'main' into PMM-11423-update-standard-labels
michal-kralik Feb 8, 2023
0004743
Merge branch 'main' into PMM-11423-update-standard-labels
michal-kralik Feb 16, 2023
c7f5282
Merge branch 'main' into PMM-11423-update-standard-labels
michal-kralik Feb 22, 2023
d675dcb
Merge branch 'main' of github.com:percona/pmm into PMM-11423-update-s…
Jul 7, 2023
bb89338
PMM-11423 fix conflicts
Jul 7, 2023
410d76e
PMM-11423 fix linter errors
Jul 7, 2023
cf0e707
Merge branch 'main' into PMM-11423-update-standard-labels
Jul 24, 2023
a4acb17
Merge branch 'main' into PMM-11423-update-standard-labels
Jul 31, 2023
85d660f
Merge branch 'main' into PMM-11423-update-standard-labels
matejkubinec Aug 7, 2023
d2c1844
Merge branch 'main' into PMM-11423-update-standard-labels
matejkubinec Aug 16, 2023
1d07cb9
Merge branch 'main' into PMM-11423-update-standard-labels
matejkubinec Aug 23, 2023
e28b056
Merge branch 'main' into PMM-11423-update-standard-labels
matejkubinec Aug 25, 2023
969c028
Merge branch 'main' into PMM-11423-update-standard-labels
matejkubinec Aug 25, 2023
b43a5f6
Merge branch 'main' into PMM-11423-update-standard-labels
matejkubinec Aug 28, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion api/inventorypb/json/client/services/services_client.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion api/inventorypb/json/inventorypb.json
Original file line number Diff line number Diff line change
Expand Up @@ -11533,7 +11533,7 @@
},
"/v1/inventory/Services/Change": {
"post": {
"description": "Changes service configuration.",
"description": "Changes service configuration. If a new cluster label is specified, it removes all backup/restore tasks scheduled for the related services. Fails if there are running backup/restore tasks.",
"tags": [
"Services"
],
Expand Down
46 changes: 28 additions & 18 deletions api/inventorypb/services.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion api/inventorypb/services.proto
Original file line number Diff line number Diff line change
Expand Up @@ -624,7 +624,7 @@ service Services {
};
option (grpc.gateway.protoc_gen_openapiv2.options.openapiv2_operation) = {
summary: "Change service"
description: "Changes service configuration."
description: "Changes service configuration. If a new cluster label is specified, it removes all backup/restore tasks scheduled for the related services. Fails if there are running backup/restore tasks."
};
}
}
2 changes: 1 addition & 1 deletion api/swagger/swagger-dev.json
Original file line number Diff line number Diff line change
Expand Up @@ -15542,7 +15542,7 @@
},
"/v1/inventory/Services/Change": {
"post": {
"description": "Changes service configuration.",
"description": "Changes service configuration. If a new cluster label is specified, it removes all backup/restore tasks scheduled for the related services. Fails if there are running backup/restore tasks.",
"tags": [
"Services"
],
Expand Down
2 changes: 1 addition & 1 deletion api/swagger/swagger.json
Original file line number Diff line number Diff line change
Expand Up @@ -12830,7 +12830,7 @@
},
"/v1/inventory/Services/Change": {
"post": {
"description": "Changes service configuration.",
"description": "Changes service configuration. If a new cluster label is specified, it removes all backup/restore tasks scheduled for the related services. Fails if there are running backup/restore tasks.",
"tags": [
"Services"
],
Expand Down
14 changes: 10 additions & 4 deletions managed/cmd/pmm-managed/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ import (
"github.com/percona/pmm/managed/services/management"
"github.com/percona/pmm/managed/services/management/alerting"
managementbackup "github.com/percona/pmm/managed/services/management/backup"
"github.com/percona/pmm/managed/services/management/common"
managementdbaas "github.com/percona/pmm/managed/services/management/dbaas"
managementgrpc "github.com/percona/pmm/managed/services/management/grpc"
"github.com/percona/pmm/managed/services/management/ia"
Expand Down Expand Up @@ -256,8 +257,13 @@ func runGRPCServer(ctx context.Context, deps *gRPCServerDeps, features gRPCServe
deps.db, deps.agentsRegistry, deps.agentsStateUpdater,
deps.vmdb, deps.connectionCheck, deps.agentService)

mgmtBackupsService := managementbackup.NewBackupsService(deps.db, deps.backupService, deps.compatibilityService, deps.schedulerService)
mgmtArtifactsService := managementbackup.NewArtifactsService(deps.db, deps.backupRemovalService, deps.pitrTimerangeService)
mgmtRestoreHistoryService := managementbackup.NewRestoreHistoryService(deps.db)
mgmtServices := common.MgmtServices{BackupsService: mgmtBackupsService, ArtifactsService: mgmtArtifactsService, RestoreHistoryService: mgmtRestoreHistoryService}

inventorypb.RegisterNodesServer(gRPCServer, inventorygrpc.NewNodesServer(nodesSvc))
inventorypb.RegisterServicesServer(gRPCServer, inventorygrpc.NewServicesServer(servicesSvc))
inventorypb.RegisterServicesServer(gRPCServer, inventorygrpc.NewServicesServer(servicesSvc, mgmtServices))
inventorypb.RegisterAgentsServer(gRPCServer, inventorygrpc.NewAgentsServer(agentsSvc))

nodeSvc := management.NewNodeService(deps.db, deps.grafanaClient)
Expand Down Expand Up @@ -289,10 +295,10 @@ func runGRPCServer(ctx context.Context, deps *gRPCServerDeps, features gRPCServe
iav1beta1.RegisterAlertsServer(gRPCServer, deps.alertsService)
alertingpb.RegisterAlertingServer(gRPCServer, deps.templatesService)

backuppb.RegisterBackupsServer(gRPCServer, managementbackup.NewBackupsService(deps.db, deps.backupService, deps.compatibilityService, deps.schedulerService))
backuppb.RegisterBackupsServer(gRPCServer, mgmtBackupsService)
backuppb.RegisterLocationsServer(gRPCServer, managementbackup.NewLocationsService(deps.db, deps.minioClient))
backuppb.RegisterArtifactsServer(gRPCServer, managementbackup.NewArtifactsService(deps.db, deps.backupRemovalService, deps.pitrTimerangeService))
backuppb.RegisterRestoreHistoryServer(gRPCServer, managementbackup.NewRestoreHistoryService(deps.db))
backuppb.RegisterArtifactsServer(gRPCServer, mgmtArtifactsService)
backuppb.RegisterRestoreHistoryServer(gRPCServer, mgmtRestoreHistoryService)

k8sServer := managementdbaas.NewKubernetesServer(deps.db, deps.dbaasClient, deps.kubernetesClient, deps.versionServiceClient, deps.grafanaClient)
deps.dbaasInitializer.RegisterKubernetesServer(k8sServer)
Expand Down
28 changes: 27 additions & 1 deletion managed/cmd/pmm-managed/main_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ func TestPackages(t *testing.T) {
func TestImports(t *testing.T) {
type constraint struct {
blacklistPrefixes []string
whitelistPrefixes []string
}

constraints := make(map[string]constraint)
Expand Down Expand Up @@ -76,7 +77,6 @@ func TestImports(t *testing.T) {
// those services should be independent too, but has some common code
// as converters, errors, ...
"github.com/percona/pmm/managed/services/grafana",
"github.com/percona/pmm/managed/services/inventory",
"github.com/percona/pmm/managed/services/management",
"github.com/percona/pmm/managed/services/server",
"github.com/percona/pmm/managed/services/checks",
Expand All @@ -89,6 +89,20 @@ func TestImports(t *testing.T) {
}
}

for _, service := range []string{
// TODO come up with a new code structure that allows cross-service communication without the need to do tricks.
"github.com/percona/pmm/managed/services/inventory",
} {
constraints[service] = constraint{
blacklistPrefixes: []string{
"github.com/percona/pmm/managed/services/",
},
whitelistPrefixes: []string{
"github.com/percona/pmm/managed/services/management/common",
},
}
}

// validators should not import gRPC stack, including errors
constraints["github.com/percona/pmm/managed/utils/validators"] = constraint{
blacklistPrefixes: []string{
Expand Down Expand Up @@ -127,6 +141,18 @@ func TestImports(t *testing.T) {
continue
}

// check allowlist
var allow bool
for _, a := range c.whitelistPrefixes {
if strings.HasPrefix(i, a) {
allow = true
break
}
}
if allow {
continue
}

// check blacklist
if strings.HasPrefix(i, b) {
t.Errorf("Package %q should not import package %q (blacklisted by %q).", path, i, b)
Expand Down
31 changes: 27 additions & 4 deletions managed/services/inventory/grpc/services_server.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,21 +20,29 @@ import (
"fmt"

"github.com/AlekSi/pointer"
"github.com/pkg/errors"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"

"github.com/percona/pmm/api/inventorypb"
"github.com/percona/pmm/managed/models"
"github.com/percona/pmm/managed/services/inventory"
"github.com/percona/pmm/managed/services/management/common"
)

type servicesServer struct {
s *inventory.ServicesService
s *inventory.ServicesService
mgmtServices common.MgmtServices

inventorypb.UnimplementedServicesServer
}

// NewServicesServer returns Inventory API handler for managing Services.
func NewServicesServer(s *inventory.ServicesService) inventorypb.ServicesServer {
return &servicesServer{s: s}
func NewServicesServer(s *inventory.ServicesService, mgmtServices common.MgmtServices) inventorypb.ServicesServer {
ademidoff marked this conversation as resolved.
Show resolved Hide resolved
return &servicesServer{
s: s,
mgmtServices: mgmtServices,
}
}

var serviceTypes = map[inventorypb.ServiceType]models.ServiceType{
Expand Down Expand Up @@ -281,11 +289,26 @@ func (s *servicesServer) RemoveCustomLabels(ctx context.Context, req *inventoryp

// ChangeService changes service configuration.
func (s *servicesServer) ChangeService(ctx context.Context, req *inventorypb.ChangeServiceRequest) (*inventorypb.ChangeServiceResponse, error) {
return s.s.ChangeService(ctx, &models.ChangeStandardLabelsParams{
err := s.s.ChangeService(ctx, s.mgmtServices, &models.ChangeStandardLabelsParams{
ServiceID: req.ServiceId,
Cluster: req.Cluster,
Environment: req.Environment,
ReplicationSet: req.ReplicationSet,
ExternalGroup: req.ExternalGroup,
})
if err != nil {
return nil, toAPIError(err)
}

return &inventorypb.ChangeServiceResponse{}, nil
}

// toAPIError converts GO errors into API-level errors.
func toAPIError(err error) error {
switch {
case errors.Is(err, common.ErrClusterLocked):
return status.Error(codes.FailedPrecondition, err.Error())
default:
return err
}
}
13 changes: 9 additions & 4 deletions managed/services/inventory/services.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import (
"github.com/percona/pmm/api/inventorypb"
"github.com/percona/pmm/managed/models"
"github.com/percona/pmm/managed/services"
"github.com/percona/pmm/managed/services/management/common"
)

// ServicesService works with inventory API Services.
Expand Down Expand Up @@ -434,7 +435,11 @@ func (ss *ServicesService) RemoveCustomLabels(ctx context.Context, req *inventor
}

// ChangeService changes service configuration.
func (ss *ServicesService) ChangeService(ctx context.Context, params *models.ChangeStandardLabelsParams) (*inventorypb.ChangeServiceResponse, error) {
func (ss *ServicesService) ChangeService(ctx context.Context, mgmtServices common.MgmtServices, params *models.ChangeStandardLabelsParams) error {
if err := mgmtServices.RemoveScheduledTasks(ctx, ss.db, params); err != nil {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PavelKhripkov would it make more sense to execute this in a transaction along with the labels change?
In case the labels change fail, we already removed scheduled tasks and there's no way back.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For sure, but the calls made on api-level that doesn't allow to rollback. To implement the same on a deeper level, we need to duplicate alot of code lines. And without proper testing It won't be reliable.

return err
}

errTx := ss.db.InTransactionContext(ctx, nil, func(tx *reform.TX) error {
err := models.ChangeStandardLabels(tx.Querier, params.ServiceID, models.ServiceStandardLabelsParams{
Cluster: params.Cluster,
Expand All @@ -445,14 +450,14 @@ func (ss *ServicesService) ChangeService(ctx context.Context, params *models.Cha
return err
})
if errTx != nil {
return nil, errTx
return errTx
}

if err := ss.updateScrapeConfig(ctx, params.ServiceID); err != nil {
return nil, err
return err
}

return &inventorypb.ChangeServiceResponse{}, nil
return nil
}

func (ss *ServicesService) updateScrapeConfig(ctx context.Context, serviceID string) error {
Expand Down
Loading