Skip to content

Commit aa41e59

Browse files
authored
Bug fix: reconcile a "create" change with a new extension map. (#574)
* Bug fix, reconcile a create change with a new extensions map to avoid minifying the source service extensions. * Added tests to cover the bug fix including reporting to include the target environment during a failure.
1 parent 6e24fa8 commit aa41e59

File tree

9 files changed

+111
-39
lines changed

9 files changed

+111
-39
lines changed

go.sum

+3-10
Original file line numberDiff line numberDiff line change
@@ -541,8 +541,6 @@ github.com/go-playground/locales v0.13.0 h1:HyWk6mgj5qFqCT5fjGBuRArbVDfE4hi8+e8c
541541
github.com/go-playground/locales v0.13.0/go.mod h1:taPMhCMXrRLJO55olJkUXHZBHCxTMfnGwq/HNwmWNS8=
542542
github.com/go-playground/universal-translator v0.17.0 h1:icxd5fm+REJzpZx7ZfpaD876Lmtgy7VtROAbHHXk8no=
543543
github.com/go-playground/universal-translator v0.17.0/go.mod h1:UkSxE5sNxxRwHyU+Scu5vgOQjsIJAF8j9muTVoKLVtA=
544-
github.com/go-playground/validator/v10 v10.4.1 h1:pH2c5ADXtd66mxoE0Zm9SUhxE20r7aM3F26W0hOn+GE=
545-
github.com/go-playground/validator/v10 v10.4.1/go.mod h1:nlOn6nFhuKACm19sB/8EGNn9GlaMV7XkbRSipzJ0Ii4=
546544
github.com/go-playground/validator/v10 v10.6.1 h1:W6TRDXt4WcWp4c4nf/G+6BkGdhiIo0k417gfr+V6u4I=
547545
github.com/go-playground/validator/v10 v10.6.1/go.mod h1:xm76BBt941f7yWdGnI2DVPFFg1UK3YY04qifoXU3lOk=
548546
github.com/go-sql-driver/mysql v1.4.0/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w=
@@ -613,9 +611,9 @@ github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:W
613611
github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
614612
github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=
615613
github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
616-
github.com/golang/protobuf v1.4.3 h1:JjCZWpVbqXDqFVmTfYWEVTMIYrL/NPdPSCHPJ0T/raM=
617614
github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
618615
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
616+
github.com/golang/protobuf v1.5.2 h1:ROPKBNFfQgOUMifHyP+KYbvpjbdoFNs+aK7DXlji0Tw=
619617
github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
620618
github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
621619
github.com/golangci/check v0.0.0-20180506172741-cfe4005ccda2/go.mod h1:k9Qvh+8juN+UKMCS/3jFtGICgW8O96FVaZsaxdzDkR4=
@@ -1032,8 +1030,7 @@ github.com/onsi/ginkgo v1.10.3/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+
10321030
github.com/onsi/ginkgo v1.11.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
10331031
github.com/onsi/ginkgo v1.12.0/go.mod h1:oUhWkIvk5aDxtKvDDuw8gItl8pKl42LzjC9KZE0HfGg=
10341032
github.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk=
1035-
github.com/onsi/ginkgo v1.15.2 h1:l77YT15o814C2qVL47NOyjV/6RbaP7kKdrvZnxQ3Org=
1036-
github.com/onsi/ginkgo v1.15.2/go.mod h1:Dd6YFfwBW84ETqqtL0CPyPXillHgY6XhQH3uuCCTr/o=
1033+
github.com/onsi/ginkgo v1.16.2/go.mod h1:CObGmKUOKaSC0RjmoAK7tKyn4Azo5P2IWuoMnvwxz1E=
10371034
github.com/onsi/ginkgo v1.16.4 h1:29JGrr5oVBm5ulCWet69zQkzWipVXIol6ygQUe/EzNc=
10381035
github.com/onsi/ginkgo v1.16.4/go.mod h1:dX+/inL/fNMqNlz0e9LfyB9TswhZpCVdJM/Z6Vvnwo0=
10391036
github.com/onsi/gomega v0.0.0-20170829124025-dcabb60a477c/go.mod h1:C1qb7wdrVGGVU+Z6iS04AVkA3Q65CEZX59MT0QO5uiA=
@@ -1046,8 +1043,6 @@ github.com/onsi/gomega v1.9.0/go.mod h1:Ho0h+IUsWyvy1OpqCwxlQ/21gkhVunqlU8fDGcoT
10461043
github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo=
10471044
github.com/onsi/gomega v1.10.3/go.mod h1:V9xEwhxec5O8UDM77eCW8vLymOMltsqPVYWrpDsH8xc=
10481045
github.com/onsi/gomega v1.10.4/go.mod h1:g/HbgYopi++010VEqkFgJHKC09uJiW9UkXvMUuKHUCQ=
1049-
github.com/onsi/gomega v1.11.0 h1:+CqWgvj0OZycCaqclBD1pxKHAU+tOkHmQIWvDHq2aug=
1050-
github.com/onsi/gomega v1.11.0/go.mod h1:azGKhqFUon9Vuj0YmTfLSmx0FUwqXYSTl5re8lQLTUg=
10511046
github.com/onsi/gomega v1.13.0 h1:7lLHu94wT9Ij0o6EWWclhu0aOh32VxhkwEJvzuWPeak=
10521047
github.com/onsi/gomega v1.13.0/go.mod h1:lRk9szgn8TxENtWd0Tp4c3wjlRfMTMH27I+3Je41yGY=
10531048
github.com/op/go-logging v0.0.0-20160315200505-970db520ece7/go.mod h1:HzydrMdWErDVzsI23lYNej1Htcns9BCg93Dk0bBINWk=
@@ -1536,7 +1531,6 @@ golang.org/x/net v0.0.0-20201031054903-ff519b6c9102/go.mod h1:sp8m0HH+o8qH0wwXwY
15361531
golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
15371532
golang.org/x/net v0.0.0-20201202161906-c7110b5ffcbb/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
15381533
golang.org/x/net v0.0.0-20210119194325-5f4716e94777/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
1539-
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110 h1:qWPm9rbaAMKs8Bq/9LRpbMqxWRVUAQwMI9fVrssnTfw=
15401534
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
15411535
golang.org/x/net v0.0.0-20210428140749-89ef3d95e781 h1:DzZ89McO9/gWPsQXS/FVKAlG02ZjaQ6AlZRBimEYOd0=
15421536
golang.org/x/net v0.0.0-20210428140749-89ef3d95e781/go.mod h1:OJAsFXCWl8Ukc7SiCT/9KSuxbyM7479/AVlXFRxuMCk=
@@ -1669,7 +1663,6 @@ golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3
16691663
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
16701664
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
16711665
golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
1672-
golang.org/x/text v0.3.5 h1:i6eZZ+zk0SOf0xgBpEpPD18qWcJda6q1sxt3S0kzyUQ=
16731666
golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
16741667
golang.org/x/text v0.3.6 h1:aRYxNxv6iGQlyVaZmk6ZgYEDa+Jg18DxebPSrd6bg1M=
16751668
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
@@ -1906,9 +1899,9 @@ google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2
19061899
google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
19071900
google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
19081901
google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4=
1909-
google.golang.org/protobuf v1.25.0 h1:Ejskq+SyPohKW+1uil0JJMtmHCgJPJ/qWTxr8qp+R4c=
19101902
google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
19111903
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
1904+
google.golang.org/protobuf v1.26.0 h1:bxAC2xTBsZGibn2RTntX0oH50xLsqy1OxA9tTL3p/lk=
19121905
google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
19131906
gopkg.in/airbrake/gobrake.v2 v2.0.9/go.mod h1:/h5ZAUhDkGaJfjzjKLSjv6zCL6O0LLBxU4K+aSYdM/U=
19141907
gopkg.in/alecthomas/kingpin.v2 v2.2.6 h1:jMFz6MfLP0/4fUyZle81rXUoxOBFi19VUFKVDOQfozc=

pkg/kev/changeset.go

+6-4
Original file line numberDiff line numberDiff line change
@@ -101,8 +101,9 @@ func (chg change) patchService(override *composeOverride) (string, error) {
101101
return "", err
102102
}
103103

104-
newValue.Extensions[config.K8SExtensionKey] = minified
105-
override.Services = append(override.Services, newValue)
104+
override.Services = append(override.Services, ServiceConfig{Name: newValue.Name, Extensions: map[string]interface{}{
105+
config.K8SExtensionKey: minified,
106+
}})
106107

107108
msg := fmt.Sprintf("added service: %s", newValue.Name)
108109
log.Debugf(msg)
@@ -154,8 +155,9 @@ func (chg change) patchVolume(override *composeOverride) (string, error) {
154155
return "", err
155156
}
156157

157-
newValue.Extensions[config.K8SExtensionKey] = minified
158-
override.Volumes[chg.Index.(string)] = newValue
158+
override.Volumes[chg.Index.(string)] = VolumeConfig{Name: newValue.Name, Extensions: map[string]interface{}{
159+
config.K8SExtensionKey: minified,
160+
}}
159161

160162
msg := fmt.Sprintf("added volume: %s", chg.Index.(string))
161163
log.Debugf(msg)

pkg/kev/kev_reconcile_test.go

+39-14
Original file line numberDiff line numberDiff line change
@@ -241,7 +241,10 @@ var _ = Describe("Reconcile", func() {
241241
Context("and the service has no deploy or healthcheck config", func() {
242242
BeforeEach(func() {
243243
workingDir = "testdata/reconcile-service-basic"
244-
overrideFiles = []string{workingDir + "/docker-compose.kev.dev.yaml"}
244+
overrideFiles = []string{
245+
workingDir + "/docker-compose.kev.dev.yaml",
246+
workingDir + "/docker-compose.kev.stage.yaml",
247+
}
245248
})
246249

247250
It("confirms the number of services pre reconciliation", func() {
@@ -250,19 +253,32 @@ var _ = Describe("Reconcile", func() {
250253
})
251254

252255
It("should add the new service to all environments", func() {
253-
Expect(env.GetServices()).To(HaveLen(2))
254-
Expect(env.GetServices()[0].Name).To(Equal("db"))
255-
Expect(env.GetServices()[1].Name).To(Equal("wordpress"))
256+
envs, err := manifest.GetEnvironments([]string{"dev", "stage"})
257+
Expect(err).ToNot(HaveOccurred())
258+
for _, env := range envs {
259+
Expect(env.GetServices()).To(HaveLen(2), "failed for env: %s", env.Name)
260+
Expect(env.GetServices()[0].Name).To(Equal("db"), "failed for env: %s", env.Name)
261+
Expect(env.GetServices()[1].Name).To(Equal("wordpress"), "failed for env: %s", env.Name)
262+
}
256263
})
257264

258-
It("should configure the added service extension value defaults", func() {
265+
It("should configure the added service extension value defaults in all environments", func() {
259266
expected, err := newMinifiedServiceExtensions("wordpress")
260267
Expect(err).NotTo(HaveOccurred())
261-
Expect(env.GetServices()[1].Extensions).To(Equal(expected))
268+
269+
envs, err := manifest.GetEnvironments([]string{"dev", "stage"})
270+
Expect(err).ToNot(HaveOccurred())
271+
for _, env := range envs {
272+
Expect(env.GetServices()[1].Extensions).To(Equal(expected), "failed for env: %s", env.Name)
273+
}
262274
})
263275

264-
It("should not include any env vars", func() {
265-
Expect(env.GetServices()[1].Environment).To(HaveLen(0))
276+
It("should not include any env vars in any environments", func() {
277+
envs, err := manifest.GetEnvironments([]string{"dev", "stage"})
278+
Expect(err).ToNot(HaveOccurred())
279+
for _, env := range envs {
280+
Expect(env.GetServices()[1].Environment).To(HaveLen(0), "failed for env: %s", env.Name)
281+
}
266282
})
267283

268284
It("should log the change summary using the debug level", func() {
@@ -321,21 +337,30 @@ var _ = Describe("Reconcile", func() {
321337
Context("when a new compose volume has been added", func() {
322338
BeforeEach(func() {
323339
workingDir = "testdata/reconcile-volume-add"
324-
overrideFiles = []string{workingDir + "/docker-compose.kev.dev.yaml"}
340+
overrideFiles = []string{
341+
workingDir + "/docker-compose.kev.dev.yaml",
342+
workingDir + "/docker-compose.kev.stage.yaml",
343+
}
325344
})
326345

327346
It("confirms the number of volumes pre reconciliation", func() {
328347
Expect(override.Volumes).To(HaveLen(0))
329348
})
330349

331350
It("should add the new volumes to all environments", func() {
332-
Expect(env.GetVolumes()).To(HaveLen(1))
351+
envs, err := manifest.GetEnvironments([]string{"dev", "stage"})
352+
Expect(err).ToNot(HaveOccurred())
353+
354+
for _, env := range envs {
355+
Expect(env.GetVolumes()).To(HaveLen(1))
356+
357+
v, _ := env.GetVolume("db_data")
358+
volExt := v.Extensions[config.K8SExtensionKey].(map[string]interface{})
333359

334-
v, _ := env.GetVolume("db_data")
335-
volExt := v.Extensions[config.K8SExtensionKey].(map[string]interface{})
360+
Expect(v.Extensions).To(HaveLen(1), "failed for env: %s", env.Name)
361+
Expect(volExt["size"]).To(Equal("100Mi"), "failed for env: %s", env.Name)
362+
}
336363

337-
Expect(v.Extensions).To(HaveLen(1))
338-
Expect(volExt["size"]).To(Equal("100Mi"))
339364
})
340365

341366
It("should log the change summary using the debug level", func() {

pkg/kev/override.go

+8-2
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ package kev
1919
import (
2020
"fmt"
2121

22+
"github.com/appvia/kev/pkg/kev/log"
2223
kmd "github.com/appvia/komando"
2324
composego "github.com/compose-spec/compose-go/types"
2425
"github.com/imdario/mergo"
@@ -63,7 +64,7 @@ func (o *composeOverride) diffAndPatch(dst *composeOverride) error {
6364
o.detectAndPatchVersionUpdate(dst)
6465

6566
if err := o.detectAndPatchServicesCreate(dst); err != nil {
66-
return nil
67+
return err
6768
}
6869

6970
if err := o.detectAndPatchServicesDelete(dst); err != nil {
@@ -117,17 +118,20 @@ func (o *composeOverride) detectAndPatchServicesCreate(dst *composeOverride) err
117118
if !dstSvcSet[srcSvc.Name] {
118119
cset.services = append(cset.services, change{
119120
Type: CREATE,
120-
Value: srcSvc.minusEnvVars(),
121+
Value: srcSvc,
121122
})
123+
log.Debugf("detected a new service named: %s", srcSvc.Name)
122124
}
123125
}
126+
124127
if cset.HasNoPatches() {
125128
step.Success("No service additions detected")
126129
return nil
127130
}
128131

129132
msgs, err := cset.applyServicesPatchesIfAny(dst)
130133
if err != nil {
134+
step.Error()
131135
return err
132136
}
133137
step.Success("Applied service additions")
@@ -230,6 +234,7 @@ func (o *composeOverride) detectAndPatchVolumesCreate(dst *composeOverride) erro
230234
Index: srcVolKey,
231235
Value: srcVolConfig,
232236
})
237+
log.Debugf("detected a new volume named: %s", srcVolKey)
233238
}
234239
}
235240

@@ -240,6 +245,7 @@ func (o *composeOverride) detectAndPatchVolumesCreate(dst *composeOverride) erro
240245

241246
msgs, err := cset.applyVolumesPatchesIfAny(dst)
242247
if err != nil {
248+
step.Error()
243249
return err
244250
}
245251

pkg/kev/services.go

-9
Original file line numberDiff line numberDiff line change
@@ -89,12 +89,3 @@ func (sc ServiceConfig) detectSecretsInEnvVars(matchers []map[string]string) []s
8989

9090
return matches
9191
}
92-
93-
// minusEnvVars returns a copy of the ServiceConfig with blank env vars
94-
func (sc ServiceConfig) minusEnvVars() ServiceConfig {
95-
return ServiceConfig{
96-
Name: sc.Name,
97-
Environment: map[string]*string{},
98-
Extensions: sc.Extensions,
99-
}
100-
}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
version: "3.7"
2+
services:
3+
db:
4+
x-k8s:
5+
workload:
6+
replicas: 1
7+
livenessProbe:
8+
type: exec
9+
exec:
10+
command: ["echo", "Define healthcheck command for service db"]
11+
initialDelay: 1m0s
12+
period: 1m0s
13+
failureThreashold: 3
14+
timeout: 10s
15+
service:
16+
type: None
17+
volumes:
18+
db_data:
19+
x-k8s:
20+
size: 100Mi
21+
storageClass: standard

pkg/kev/testdata/reconcile-service-basic/kev.yaml

+1
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,4 @@ compose:
33
- testdata/reconcile-service-basic/docker-compose.yaml
44
environments:
55
dev: testdata/reconcile-service-basic/docker-compose.kev.dev.yaml
6+
stage: testdata/reconcile-service-basic/docker-compose.kev.stage.yaml
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
version: "3.7"
2+
services:
3+
db:
4+
x-k8s:
5+
workload:
6+
type: StatefulSet
7+
replicas: 1
8+
livenessProbe:
9+
type: exec
10+
exec:
11+
command: ["echo", "Define healthcheck command for service db"]
12+
initialDelay: 1m0s
13+
period: 1m0s
14+
failureThreashold: 3
15+
timeout: 10s
16+
service:
17+
type: None
18+
wordpress:
19+
x-k8s:
20+
workload:
21+
type: Deployment
22+
replicas: 1
23+
livenessProbe:
24+
type: exec
25+
exec:
26+
command: ["echo", "Define healthcheck command for service wordpress"]
27+
initialDelay: 1m0s
28+
period: 1m0s
29+
failureThreashold: 3
30+
timeout: 10s
31+
service:
32+
type: LoadBalancer

pkg/kev/testdata/reconcile-volume-add/kev.yaml

+1
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,4 @@ compose:
33
- testdata/reconcile-volume-add/docker-compose.yaml
44
environments:
55
dev: testdata/reconcile-volume-add/docker-compose.kev.dev.yaml
6+
stage: testdata/reconcile-volume-add/docker-compose.kev.stage.yaml

0 commit comments

Comments
 (0)