-
Notifications
You must be signed in to change notification settings - Fork 26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ver 99110 allow users to specify tls certis for https server and client-server authentication #1041
base: main
Are you sure you want to change the base?
Conversation
For you test failure you need to add vertica-license: ${{ secrets.VERTICA_LICENSE }} to e2e-leg-6 in e2e.yml |
api/v1/verticadb_types.go
Outdated
HTTPSTLSSecret string `json:"httpsTLSSecret,omitempty"` | ||
|
||
// +operator-sdk:csv:customresourcedefinitions:type=spec,xDescriptors="urn:alm:descriptor:com.tectonic.ui:hidden" | ||
// +kubebuilder:default:="" | ||
// +kubebuilder:validation:Optional | ||
// A secret that contains the TLS credentials to be used by Vertica's client | ||
// (vsql). If this is empty, the operator will create a secret to use and add | ||
// the name of the generate secret to this field. | ||
// When set, the secret must have the following keys defined: tls.key, | ||
// tls.crt and ca.crt. To store this secret outside of Kubernetes, you can | ||
// use a secret path reference prefix, such as gsm://. Everything after the | ||
// prefix is the name of the secret in the service you are storing. | ||
ClientTLSSecret string `json:"clientTLSSecret,omitempty"` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How is this going to work?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This secret ClientTLSSecret stores the TLS cert. After the DB is created, we will use the new DDL (added by Fen) to let Vertica server load the certificate directly from the secret. Vertica Server will use the this certificate when vsql connects to it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is a lot of code duplication with tlsservercertgen_reconciler.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, I've resolved the duplicated code.
@@ -185,7 +185,11 @@ func (r *VerticaDBReconciler) constructActors(log logr.Logger, vdb *vapi.Vertica | |||
// of the operator. | |||
MakeUpgradeOperatorReconciler(r, log, vdb), | |||
// Create a TLS secret for the NMA service | |||
MakeHTTPServerCertGenReconciler(r, log, vdb), | |||
MakeNMACertGenReconciler(r, log, vdb), | |||
// use the same TLS secret used by the NMA service for https service |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought you added a new secret for http service
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Originally we only had 'http' which was used for NMA.
Now we will have separate certificates for NMA, Https, Client.
So here, I renamed http to NMA.
volMnts := []corev1.VolumeMount{ | ||
buildScrutinizeSharedVolumeMount(vscr), | ||
} | ||
|
||
if vmeta.UseNMACertsMount(vdb.Annotations) && |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why can't we no longer mount nma certs?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OpenText's new security policy does not allow the certificates to be stored on hard drives.
Previously, we offered volume mount and environmental variables to pass the certificate to NMA.
Now we no longer support volume mount. We still support environment variables. But the
environment variables are mapped from a config map. When users try to rotate NMA cert, they
need to update the configmap with new cert secret name and restart NMA.
h.Log.Info("not all tls secret names are ready. wait for them to be created") | ||
return false | ||
} | ||
found, err := vapi.IsK8sSecretFound(ctx, h.Vdb, h.VRec.Client, &h.Vdb.Spec.NMATLSSecret) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think you need err
given that found will always be false
when err != nil
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch. I added logging for err.
@cchen-vertica there is a lot of things I don't understand about these changes. Can you add your comments? Can we also have a discussion tomorrow so you can explain what we are doing. |
I will update it. You'd better wait until it is updated. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You need to add two changie logs: 1. added fields ... 2. removed cert mount for NMA container.
api/v1/verticadb_types.go
Outdated
// +kubebuilder:default:="" | ||
// +kubebuilder:validation:Optional | ||
// A secret that contains the TLS credentials to be used by Vertica's embedded | ||
// http service. If this is empty, the operator will create a secret to use |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Modify "http" to "https". We should comment out or hide this field for now. The reason is : if the user specifies different secrets for "HTTPSTLSSecret" and "NMATLSSecret", vclusterOps will reject the request.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is removed. Lint does not allow commenting out code.
api/v1/verticadb_types.go
Outdated
// +operator-sdk:csv:customresourcedefinitions:type=spec,xDescriptors="urn:alm:descriptor:com.tectonic.ui:hidden" | ||
// +kubebuilder:default:="" | ||
// +kubebuilder:validation:Optional | ||
// A secret that contains the TLS credentials to be used by Vertica's client |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
change it to "... to be used to authenticate Vertica clients' certificates".
In addition, we should rename "ClientTLSSecret" to "ServerTLSSecret" or "ClientServerTLSSecret". Also, we need to add one more field "ServerTLSMode" or "ClientServerTLSMode" to configure TLS mode in the server.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"ClientServerTLSMode" and "ClientServerTLSSecret" will be used.
@@ -555,65 +555,6 @@ var _ = Describe("builder", func() { | |||
} | |||
}) | |||
|
|||
It("should mount or not mount NMA certs volume based on NMA container", func() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we have a test to verify the ENV var are coming from the config map?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are we able to add this kind of unit test?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think that is doable in unit tests as the ENV var is in nma container env. I already have that covered in e2e tests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You should have a unit test to verify your config map is having the correct data. That is for verifying your new function BuildNMATLSConfigMap().
) | ||
|
||
// NMACertGenReconciler will create a secret that has TLS credentials. This | ||
// secret will be used to authenticate with the http server. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Update "http" to "https" in this file
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This needs to be done.
if err != nil { | ||
return ctrl.Result{}, err | ||
} | ||
h.Log.Info("created certificate and secret for https service") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use secretFieldName to replace "https service".
// Reconcile will create a TLS secret for the http server if one is missing | ||
func (h *TLSServerCertGenReconciler) Reconcile(ctx context.Context, _ *ctrl.Request) (ctrl.Result, error) { | ||
secretFieldNameMap := map[string]string{ | ||
HTTPSTLSSecret: h.Vdb.Spec.HTTPSTLSSecret, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When HTTPSTLSSecret is not specified in the vdb, we should use NMATLSSecret, not need to generate another one.
|
||
// reconcileOneSecret will create a TLS secret for the http server if one is missing | ||
func (h *TLSServerCertGenReconciler) reconcileOneSecret(secretFieldName, secretName string, | ||
ctx context.Context) (ctrl.Result, error) { //nolint:unparam |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why disabling "unparam" lint here? We should check if the parameters are set reasonable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here is the error I got after enabling it. Any idea about how to get rid of it?
pkg/controllers/vdb/tlsservercertgen_reconciler.go:76:24: (*TLSServerCertGenReconciler).reconcileOneSecret - result 0 (sigs.k8s.io/controller-runtime/pkg/reconcile.Result) is always nil (unparam)
ctx context.Context) (ctrl.Result, error) {
^
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This error occurs because your function can only return "ctrl.Result{}" and an error. You need to remove your first returned value.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point!
apiVersion: v1 | ||
kind: ConfigMap | ||
metadata: | ||
name: nma-tls-config |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should be able to verify the data in the config map as well.
@@ -14,5 +14,6 @@ | |||
apiVersion: kuttl.dev/v1beta2 | |||
kind: TestStep | |||
commands: | |||
# nma certs volume mount path should not exist. | |||
- command: kubectl exec -n $NAMESPACE v-nma-certs-sc1-0 -- bash -c "! test -d /certs/nma" | |||
# nma env variable should exist. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's also verify if the env variable value matches the value we set.
annotations: | ||
vertica.com/vcluster-ops: "true" | ||
vertica.com/mount-nma-certs: "false" | ||
spec: | ||
initPolicy: CreateSkipPackageInstall | ||
image: kustomize-vertica-image |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could set the field "NMATLSSecret" here, and verify if the customized value equal to the one in the environment variable.
return &NMACertConfigMapGenReconciler{ | ||
VRec: vdbrecon, | ||
Vdb: vdb, | ||
Log: log.WithName("TLSCertConfigMapGenReconciler"), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Log: log.WithName("TLSCertConfigMapGenReconciler"), | |
Log: log.WithName("NMACertConfigMapGenReconciler"), |
} | ||
|
||
// buildTLSConfigMap return a ConfigMap. Key is the json file name and value is the json file content | ||
func (h *NMACertConfigMapGenReconciler) buildTLSConfigMap(vdb *vapi.VerticaDB) *corev1.ConfigMap { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Move this to builder.go
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is still a lot of code duplication with nmaservertgen_reconciler
: the 2 secrets are quite similar so must of the code can be reused. May be you can move it to a separate package and reuse the code. The secrets have the same structure so there is no reason why we cannot reuse the same logic to construct both.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good observation. I almost forgot that. It is pretty easy to use tlsservertgen_reconciler for nma.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is still not address. I see functions like getDNSName(), createSecret(), setSecretNameInVDB() in both files. Also reconcileOneSecret() code is similar to (h *NMACertGenReconciler) Reconcile(). Move those logics to a separate file (ex k8s.go in the same package) and use the functions in the 2 files. The 2 reconcilers do the same thing for different secret so a lot of code can be reuse in both. Please address this!!!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
NMACertGenReconciler is removed. No duplicates any more.
- command: kubectl delete sts v-tls-certs-sc1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need to add another test that only kill NMA containers without shutting down the database.
api/v1/verticadb_types.go
Outdated
// +kubebuilder:default:="" | ||
// +kubebuilder:validation:Optional | ||
// A secret that contains the TLS credentials to be used to authenticate Vertica clients' certificates | ||
// (vsql). If this is empty, the operator will create a secret to use and add |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove "(vsql)"
// +operator-sdk:csv:customresourcedefinitions:type=spec,xDescriptors="urn:alm:descriptor:com.tectonic.ui:hidden" | ||
// +kubebuilder:default:=verify_ca | ||
// +kubebuilder:validation:Optional | ||
ClientServerTLSMode string `json:"clientServerTLSMode,omitempty"` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Write a description of this field and list all its supported values. The default value should be "try_verify".
api/v1/verticadb_webhook.go
Outdated
} | ||
} | ||
if !validMode { | ||
err := field.Invalid(field.NewPath("spec").Child("clientservertlssecret"), v.Spec.ClientServerTLSMode, "invalid tls mode") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use camel case for "clientservertlssecret" - "clientSeverTLSSecret"
@@ -0,0 +1,5 @@ | |||
kind: Changed |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be "Removed". Before NMATLSSecret supported both mount and unmount options, and now NMATLSSecret only supported unmount option, right?
body: ClientServerTLSSecret has been added to Vertica DB definition. It contains the TLS credentials to authenticate Vertica clients' certificates (vsql) | ||
time: 2025-02-04T16:14:32.906578043-05:00 | ||
custom: | ||
Issue: "99110" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The issue number should be "1041"
|
||
// reconcileOneSecret will create a TLS secret for the http server if one is missing | ||
func (h *TLSServerCertGenReconciler) reconcileOneSecret(secretFieldName, secretName string, | ||
ctx context.Context) (ctrl.Result, error) { //nolint:unparam |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This error occurs because your function can only return "ctrl.Result{}" and an error. You need to remove your first returned value.
kind: VerticaDB | ||
metadata: | ||
name: v-tls-certs | ||
status: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
verify nmaTLSSecret here as well
# nma certs volume mount path should exist. | ||
- command: kubectl exec -n $NAMESPACE v-nma-certs-sc1-0 -- bash -c "test -d /certs/nma" | ||
# nma env variable should exist. | ||
- command: kubectl exec -n $NAMESPACE v-tls-certs-sc1-0 -c nma -- bash -c "env | grep nma-tls-cert-secret" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should verify two env vars equal the target value, e.g. secretName=nma-tls-cert-secret,namespace=$NAMESPACE
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried that. It is not possible to pass a pod environmental variable. Kuttl assumes all environmental variables are from its test environment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But you can execute inside the pod and print out its value, right? Here you already can access "env".
Do something like:
k exec -it v-base-upgrade-pri1-0 -c nma -- sh -c '[ "$SECRET_NAME" = "nma-tls-cert-secret" ] || { echo "Mismatch"; exit 1; }'
kind: TestStep | ||
commands: | ||
# nma env variable should exist. | ||
- command: kubectl exec -n $NAMESPACE v-tls-certs-sc1-0 -c nma -- bash -c "env | grep nma-tls-cert-secret" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should add a couple of extra steps to verify when you change the nmaTLSSecret to another one, the config map and env vars of nma will be updated as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That is a good point. I will add that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added the steps and found a bug in the code. Fixed the bug.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where are the steps you added? I cannot see them in this test suite.
{ | ||
"apiVersion": "v1", | ||
"data": { | ||
"ca.crt": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURCVENDQWUyZ0F3SUJBZ0lSQUp2bWl1b1BPNnFCeEFLbmtHMFp3NkV3RFFZSktvWklodmNOQVFFTEJRQXcKR2pFWU1CWUdBMVVFQXhNUGF6aHpMblpsY25ScFkyRXVZMjl0TUI0WERUSTBNVEl3T1RFM01UUTBORm9YRFRJMQpNRE13T1RFM01UUTBORm93R2pFWU1CWUdBMVVFQXhNUGF6aHpMblpsY25ScFkyRXVZMjl0TUlJQklqQU5CZ2txCmhraUc5dzBCQVFFRkFBT0NBUThBTUlJQkNnS0NBUUVBdmlidlhhbkVYY004Z3EzazFhayt5N0I2VnB5Q2pkeWUKaVhEVmZTQUZVdWlpcE1VVFJ0cDRPQWs3TFh2Y2lVc3lsUk5NT3FYdGxXTUJXZEpXZkl3S2owc2hEYVhsRTkzWQprOFBEY0JPN1FGRHVLVWRUSldHbmVkQ0ZrSGlBUEJJSlRlRFQ2Q1ZEZDFhbDdrdGtmbzhxaXBLS2s2QXZ6c3VjCkxPUk81QWo4bzFqWWkySkZxYytKTjgrZFhDWERYczRMM2oxNWZaUUlrakg1ODdPMDNGcVpxZzlRa0tVTHlZdmoKOHllZ0N4UWlzdW94SjN5bVJwa21xamRqdnlVQUlTT3EyYThremhDcjBGSUhHcEk5eU1TM0RsTnVLMlJDZFNiUwpROSs2RG1VNjlUZEN6RWVLVktSeGFsK3A1UEREMHlVMzZiQ3VyVVN4eXpSaG0zZGxDNm5LTlFJREFRQUJvMFl3ClJEQU9CZ05WSFE4QkFmOEVCQU1DQmFBd0RBWURWUjBUQVFIL0JBSXdBREFrQmdOVkhSRUVIVEFiZ2dWdGFXNXAKYjRJU2JXbHVhVzh1YTNWMGRHd3RaVEpsTFhNek1BMEdDU3FHU0liM0RRRUJDd1VBQTRJQkFRQm53RFJJMUoyaApqV0lNYnJnSWhyTFVDREt2STFvN1NBRXJ4ME9BL0U0YlZSMGZnZVpZQ005UURpM21HY1Zvc1N6QlNtcEJaTHBpCjVsL1Zwc051Q1c4ZkR5UzJaeC9ybXlUMzV2RVc4dGN2aFhEeGV0RXYvTTJLRkZYVWV5TDNrQ21xTTAwcjFybmYKZG9OQUxIcCsyS3B5RDlJZ3lQdHU3NFp2SGNYWFp3SjdONlczU0k4c0E0S0NzOTNLNXRnSWVoZlFMbmFEektGYQpEU3EzN1VBclkzVU5OWlJGdnVtbXc2WmFjMG5mdTF2Q2J3bmc1cXlyaDE0TUFLcGlhMGlPcXp6aVpPVlNUSkY4CmdoajZiSnd1OG5UVVhZaU1aaGdTbno5N29IWFVBOHR1SVRMU2tmMVBUNGkxeUtCbFVDOGhLWWZhb0JDTXNVYVMKTkJyQmFsaUorZGNMCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just use K8s "kind: Certificate". Refer to "tests/e2e-leg-5/metrics-auth-cert/04-create-cert.yaml"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I copied the json from ./kustomize-base/communal-ep-cert.json. I added its name to the vdb in setup-vdb.yaml.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Basically, I don't want the certs to be directly list in the code. Some security check will report this expose. As you can see, ./kustomize-base/communal-ep-cert.json is not in our repo. It will only be generated in your local env.
@@ -0,0 +1,5 @@ | |||
kind: Added | |||
body: ClientServerTLSSecret has been added to Vertica DB definition. It contains the TLS credentials to authenticate Vertica clients' certificates (vsql) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove "(vsql)" since we have other types of clients.
@@ -555,65 +555,6 @@ var _ = Describe("builder", func() { | |||
} | |||
}) | |||
|
|||
It("should mount or not mount NMA certs volume based on NMA container", func() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You should have a unit test to verify your config map is having the correct data. That is for verifying your new function BuildNMATLSConfigMap().
) | ||
|
||
// NMACertGenReconciler will create a secret that has TLS credentials. This | ||
// secret will be used to authenticate with the http server. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This needs to be done.
func (h *NMACertConfigMapGenReconciler) Reconcile(ctx context.Context, _ *ctrl.Request) (ctrl.Result, error) { | ||
nmaSecret := corev1.Secret{} | ||
if !h.tlsSecretsReady(ctx, &nmaSecret) { | ||
return ctrl.Result{Requeue: true}, nil |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add a log here to show why we requeue the reconciler.
# nma certs volume mount path should exist. | ||
- command: kubectl exec -n $NAMESPACE v-nma-certs-sc1-0 -- bash -c "test -d /certs/nma" | ||
# nma env variable should exist. | ||
- command: kubectl exec -n $NAMESPACE v-tls-certs-sc1-0 -c nma -- bash -c "env | grep nma-tls-cert-secret" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But you can execute inside the pod and print out its value, right? Here you already can access "env".
Do something like:
k exec -it v-base-upgrade-pri1-0 -c nma -- sh -c '[ "$SECRET_NAME" = "nma-tls-cert-secret" ] || { echo "Mismatch"; exit 1; }'
kind: TestStep | ||
commands: | ||
# nma env variable should exist. | ||
- command: kubectl exec -n $NAMESPACE v-tls-certs-sc1-0 -c nma -- bash -c "env | grep nma-tls-cert-secret" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where are the steps you added? I cannot see them in this test suite.
{ | ||
"apiVersion": "v1", | ||
"data": { | ||
"ca.crt": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURCVENDQWUyZ0F3SUJBZ0lSQUp2bWl1b1BPNnFCeEFLbmtHMFp3NkV3RFFZSktvWklodmNOQVFFTEJRQXcKR2pFWU1CWUdBMVVFQXhNUGF6aHpMblpsY25ScFkyRXVZMjl0TUI0WERUSTBNVEl3T1RFM01UUTBORm9YRFRJMQpNRE13T1RFM01UUTBORm93R2pFWU1CWUdBMVVFQXhNUGF6aHpMblpsY25ScFkyRXVZMjl0TUlJQklqQU5CZ2txCmhraUc5dzBCQVFFRkFBT0NBUThBTUlJQkNnS0NBUUVBdmlidlhhbkVYY004Z3EzazFhayt5N0I2VnB5Q2pkeWUKaVhEVmZTQUZVdWlpcE1VVFJ0cDRPQWs3TFh2Y2lVc3lsUk5NT3FYdGxXTUJXZEpXZkl3S2owc2hEYVhsRTkzWQprOFBEY0JPN1FGRHVLVWRUSldHbmVkQ0ZrSGlBUEJJSlRlRFQ2Q1ZEZDFhbDdrdGtmbzhxaXBLS2s2QXZ6c3VjCkxPUk81QWo4bzFqWWkySkZxYytKTjgrZFhDWERYczRMM2oxNWZaUUlrakg1ODdPMDNGcVpxZzlRa0tVTHlZdmoKOHllZ0N4UWlzdW94SjN5bVJwa21xamRqdnlVQUlTT3EyYThremhDcjBGSUhHcEk5eU1TM0RsTnVLMlJDZFNiUwpROSs2RG1VNjlUZEN6RWVLVktSeGFsK3A1UEREMHlVMzZiQ3VyVVN4eXpSaG0zZGxDNm5LTlFJREFRQUJvMFl3ClJEQU9CZ05WSFE4QkFmOEVCQU1DQmFBd0RBWURWUjBUQVFIL0JBSXdBREFrQmdOVkhSRUVIVEFiZ2dWdGFXNXAKYjRJU2JXbHVhVzh1YTNWMGRHd3RaVEpsTFhNek1BMEdDU3FHU0liM0RRRUJDd1VBQTRJQkFRQm53RFJJMUoyaApqV0lNYnJnSWhyTFVDREt2STFvN1NBRXJ4ME9BL0U0YlZSMGZnZVpZQ005UURpM21HY1Zvc1N6QlNtcEJaTHBpCjVsL1Zwc051Q1c4ZkR5UzJaeC9ybXlUMzV2RVc4dGN2aFhEeGV0RXYvTTJLRkZYVWV5TDNrQ21xTTAwcjFybmYKZG9OQUxIcCsyS3B5RDlJZ3lQdHU3NFp2SGNYWFp3SjdONlczU0k4c0E0S0NzOTNLNXRnSWVoZlFMbmFEektGYQpEU3EzN1VBclkzVU5OWlJGdnVtbXc2WmFjMG5mdTF2Q2J3bmc1cXlyaDE0TUFLcGlhMGlPcXp6aVpPVlNUSkY4CmdoajZiSnd1OG5UVVhZaU1aaGdTbno5N29IWFVBOHR1SVRMU2tmMVBUNGkxeUtCbFVDOGhLWWZhb0JDTXNVYVMKTkJyQmFsaUorZGNMCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Basically, I don't want the certs to be directly list in the code. Some security check will report this expose. As you can see, ./kustomize-base/communal-ep-cert.json is not in our repo. It will only be generated in your local env.
1 removed nma cert mount
2 generate https and client-server tls cert secret when they are not set by users
3 put https and client-server secret names in a config json file and mount it to the vertica pod.
4 an e2e test case is added