Updated nightwatch workflow inputs and added Estuary app to trigger workflow

[MarkCalvert]

Changed

1. Updated the nightwatch workflow input names to match what the estuary API expects to send to the GitHub workflows.
   https://github.com/dpc-sdp/estuary/blob/79da49c5866435436a14795c288fc8a9a2cceaa4/pkg/github/server.go#L53
2. Removed E2E input and logic
3. Updated trigger-e2e.sh script to use the Estuary app to trigger e2e testing.

[MarkCalvert]

Hey @FleetAdmiralButter @tim-yao ,
I successfully triggered the e2e tests via estuary on a Lagoon Build.
https://dashboard.amazeeio.cloud/projects/ripple/ripple-pr-1319/deployments/lagoon-build-uazonj
https://github.com/dpc-sdp/ripple/actions/runs/6488552014.
For testing, I have added two env variables to the project using Lagoon CLI E2E_ESTUARY_URL & E2E_ESTUARY_TOKEN_PATH.
I believe these are managed somewhere else for SDP?

[tim-yao]

Hey @FleetAdmiralButter @tim-yao , I successfully triggered the e2e tests via estuary on a Lagoon Build. https://dashboard.amazeeio.cloud/projects/ripple/ripple-pr-1319/deployments/lagoon-build-uazonj https://github.com/dpc-sdp/ripple/actions/runs/6488552014. For testing, I have added two env variables to the project using Lagoon CLI E2E_ESTUARY_URL & E2E_ESTUARY_TOKEN_PATH. I believe these are managed somewhere else for SDP?

I can see the triggered action. So I guess it's working as expected? For the two new vars, @FleetAdmiralButter do you have any suggestion?

[FleetAdmiralButter]

Hi @tim-yao @MarkCalvert, since E2E_ESTUARY_URL and E2E_ESTUARY_TOKEN_PATH are consumed by a script in the Lagoon project itself, it should be configured in CMDB as a build-time variable.

Though I don't think we need to configure E2E_ESTUARY_TOKEN_PATH - the path to the service account token will always be in /run/secrets/kubernetes.io/serviceaccount/token.

[tim-yao]

Hi @tim-yao @MarkCalvert, since E2E_ESTUARY_URL and E2E_ESTUARY_TOKEN_PATH are consumed by a script in the Lagoon project itself, it should be configured in CMDB as a build-time variable.

Though I don't think we need to configure E2E_ESTUARY_TOKEN_PATH - the path to the service account token will always be in /run/secrets/kubernetes.io/serviceaccount/token.

I support fewer vars, it can reduce the maintenance work but it's up to you guys to decide.

[MarkCalvert]

Hi @tim-yao @MarkCalvert, since E2E_ESTUARY_URL and E2E_ESTUARY_TOKEN_PATH are consumed by a script in the Lagoon project itself, it should be configured in CMDB as a build-time variable.
Though I don't think we need to configure E2E_ESTUARY_TOKEN_PATH - the path to the service account token will always be in /run/secrets/kubernetes.io/serviceaccount/token.

I support fewer vars, it can reduce the maintenance work but it's up to you guys to decide.

@tim-yao @FleetAdmiralButter I don't have access to CMDB, so can one of you add the variables?
I added the token path as an env variable for security reasons because this repo is public.
However, we would have to delete the commits and this PR somehow....

[tim-yao]

@FleetAdmiralButter @MarkCalvert I would like to change the workflow name to e2e instead of nightwatch later. Like https://github.com/dpc-sdp/fvrim-vic-gov-au/pull/103/files.
It can be another PR, so no change is required in this PR. Thanks!

[tim-yao]

Hi @tim-yao @MarkCalvert, since E2E_ESTUARY_URL and E2E_ESTUARY_TOKEN_PATH are consumed by a script in the Lagoon project itself, it should be configured in CMDB as a build-time variable.
Though I don't think we need to configure E2E_ESTUARY_TOKEN_PATH - the path to the service account token will always be in /run/secrets/kubernetes.io/serviceaccount/token.

I support fewer vars, it can reduce the maintenance work but it's up to you guys to decide.

@tim-yao @FleetAdmiralButter I don't have access to CMDB, so can one of you add the variables? I added the token path as an env variable for security reasons because this repo is public. However, we would have to delete the commits and this PR somehow....

@FleetAdmiralButter It will need to be implemented into existing projects and new project templates. I am not sure what's the best way to do it now. Can you help on this? Thanks!

[FleetAdmiralButter]

@MarkCalvert The path to the service account token (/run/secrets/kubernetes.io/serviceaccount/token) is a standard path in Kubernetes, so technically that information is public already.

Could we set default values in the script? Like:

E2E_ESTUARY_URL="${E2E_ESTUARY_URL:-http://estuary.sdp-services/}"
E2E_ESTUARY_TOKEN_PATH ="${E2E_ESTUARY_TOKEN_PATH:-/run/secrets/kubernetes.io/serviceaccount/token}"

[MarkCalvert]

@MarkCalvert The path to the service account token (/run/secrets/kubernetes.io/serviceaccount/token) is a standard path in Kubernetes, so technically that information is public already.

Could we set default values in the script? Like:

E2E_ESTUARY_URL="${E2E_ESTUARY_URL:-http://estuary.sdp-services/}"
E2E_ESTUARY_TOKEN_PATH ="${E2E_ESTUARY_TOKEN_PATH:-/run/secrets/kubernetes.io/serviceaccount/token}"

@FleetAdmiralButter Ahh I see, good to know.
Yes we can set the default values like that.

[MarkCalvert]

Removed the env variables using Lagoon CLI and added default values in the script.
E2E workflow was triggered on lagoon build
https://github.com/dpc-sdp/ripple/actions/runs/6491017510