envsubst: remove explicit subst of exported vars

[alxndrsn]

Approach suggested in #818 (comment)

---

For nginx config, a new approach is implemented with mawk.

This is similar to envsubst, but more ergonomic:

• no need to explicitly list all variables
• throw error on missing variables
• do not replace nginx vars like $host, $request_uri with empty strings (in contrast to envsubst when executed without an explicit variable list)

Risks:

There are a couple of changes which may break existing deployments:

• changing client-config.json.template
• requiring all substituted variable to be defined

Closes #473

---

What has been done to verify that this works as intended?

Added tests, added tests to CI, run tests.

Why is this the best possible solution? Were any other approaches considered?

Considered perl, bash & sed, as they are other scripting languages present in the images in which we currently use enbsubst. awk or perl seem the most suitable. I am not aware of a reason to use one over the other.

# jonasal/nginx-certbot:5.4.0

* perl: v5.36.0
* awk: mawk 1.3.4 20200120
* node: not found
* bash: GNU bash, version 5.2.15(1)-release (x86_64-pc-linux-gnu)
* sed: sed (GNU sed) 4.9

# node:20.17.0-slim

* perl: v5.36.0
* awk: mawk 1.3.4 20200120
* node: v20.17.0
* bash: GNU bash, version 5.2.15(1)-release (x86_64-pc-linux-gnu)
* sed: sed (GNU sed) 4.9

# ghcr.io/enketo/enketo:7.4.0

* perl: v5.36.0
* awk: mawk 1.3.4 20200120
* node: v20.17.0
* bash: GNU bash, version 5.2.15(1)-release (x86_64-pc-linux-gnu)
* sed: sed (GNU sed) 4.9

How does this change affect users? Describe intentional changes to behavior and behavior that could have accidentally been affected by code changes. In other words, what are the regression risks?

As mentioned above, there's a risk that throwing when a variable is not defined will break existing deployments. That wouldn't be great.

Does this change require updates to documentation? If so, please file an issue here and include the link below.

Maybe? If there is some genuine risk in the previous answer.

Before submitting this PR, please make sure you have:

• branched off and targeted the next branch OR only changed documentation/infrastructure (master is stable and used in production)
• verified that any code or assets from external sources are properly credited in comments or that everything is internally sourced

[brontolosone]

Splendid!

[lognaturel]

Can we remove the gettext dependency installed in all the images?

[alxndrsn]

Can we remove the gettext dependency installed in all the images?

Good catch - that seems very likely. I've removed gettext installations.

[lognaturel]

One small change inline and one quick confirmation -- this works fine if a user's configuration file omits certain config keys, right (e.g. the OIDC ones)? I imagine a bunch of tests would fail if that weren't the case but want to double check.

[alxndrsn]

one quick confirmation -- this works fine if a user's configuration file omits certain config keys, right (e.g. the OIDC ones)?

I think this is handled in the production docker-compose.yml file by the environment defaults:

central/docker-compose.yml

Lines 49 to 76 in 279c4cb

	environment:
	- DOMAIN=${DOMAIN}
	- SYSADMIN_EMAIL=${SYSADMIN_EMAIL}
	- HTTPS_PORT=${HTTPS_PORT:-443}
	- NODE_OPTIONS=${SERVICE_NODE_OPTIONS:-}
	- DB_HOST=${DB_HOST:-postgres14}
	- DB_USER=${DB_USER:-odk}
	- DB_PASSWORD=${DB_PASSWORD:-odk}
	- DB_NAME=${DB_NAME:-odk}
	- DB_SSL=${DB_SSL:-null}
	- EMAIL_FROM=${EMAIL_FROM:-no-reply@$DOMAIN}
	- EMAIL_HOST=${EMAIL_HOST:-mail}
	- EMAIL_PORT=${EMAIL_PORT:-25}
	- EMAIL_SECURE=${EMAIL_SECURE:-false}
	- EMAIL_IGNORE_TLS=${EMAIL_IGNORE_TLS:-true}
	- EMAIL_USER=${EMAIL_USER:-}
	- EMAIL_PASSWORD=${EMAIL_PASSWORD:-}
	- OIDC_ENABLED=${OIDC_ENABLED:-false}
	- OIDC_ISSUER_URL=${OIDC_ISSUER_URL:-}
	- OIDC_CLIENT_ID=${OIDC_CLIENT_ID:-}
	- OIDC_CLIENT_SECRET=${OIDC_CLIENT_SECRET:-}
	- SENTRY_ORG_SUBDOMAIN=${SENTRY_ORG_SUBDOMAIN:-o130137}
	- SENTRY_KEY=${SENTRY_KEY:-3cf75f54983e473da6bd07daddf0d2ee}
	- SENTRY_PROJECT=${SENTRY_PROJECT:-1298632}
	- S3_SERVER=${S3_SERVER:-}
	- S3_ACCESS_KEY=${S3_ACCESS_KEY:-}
	- S3_SECRET_KEY=${S3_SECRET_KEY:-}
	- S3_BUCKET_NAME=${S3_BUCKET_NAME:-}

If a value is undefined by a user, and omitted from these lists, then substitution would fail.

I've added an extra test case to ensure that substituting empty values is supported.