Nersc deployment adaptations (Auth, Environment variables)

[Wiebke]

This PR groups some minor changes in relation to the deployment on Nersc. Most importantly it adds the basic auth wrapper, for now with a single username-password pair.

Additionally the following minor changes are included:

• Rename API_KEY to TILED_API_KEY to prepare for adding other keys (e.g. to Splash ML)
• Update the TILED_URI to https://tiled-seg.als.lbl.gov in README.md
• Make it possible for EXPORT_FILE_PATH to be an environment variable
• Add missing volume mount for constants.py in docker-compose.yml
• Ignore differently named Python environments *env/

[github-actions]

Staging application has been deployed and is available at: https://dash5-services.plotly.host/ml-exchange-staging
Production app name: ml-exchange
Current branch name: nersc-deployment-adaptations
Commit: be7f176

[hannahker]

See the couple suggestions relating to preserving behaviour in our local development.

[hannahker]

Wondering if we want to make this something like:

auth = dash_auth.BasicAuth(app, VALID_USER_NAME_PASSWORD_PAIRS) if os.getenv("env") != "local" else None

So that we can set a local mode where we don't have to log in each time while developing.

[Wiebke]

I have added a DASH_DEPLOYMENT_LOC variable that steers if authentication is enabled (when that variable is set to 'Local') or not.

[hannahker]

I'm testing this against our local Tiled server as well and get surprising behaviour with this change. My Python code throws an authentication error:

    raise ClientError(message, exc.request, exc.response) from exc
tiled.client.utils.ClientError: 401: Invalid API key http://localhost:8000/api/v1/metadata/

And the local Tiled server blocks my requests:

INFO:     127.0.0.1:62131 - "GET /api/v1/metadata/clay_testZMQ HTTP/1.1" 401 Unauthorized

This seems to be only the case if this variable is called TILED_API_KEY. Not a problem, for example, if I change this to TLD_API_KEY. Wondering if this is because Tiled is trying to be too smart under the hood and something in the local client detects any env var with this name and tries to apply it?

[Wiebke]

Indeed, you are right. My current understanding of what causes this error is as follows:

The client picks up on TILED_API_KEY being set as an environment variables and then uses that as an API key to authenticate, see [tiled/client/context.py#L79].(https://github.com/bluesky/tiled/blob/7566d0f2d61cb0477bf187b33e94d7449358c4e5/tiled/client/context.py#L79)

Our local server started with tiled serve directory --public data is in principle accessible without a key (we are allowed to read data without having an api_key in the request), but it does generate a key on startup (which would be needed for writing data). Our requests having a different key then causes the error.

[Wiebke]

I see 2 main ways to address this:

1. Rename the TILED_API_KEY environment variable to something else, or
2. Start our local tiled server with the api key from the environment, such that we get a match again.

Keeping in mind that we may have this application run in different deployment environments and hooked up to different Tiled servers, I think I would like to stay away from baking in more semantics into the variable. In the latest update, I thus added a script that starts the local server with the TILED_API_KEY from the environment file by setting the TILED_SINGLE_API_KEY variable that the server makes use of (if set).

Let me know if that works for your test workflow @hannahker.

[hannahker]

Thanks @Wiebke for the changes. Confirming that this works locally from my end now 💃