Documentation

README.md

@@ -5,3 +5,187 @@ It attempts to simulate user interactions with Science Platform services continu
 It can be used for both monitoring and synthetic load testing.
 
 mobu is developed with the [Safir](https://safir.lsst.io/) framework.
+
+A full-blown [Documenteer](https://documenteer.lsst.io/) site with more detailed documentation is coming soon.
+
+## Developers
+
+### GitHub Integration
+
+Mobu offers two integrations with GitHub:
+* Automatic notebook refreshing in flocks
+* GitHub Actions checks for PR commits
+
+Each is enabled by enabling a GitHub application on a repo full of notebooks.
+There is a separate GitHub application for each integration in each environment.
+This lets you enable these integrations for different combinations of environments.
+For example, you can enable the auto-refresh integration in `idfdev`, `idfint`, `usdfdev`, `usdfint`, and `usdfprod`, but the CI integration only in `idfint` and `usdfint`.
+
+#### Auto-refresh notebook flocks
+
+*Problem*
+
+You have a GitHub repo filled with Python notebooks.
+You've already configured Mobu to run a flock that executes these notebooks.
+But every time you commit changes to the notebooks in this repo, you have to restart Mobu to get it to pick up the changes!
+
+*Solution*
+
+Enable a GitHub app for your repo, and Mobu will automatically pick up any changes!
+
+##### Configuration
+
+There is a `mobu refresh (<env url>)` GitHub app for every environment that runs Mobu (Except environments behind a VPN).
+For every environment in which you want your repo to auto refresh, you have to:
+
+*Install the app in your repo's organization, if it's not already installed*
+1. Have a Mobu autorun flock configured and running
+1. Install that environment's `mobu refresh` app in your repo's GitHub organization. Someone with appropriate permissions can do that from the [app's homepage](#mobu-refresh-github-app-urls)
+1. Add your organization to the `accetped_github_orgs` list in the Mobu configuration in Phalanx for the matching env
+
+*Enable the app for your repo*
+1. Go to your repo's organization settings page in GitHub, and go to the "GitHub Apps" page in the "Third-party Access" section in the left sidebar. [Here](https://github.com/organizations/lsst-sqre/settings/installations) is that page for the `lsst-sqre` organization
+1. Click the `Configure` button in the `mobu refresh (<env url>)` row
+1. In the "Repository access" section, click the "Select repositories" dropdown and add your repository
+
+##### Mobu Refresh GitHub App URLs
+* [data-dev.lsst.cloud](https://github.com/apps/mobu-refresh-data-dev-lsst-cloud)
+
+#### Run notebooks in Mobu as a GitHub CI check
+
+*Problem*
+
+You have a GitHub repo filled with Python notebooks.
+You've already configured Mobu to run a flock that executes these notebooks.
+You want to make sure that any changes to these notebooks don't cause them to break when they run in Mobu, but you don't want to have to commit the possibly-broken changes just to get Mobu to run them to test the changes.
+
+*Solution*
+
+Enable a GitHub app for your repo, and Mobu will create a GitHub Actions check that runs changed notebooks in Mobu on any commit associated with a pull request!
+
+##### Configuration
+There is a `mobu CI (<env url>)` GitHub app for every non-production environment that runs Mobu (Except environments behind a VPN).
+For every environment in which you want to run your changed notebooks on every PR commit, you have to:
+*Install the app in your repo's organization, if it's not already installed*
+1. Install that environment's `mobu CI` app in your repo's GitHub organization. Someone with appropriate permissions can do that from the [app's homepage](#mobu-ci-github-app-urls)
+1. Add your organization to the `accetped_github_orgs` list in the Mobu configuration in Phalanx for the matching env
+
+*Enable the app for your repo*
+1. Go to your repo's organization settings page in GitHub, and go to the "GitHub Apps" page in the "Third-party Access" section in the left sidebar. [Here](https://github.com/organizations/lsst-sqre/settings/installations) is that page for the `lsst-sqre` organization
+1. Click the `Configure` button in the `mobu CI (<env url>)` row
+1. In the "Repository access" section, click the "Select repositories" dropdown and add your repository
+
+##### Optional Configuration
+You can have mobu ignore changed notebooks in certain directories in these CI checks by listing them in a file named `mobu.yaml` at the root of your repo, like this:
+
+```yaml
+exclude_dirs:
+- somedir
+- some/other/dir
+```
+With this configuration, no notebooks in the `somedir`, or `some/other/dir` directories, or any of their descendant directories, will be executed, even if they changed in the commit.
+
+##### Troubleshooting
+There is a small chance that a GitHub check could get stuck in a forever-in-progress state for a given commit if the stars align in a very specific way when mobu restarts.
+If this happens, you can push a commit to your branch, and it will start a new check run.
+If you don't have any actual changes to commit, you can push an empty commit like this:
+```
+git commit --allow-empty -m "Empty commit"
+```
+You can squash that commit later if you want a clean history.
+
+##### Mobu CI GitHub App URLs
+* [data-dev.lsst.cloud](https://github.com/apps/mobu-ci-data-dev-lsst-cloud)
+
+## Administrators
+
+### GitHub Integration
+Each integration has as GitHub application created in the [lsst-sqre org](https://github.com/organizations/lsst-sqre/settings/apps) for every environment in which it is enabled.
+All of the applications:
+* [mobu refresh (data-dev.lsst.cloud)](https://github.com/organizations/lsst-sqre/settings/apps/mobu-refresh-data-dev-lsst-cloud)
+* [mobu CI (data-dev.lsst.cloud)](https://github.com/organizations/lsst-sqre/settings/apps/mobu-ci-data-dev-lsst-cloud)
+
+### GitHub Application configuration
+To enable the GitHub integrations for another mobu env, you have to create a new GitHub application and sync Phalanx secrets.
+
+#### Refresh app
+*Create a new GitHub app*
+1. Click the `New GitHub App` button in the [lsst-sqre org Developer Settings apps page](https://github.com/organizations/lsst-sqre/settings/apps)
+1. Name it `mobu refresh (<env URL or id if the URL is too long>)`
+1. Make sure the `Active` checkbox is checked in the `Webhook` section
+1. Enter `https://<env URL>/mobu/github/refresh/webhook` in the `Webhook URL` input
+1. Generate a strong password to use as the webhook secret
+1. Store this in the `SQuaRE` vault in the `LSST IT` 1Password account in an item named `mobu refresh app webhook secret (<env URL>)`
+1. Get this into the Phalanx secret store for that env under the key: `github-refresh-app-webhook-secret` ([this process](https://phalanx.lsst.io/admin/add-new-secret.html) is different for different envs)
+1. Enter this secret in the `Webhook secret (optional)` box in the GitHub App config
+1. Select `Read-only` in the dropdown of the `Contents` access category in the `Repository Permissions` section
+1. Check the `Push` checkbox in the `Subscribe to events` section
+1. Select the `Any account` radio button in the `Where can this GitHub App be installed?` section
+1. Click the `Create GitHub App` button
+1. Do the [Phalanx configuration](#phalanx-configuration)
+
+*Install the app for a repo*
+1. Go to new app's homepage (something like https://github.com/apps/mobu-refresh-usdfdev)
+1. Click the `Install` button
+1. Select the `Only select repositories` radio button
+1. Select the repo in the dropdown
+1. Click `Install`
+
+#### CI app
+*Create a new GitHub app*
+1. Click the `New GitHub App` button in the [lsst-sqre org Developer Settings apps page](https://github.com/organizations/lsst-sqre/settings/apps)
+1. Name it `mobu CI (<env URL or id if the URL is too long>)`
+1. Make sure the `Active` checkbox is checked in the `Webhook` section
+1. Enter `https://<env URL>/mobu/github/ci/webhook` in the `Webhook URL` input
+1. Generate a strong password to use as the webhook secret
+1. Store this in the `SQuaRE` vault in the `LSST IT` 1Password account in an item named `mobu CI app webhook secret (<env URL>)`
+1. Get this into the Phalanx secret store for that env under the key: `github-ci-app-webhook-secret` ([this process](https://phalanx.lsst.io/admin/add-new-secret.html) is different for different envs)
+1. Enter this secret in the `Webhook secret (optional)` box in the GitHub App config
+1. Select `Read and Write` in the dropdown of the `Checks` access category in the `Repository Permissions` section
+1. Select `Read-only` in the dropdown of the `Contents` access category in the `Repository Permissions` section
+1. Check the `Check suite` and `Check run` checkboxes in the `Subscribe to events` section
+1. Select the `Any account` radio button in the `Where can this GitHub App be installed?` section
+1. Click the `Create GitHub App` button
+1. Find the `App ID` (an integer) in the `About` section. Get this into the Phalanx secret store for that env under the key: `github-ci-app-id` ([this process](https://phalanx.lsst.io/admin/add-new-secret.html) is different for different envs)
+1. Click the `Generate a private key` button in the `Private keys` section
+1. Store this private key in the `SQuaRE` vault in the `LSST IT` 1Password account in an item named `mobu CI app private key (<env URL>)`
+1. Get this into the Phalanx secret store for that env under the key: `github-ci-app-private-key` ([this process](https://phalanx.lsst.io/admin/add-new-secret.html) is different for different envs)
+1. Do the [Phalanx configuration](#phalanx-configuration)
+
+*Install the app for a repo*
+1. Go to new app's homepage (something like https://github.com/apps/mobu-refresh-usdfdev)
+1. Click the `Install` button
+1. Select the `Only select repositories` radio button
+1. Select the repo in the dropdown
+1. Click `Install`
+
+#### Phalanx Configuration
+The GitHub integrations each need to be explicitly enabled in Phalanx for a given environment.
+If an integration is not enabled, then the webhook route for that integration will not be mounted, GitHub webhook requests will get `404` responses.
+To enable these integrations for an environment, set these values to `true`:
+* `config.githubRefreshAppEnabled`
+* `config.githubCiAppEnabled`
+
+If you want to enable either GitHub integration in a given environment, you also need to add a `config.github` section to that env's values in Mobu.
+That needs to be a dict with at `users` and `accepted_github_orgs` entries.
+It should look something like this:
+
+```yaml
+config:
+  github:
+    accepted_github_orgs:
+      - lsst-sqre
+    users:
+      - username: "bot-mobu-ci-user-1"
+        uidnumber: 123
+        gidnumber: 456
+      - username: "bot-mobu-ci-user-2"
+        uidnumber: 789
+        gidnumber: 876
+```
+The organization of any repo that uses any of the GitHub integrations in an env must be added to the `accepted_github_orgs` list, otherwise Github webhook requests will get `403` responses.
+
+The `users` list follows the same rules as the `users` list in a flock autostart config.
+The usernames must all start with `bot-mobu`.
+In envs with Firestore integration, you only need to specify `username`.
+In envs without it, you need to ensure that users are manually provisioned, and then you need all three of `username`, `uidnumber`, and `gidnumber`.