From e3a25f1fe84834b87fde960080d430dd90d0da05 Mon Sep 17 00:00:00 2001 From: caetano melone Date: Tue, 30 Apr 2024 21:51:14 -0700 Subject: [PATCH] add some details about k8s deployments and how to run the test suite --- docs/deploy.md | 14 +++++++++++++- docs/readme.md | 5 +++-- docs/testing.md | 19 +++++++++++++++++++ 3 files changed, 35 insertions(+), 3 deletions(-) create mode 100644 docs/testing.md diff --git a/docs/deploy.md b/docs/deploy.md index fd17d0b..8559d3a 100644 --- a/docs/deploy.md +++ b/docs/deploy.md @@ -16,7 +16,7 @@ podman run -it -p 8080:8080 --env-file .env -v LOCAL_DB_PATH:DB_FILE gantry Where `LOCAL_DB_PATH` is the absolute path to the database file on your local system. `DB_FILE` is where you would like the application to access the database. Make sure this lines up with your environment. -When running Gantry within Kubernetes, you could use [persistent volumes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/). The only requirement is that the database should exist outside the container for backup and persistence purposes. +When running Gantry within Kubernetes, you could use [persistent volumes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/). The only requirement is that a copy of the database should exist outside the container for backup and persistence purposes. ## Environment @@ -28,3 +28,15 @@ The following variables should be exposed to the container. Those **bolded** are - **`GITLAB_API_TOKEN`** - this token should have API read access - **`GITLAB_WEBHOOK_TOKEN`** - coordinate this value with the collection webhook - **`DB_FILE`** - path where the application can access the SQLite file + +## Kubernetes + +Gantry can be deployed in a Kubernetes cluster. For an example of how we do this within Spack's web infrastructure, see [this folder](https://github.com/spack/spack-infrastructure/tree/main/k8s/production/spack/gantry-spack-io) containing configuration files and instructions. + +While most details are better suited to be documented with the cluster, there are some considerations that are general to any Gantry deployment. + +### Database + +We have made an architectural decision to depend on SQLite as the database engine. Before you deploy Gantry into a cluster, you should ensure that the file will be backed up on a regular basis, in the case that unexpected circumstances corrupt your data. This can be achieved using [Litestream](https://litestream.io), which will continuously replicate the database with the storage provider of your choice. See the cluster configuration linked above for details. + +When you first deploy the application on the cluster, be sure to run `/db/init_db.py ` to initialize an SQLite file with default tables, and to apply any migrations. diff --git a/docs/readme.md b/docs/readme.md index 7a4587a..b50f6f8 100644 --- a/docs/readme.md +++ b/docs/readme.md @@ -6,5 +6,6 @@ 2. [Data Collection](data-collection.md) 3. [Architecture](arch.md) 4. [Prediction](prediction.md) -5. [Deployment](deploy.md) -6. [API Endpoints](api.md) +5. [Testing](testing.md) +6. [Deployment](deploy.md) +7. [API Endpoints](api.md) diff --git a/docs/testing.md b/docs/testing.md new file mode 100644 index 0000000..2743c6b --- /dev/null +++ b/docs/testing.md @@ -0,0 +1,19 @@ +# Testing + +Gantry uses `pytest` to manage its testing suite. Make sure you have installed `pytest`, `pytest-aiohttp`, and `pytest-mock` in your environment. + +The tests cover the collection and prediction functionalities, as well as core functionalities like helper functions and database actions. + +Running all tests: + +``` +python -m pytest -s gantry +``` + +If you would like to see details about test coverage, install `coverage` and run: + +``` +coverage run -m pytest -s gantry && coverage html +``` + +This will create a folder in the top-level directory containing an `index.html` file which you can open in a browser.