diff --git a/CHANGELOG.md b/CHANGELOG.md index 6d972b8..a17ce5a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,7 +2,32 @@ All changes that impact users of this module are documented in this file, in the [Common Changelog](https://common-changelog.org) format with some additional specifications defined in the CONTRIBUTING file. This codebase adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). -## Unreleased +## Unreleased [major] + +_Full changeset and discussions: [#41](https://github.com/OpenTermsArchive/deployment/pull/41)._ + +> Development of this release was supported by the [French Ministry for Foreign Affairs](https://www.diplomatie.gouv.fr/fr/politique-etrangere-de-la-france/diplomatie-numerique/) through its ministerial [State Startups incubator](https://beta.gouv.fr/startups/open-terms-archive.html) under the aegis of the Ambassador for Digital Affairs. + +### Changed + +- **Breaking:** Replace all playbooks with a single `deploy` playbook; update your scripts by using `ansible-playbook opentermsarchive.deployment.deploy` and use [tags to refine execution](./README.md#refining-playbook-execution) +- **Breaking:** Change the `federation-api` deployment process to be a dependency of the source repository +- **Breaking:** Require `engine>=2` and `federation-api>=v2` +- **Breaking:** Make `ota_source_repository` variable mandatory +- **Breaking:** Define environment variables in a `.env` file instead of in inventory variables; move `ota_engine_github_token`, `ota_engine_smtp_password`, `ota_engine_sendinblue_api_key`, and `ota_federated_api_smtp_password` to a `.env` file placed next to the inventory file and rename them according to the new naming conventions in `engine` v2 and `federation-api` v2 +- **Breaking:** Rename `ota_engine_declarations_branch` to `ota_source_repository_branch` +- **Breaking:** Rename `ota_engine_declarations_directory` to `ota_directory` +- **Breaking:** Extract the PM2 config file `pm2.config.cjs`; provide this file next to the inventory file +- **Breaking:** Remove the `ota_engine_restart_delay` variable; define this setting directly in the `pm2.config.cjs` +- **Breaking:** Remove the `ota_engine_github_bot_private_key` variable in favor of the `github-bot-private-key` file; define the GitHub SSH private key in the `github-bot-private-key` + +### Removed + +- **Breaking:** Remove `ota_reverse_proxy_federated_api_path` config; define this path in your `config/production.json` under the key `@opentermsarchive/federation-api: { basePath: "" }` +- **Breaking:** Remove `ota_reverse_proxy_engine_path` config; define this path in your `config/production.json` under the key `@opentermsarchive/engine: { collection-api: { basePath: "" } }` +- **Breaking:** Remove obsolete `ota_engine_config_path`, `ota_federated_api_branch`, `ota_federated_api_directory`, and `ota_federated_api_repo` configs +- **Breaking:** Remove obsolete `update-declarations` tag + ## 1.2.1 - 2024-05-22 diff --git a/README.md b/README.md index 1f684d1..f7cb3fb 100644 --- a/README.md +++ b/README.md @@ -2,11 +2,13 @@ This repository contains the `opentermsarchive.deployment` Ansible collection. This Ansible collection provides playbooks to set up the infrastructure of and deploy Open Terms Archive applications. +To prevent confusion between the notion of Ansible [Collection](https://docs.ansible.com/ansible/latest/collections_guide/index.html) and an Open Terms Archive [Collection](https://docs.opentermsarchive.org/#collection), this documentation will refer to Ansible Collection only as “Playbook”, as this is the main entry point to interact with it. + ## Installation -[Ansible](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html) is required to use this collection. +[Ansible](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html) is required to use this playbook. -This collection can be installed from Ansible Galaxy manually with the `ansible-galaxy` command-line tool: +This playbook can be installed from Ansible Galaxy manually with the `ansible-galaxy` command-line tool: ```sh ansible-galaxy collection install opentermsarchive.deployment @@ -28,200 +30,113 @@ ansible-galaxy collection install -r requirements.yml ## Usage -Once installed, some playbooks are available to deploy the two main Open Terms Archive applications: [Engine](https://github.com/OpenTermsArchive/engine) and [Federated API](https://github.com/OpenTermsArchive/federated-api). +Once installed, the playbook `deploy` allows to set up the two main Open Terms Archive applications: [Engine](https://github.com/OpenTermsArchive/engine) and [Federation API](https://github.com/OpenTermsArchive/federation-api). -Each playbook can be executed using the `ansible-playbook` command-line tool: +The playbook can be executed using the `ansible-playbook` command-line tool: ```sh -ansible-playbook opentermsarchive.deployment. +ansible-playbook opentermsarchive.deployment.deploy ``` -Refer to the application related sections below for a list of available playbooks. - -_It is possible to check a playbook execution without actually applying changes with `check` and `diff` options:_ +_It is possible to check the playbook execution without actually applying changes with the `check` and `diff` options:_ ```sh -ansible-playbook opentermsarchive.deployment. --check --diff +ansible-playbook opentermsarchive.deployment.deploy --check --diff ``` > See “[Using collections](https://docs.ansible.com/ansible/latest/user_guide/collections_using.html)” in Ansible’s user guide for more information about Ansible collections. -- - - - -### Engine application - -Available playbooks for the engine application: - -| Playbook name | Description | Command example | -| --- | --- | --- | -| `engine.infrastructure` | Set up and configure the infrastructure required by the Open Terms Archive engine | `ansible-playbook opentermsarchive.deployment.engine.infrastructure` | -| `engine.application` | Deploy the Open Terms Archive engine | `ansible-playbook opentermsarchive.deployment.engine.application` | -| `engine.all` | Set up infrastructure and deploy the Open Terms Archive engine | `ansible-playbook opentermsarchive.deployment.engine.all` | +## Configuration -#### Configuration +Configuration is done through various files located in the `deployment` folder. Below is an example of the directory structure: -Available [variables](https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_variables.html) are listed below, along with default values: - -| Variable | Description | Default value | Required | -| --- | --- | --- | --- | -| `ota_engine_github_bot_private_key` | SSH private key contents for GitHub user with privileges on snapshots and versions repositories | No default value | ✔︎ | -| `ota_engine_github_token` | GitHub token to enable issue creation on the declarations repository and publish releases on versions repository | No default value | ✔︎ | -| `ota_engine_smtp_password` | Password for the SMTP server used for sending error notifications by email | No default value | - | -| `ota_engine_sendinblue_api_key` | SendInBlue API key used to send email notifications | No default value | - | -| `ota_engine_config_path` | Path to the engine config file, relative to the inventory file | `../config/production.json` | - | -| `ota_engine_declarations_branch` | [Git branch or tag](https://git-scm.com/docs/gitglossary#Documentation/gitglossary.txt-aiddeftree-ishatree-ishalsotreeish) of the declarations repository to use | `main` | - | -| `ota_engine_snapshots_branch` | [Git branch or tag](https://git-scm.com/docs/gitglossary#Documentation/gitglossary.txt-aiddeftree-ishatree-ishalsotreeish) of the snapshots repository to use | `main` | - | -| `ota_engine_versions_branch` | [Git branch or tag](https://git-scm.com/docs/gitglossary#Documentation/gitglossary.txt-aiddeftree-ishatree-ishalsotreeish) of the versions repository to use | `main` | - | -| `ota_engine_declarations_directory` | Path of the directory where the code will be deployed on the server | Value declared in the `name` key in the engine config file | - | -| `ota_engine_restart_delay` | Delay, in milliseconds, before restarting the engine after a crash | `10800000` _(3 hours)_ | - | - -For encryption of sensitive configuration entries, please refer to the [dedicated section](#encrypt-sensitive-configuration-entries). - -These variables can be overriden in the inventory file, for example: - -```yml -all: - hosts: - 127.0.0.1: - ansible_user: debian - ota_engine_config_path: ./engine_config.json - ota_engine_declarations_branch: new-feature +```plaintext +deployment/ + ├── inventory.yml + ├── pm2.config.cjs + ├── github-bot-private-key + └── .env ``` -#### Tags - -Available [tags](https://docs.ansible.com/ansible/latest/user_guide/playbooks_tags.html) to refine what will happen, use them with `--tags`: - -| Tag | Description | Command example | -| --- | --- | --- | -| `start` | Start the engine | `ansible-playbook opentermsarchive.deployment.engine.application --tags start` | -| `stop` | Stop the engine | `ansible-playbook opentermsarchive.deployment.engine.application --tags stop` | -| `restart` | Restart the engine | `ansible-playbook opentermsarchive.deployment.engine.application --tags restart` | -| `update-declarations` | Update service declarations (pull declarations, install dependencies, and restart engine) | `ansible-playbook opentermsarchive.deployment.engine.application --tags update-declarations` | - -- - - - -### Federated API application - -Available playbooks for the Federated API application: - -| Playbook name | Description | Command example | -| --- | --- | --- | -| `federated_api.infrastructure` | Set up and configure the infrastructure required by the Open Terms Archive federated API | `ansible-playbook opentermsarchive.deployment.federated_api.infrastructure` | -| `federated_api.application` | Deploy the Open Terms Archive federated API | `ansible-playbook opentermsarchive.deployment.federated_api.application` | -| `federated_api.all` | Set up infrastructure and deploy the Open Terms Archive federated API | `ansible-playbook opentermsarchive.deployment.federated_api.all` | +- ### Inventory File — `inventory.yml` -#### Configuration +**This file is mandatory** -Available variables are listed below, along with default values: +The `inventory.yml` file defines the hosts and the variables required for the deployment. This file should contain all the necessary variables as described below. -| Variable | Description | Default value | Required | -| --- | --- | --- | --- | -| `ota_federated_api_repo` | Repository URL of the federated API code | `https://github.com/OpenTermsArchive/federated-api.git` | - | -| `ota_federated_api_directory` | Path of the directory where the code will be deployed on the server | `federated-api` | - | -| `ota_federated_api_branch` | [Git branch or tag](https://git-scm.com/docs/gitglossary#Documentation/gitglossary.txt-aiddeftree-ishatree-ishalsotreeish) of the federated API repository to use | `main` | - | -| `ota_federated_api_smtp_password` | Password for the SMTP server used for sending errors notifications by email. | - | - | +| Variable | Description | Required or default value | +|--------------------------------|----------------------------------------------------------------------------------------------------------------------------|------------------------| +| `ota_source_repository` | URL of the declarations repository to deploy | **required** | +| `ota_source_repository_branch` | [Git branch or tag](https://git-scm.com/docs/gitglossary#Documentation/gitglossary.txt-aiddeftree-ishatree-ishalsotreeish) of the source repository | `main` | +| `ota_directory` | Directory path where the code will be deployed on the server | Name of the repository | -For encryption of sensitive configuration entries, please refer to the [dedicated section](#encrypt-sensitive-configuration-entries). - -These variables can be overridden in the inventory file, for example: +These variables are defined in the inventory file, for example: ```yml all: hosts: 127.0.0.1: ansible_user: debian - ota_federated_api_repo: https://github.com/OpenTermsArchive/federated-api.git - ota_federated_api_branch: new-feature + ota_source_repository: https://github.com/OpenTermsArchive/demo-declarations.git + ota_source_repository_branch: master + ota_directory: demo ``` -#### Tags - -Available [tags](https://docs.ansible.com/ansible/latest/user_guide/playbooks_tags.html) to refine what will happen, use them with `--tags`: +- ### PM2 Configuration File — `pm2.config.cjs` -| Tag | Description | Command example | -| --- | --- | --- | -| `start` | To only start the Federated API | `ansible-playbook opentermsarchive.deployment.federated_api.application --tags start` | -| `stop` | To only stop the Federated API | `ansible-playbook opentermsarchive.deployment.federated_api.application --tags stop` | -| `restart` | To only restart the Federated API | `ansible-playbook opentermsarchive.deployment.federated_api.application --tags restart` | +**This file is mandatory** -- - - +The `pm2.config.cjs` file is used to configure the [PM2](https://pm2.keymetrics.io) process manager, which is used to start the applications. -### Engine and Federated API applications +- ### GitHub Bot Private Key — `github-bot-private-key` -Available playbooks to deploy both the Open Terms Archive Engine and Federated API applications on a single server. +The `github-bot-private-key` file contains a private SSH key for accessing and pushing to SSH Git URLs. This file is required if `ota_source_repository` is an SSH Git URL or if the URLs for versions and/or snapshots repositories in the `config/production.json` file of the source repository are SSH Git URLs. -| Playbook name | Description | Command example | -| --- | --- | --- | -| `engine_and_federated_api.infrastructure` | Set up and configure the infrastructure required by the Open Terms Archive engine and federated API applications | `ansible-playbook opentermsarchive.deployment.engine_and_federated_api.infrastructure` | -| `engine_and_federated_api.application` | Deploy the Open Terms Archive engine and federated API applications | `ansible-playbook opentermsarchive.deployment.engine_and_federated_api.application` | -| `engine_and_federated_api.all` | Set up infrastructure and deploy the Open Terms Archive engine and federated API applications | `ansible-playbook opentermsarchive.deployment.engine_and_federated_api.all` | +It is strongly recommended to [encrypt this file](#file-encryption) if is is checked in to a public repository. -#### Configuration +- ### Environment Variables File — `.env` -Available variables are listed below, along with default values: +The `.env` file contains the environment variables for the deployed applications. -| Variable | Description | Default value | Required | -| --- | --- | --- | --- | -| `ota_reverse_proxy_engine_path` | Path where the collection API embed with the engine will be available | `/collection-api` | - | -| `ota_reverse_proxy_federated_api_path` | Path where the federated API will be available | `/federation-api` | - | +It is strongly recommended to [encrypt this file](#file-encryption) if is is checked in to a public repository. -- - - +## File encryption -## Encrypt sensitive configuration entries +Sensitive configuration files should be encrypted using [Ansible Vault](https://docs.ansible.com/ansible/latest/vault_guide/index.html). -Certain configuration entries contain sensitive information that should be encrypted to ensure security. Ansible provides a convenient way to encrypt such strings using its built-in [vault feature](https://docs.ansible.com/ansible/2.9/user_guide/vault.html): +Examples of encrypting and decrypting a file: -```sh -ansible-vault encrypt_string --name -``` +- Encrypt the `github-bot-private-key` file: `ansible-vault encrypt github-bot-private-key` +- Decrypt the `github-bot-private-key` file: `ansible-vault decrypt github-bot-private-key` +- Encrypting with a password stored in a file: + - `echo 'your_password' > vault.key` + - `ansible-vault encrypt --vault-password-file vault.key github-bot-private-key` -For example, to encrypt the GitHub bot private key used by the engine to push updates: +To run the playbook with encrypted files: ```sh -ansible-vault encrypt_string --name 'ota_engine_github_bot_private_key' '-----BEGIN OPENSSH PRIVATE KEY----- -b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAABlwAAAAdzc2gtcn -… -UlcCkBZ5IkI0eNAAAAE25kcG50QE1CUC1OZHBudC5sYW4BAgMEBQYH ------END OPENSSH PRIVATE KEY----- -' +ansible-playbook playbook.yml --ask-vault-pass ``` -The encrypted result will look like this: +Or with a password file: ```sh -ota_engine_github_bot_private_key: !vault | - $ANSIBLE_VAULT;1.1;AES256 - 62313438616266383732353634343736623532666365643364396464633732613966636235616261 - 3136656665316437613434323561613732373361306161640a306132316531356537373862363838 - 66363763613833373530633831653163303961376331393761366261633561656463626563383931 - 3361643836623239660a333134626139626465303234313366313433653261376437316231363834 - 32643261303534366333383131633430396366343631656363663965633964663331346231663166 - 3331316462356461373134303666613035393335333139613639 +ansible-playbook playbook.yml --vault-password-file vault.key ``` -Then it can be used directly in the inventory file: +Please note that encrypted files will be decrypted and stored in plaintext on the deployment server. Always protect access to your production server. -```yml -all: - hosts: - 127.0.0.1: - ansible_user: debian - ota_engine_config_path: ./engine_config.json - ota_engine_declarations_branch: new-feature - ota_engine_github_bot_private_key: !vault | - $ANSIBLE_VAULT;1.1;AES256 - 62313438616266383732353634343736623532666365643364396464633732613966636235616261 - 3136656665316437613434323561613732373361306161640a306132316531356537373862363838 - 66363763613833373530633831653163303961376331393761366261633561656463626563383931 - 3361643836623239660a333134626139626465303234313366313433653261376437316231363834 - 32643261303534366333383131633430396366343631656363663965633964663331346231663166 - 3331316462356461373134303666613035393335333139613639 -``` +## Playbook execution refinement -Repeat the process for each sensitive configuration entry that needs encryption. +Use [tags](https://docs.ansible.com/ansible/latest/user_guide/playbooks_tags.html) to refine playbook execution. Example commands: -Please note that the data will be stored unencrypted on the deployment server. +| Tag | Description | Command Example | +|-----------------|-----------------------------------------|----------------------------------------------------------------------------------| +| `start` | Start Open Terms Archive applications | `ansible-playbook opentermsarchive.deployment.deploy --tags start` | +| `stop` | Stop Open Terms Archive applications | `ansible-playbook opentermsarchive.deployment.deploy --tags stop` | +| `restart` | Restart Open Terms Archive applications | `ansible-playbook opentermsarchive.deployment.deploy --tags restart` | +| `infrastructure`| Set up the infrastructure only | `ansible-playbook opentermsarchive.deployment.deploy --tags infrastructure` | +| `infrastructure`| Skip the infrastructure | `ansible-playbook opentermsarchive.deployment.deploy --skip-tags infrastructure` | - - - @@ -258,7 +173,7 @@ Then the code can be deployed to the running machine with all the options descri Testing the Ansible collection locally is crucial to ensure that changes function properly before submitting them as a pull request. -The testing environment is preconfigured for Open Terms Archive maintainers. For other contributors, the configuration file `tests/engine_config.json` needs to be updated to specify repositories where they have authorizations. Additionally, the `ota_engine_github_bot_private_key` value in the inventory file `tests/inventory.yml` should be updated. +The testing environment is preconfigured for Open Terms Archive maintainers. For other contributors, the inventory file `tests/inventory.yml` needs to be updated to specify repositories where they have authorizations. Additionally, the `github-bot-private-key` file should be updated. Follow these instructions to test the collection in a local environment: @@ -270,7 +185,7 @@ vagrant up - Apply the changes to the virtual machine: ```sh -ansible-playbook ../playbooks/engine/all.yml +ansible-playbook ../playbooks/deploy.yml ``` - Connect to the virtual machine to verify that changes were applied successfully: @@ -283,6 +198,22 @@ vagrant ssh # use "vagrant" as password pm2 logs ``` +#### Troubleshooting + +If you encounter a `Permission denied` error at the “Gathering Facts” step of the playbook, this most likely means the Vagrant-managed SSH key is not loaded in your SSH agent. + +1. Identify which SSH key is used by Vagrant: + +```sh +vagrant ssh-config | grep IdentityFile +``` + +2. Add the private key to your SSH agent: + +```sh +ssh-add /path/to/your/vagrant/private_key +``` + --- ## License diff --git a/playbooks/deploy.yml b/playbooks/deploy.yml new file mode 100644 index 0000000..3a79e11 --- /dev/null +++ b/playbooks/deploy.yml @@ -0,0 +1,108 @@ +--- +- name: Deploy Open Terms Archive applications + hosts: all + tasks: + - name: Load OTA applications configs + ansible.builtin.include_role: + name: ota/apps + public: true # ensure that the role's variables and defaults are accessible to the play + vars: + ota_apps_read_config_only: true + + - name: Set required variables + set_fact: + chromium_required: "{{ ota_apps_config['@opentermsarchive/engine'] is defined }}" + # Skip Debian 11 with ARM architecture as it is not currently supported by MongoDB; see https://www.mongodb.com/docs/manual/installation/#supported-platforms + mongo_required: + "{{ + (ansible_distribution != 'Debian' or (ansible_distribution == 'Debian' and ansible_facts['architecture'] != 'aarch64')) + and ( + (ota_apps_config['@opentermsarchive/engine'].recorder.versions.storage.type is defined and ota_apps_config['@opentermsarchive/engine'].recorder.versions.storage.type == 'mongo') + or + (ota_apps_config['@opentermsarchive/engine'].recorder.snapshots.storage.type is defined and ota_apps_config['@opentermsarchive/engine'].recorder.snapshots.storage.type == 'mongo') + ) | bool + }}" + + snapshots_repository: "{{ ota_apps_config['@opentermsarchive/engine'].recorder.snapshots.storage.git.repository is defined and ota_apps_config['@opentermsarchive/engine'].recorder.snapshots.storage.git.repository }}" + snapshots_path: "{{ ota_apps_config['@opentermsarchive/engine'].recorder.snapshots.storage.git.path is defined and ota_apps_config['@opentermsarchive/engine'].recorder.snapshots.storage.git.path }}" + + versions_repository: "{{ ota_apps_config['@opentermsarchive/engine'].recorder.versions.storage.git.repository is defined and ota_apps_config['@opentermsarchive/engine'].recorder.versions.storage.git.repository }}" + versions_path: "{{ ota_apps_config['@opentermsarchive/engine'].recorder.versions.storage.git.path is defined and ota_apps_config['@opentermsarchive/engine'].recorder.versions.storage.git.path }}" + + collection_api_basePath: "{{ ota_apps_config['@opentermsarchive/engine']['collection-api'].basePath is defined and ota_apps_config['@opentermsarchive/engine']['collection-api'].basePath }}" + collection_api_port: "{{ ota_apps_config['@opentermsarchive/engine']['collection-api'].port is defined and ota_apps_config['@opentermsarchive/engine']['collection-api'].port }}" + + federation_api_basePath: "{{ ota_apps_config['@opentermsarchive/federation-api'].basePath is defined and ota_apps_config['@opentermsarchive/federation-api'].basePath }}" + federation_api_port: "{{ ota_apps_config['@opentermsarchive/federation-api'].port is defined and ota_apps_config['@opentermsarchive/federation-api'].port }}" + + - name: Install infrastructure + become: true + tags: [infrastructure] + block: + - name: Install Node + ansible.builtin.include_role: + name: node + + - name: Install PM2 + ansible.builtin.include_role: + name: pm2/install + + - name: Install Chromium + ansible.builtin.include_role: + name: chromium + when: chromium_required + + - name: Install Nginx + ansible.builtin.include_role: + name: nginx/install + + - name: Install Mongo + ansible.builtin.include_role: + name: mongo/install + when: + - mongo_required + + - name: Configure Mongo + ansible.builtin.include_role: + name: mongo/configure + apply: + become: true + when: + - mongo_required + + - name: Setup Git-based versions database + ansible.builtin.include_role: + name: ota/git-database + vars: + ota_git_database_repository: "{{ versions_repository }}" + ota_git_database_directory: "{{ versions_path }}" + ota_git_database_branch: main + when: + - versions_repository and versions_path + + - name: Setup Git-based snapshots database + ansible.builtin.include_role: + name: ota/git-database + vars: + ota_git_database_repository: "{{ snapshots_repository }}" + ota_git_database_directory: "{{ snapshots_path }}" + ota_git_database_branch: main + when: + - snapshots_repository and snapshots_path + + - name: Setup OTA applications + ansible.builtin.include_role: + name: ota/apps + + - name: Start OTA applications + ansible.builtin.include_role: + name: pm2/manage + + - name: Configure NGINX + ansible.builtin.include_role: + name: nginx/configure + apply: + become: true + vars: + ota_nginx_config_template: ./templates/nginx.conf.j2 + ota_nginx_reverse_proxy_config_template: ./templates/nginx-reverse-proxy-conf.j2 diff --git a/playbooks/engine/all.yml b/playbooks/engine/all.yml deleted file mode 100644 index f361d0d..0000000 --- a/playbooks/engine/all.yml +++ /dev/null @@ -1,6 +0,0 @@ ---- -- name: Set up infrastructure and deploy the Open Terms Archive engine - hosts: all - -- import_playbook: infrastructure.yml -- import_playbook: application.yml diff --git a/playbooks/engine/application.yml b/playbooks/engine/application.yml deleted file mode 100644 index 4bdad1d..0000000 --- a/playbooks/engine/application.yml +++ /dev/null @@ -1,14 +0,0 @@ ---- -- name: Deploy the Open Terms Archive engine - hosts: all - - tasks: - - name: Load the production config - ansible.builtin.include_vars: - name: ota_engine_app_config - file: "{{ inventory_dir }}/{{ ota_engine_config_path | default('../config/production.json') }}" - tags: always - - - ansible.builtin.include_role: - name: engine - tags: always diff --git a/playbooks/engine/infrastructure.yml b/playbooks/engine/infrastructure.yml deleted file mode 100644 index 5b1c32c..0000000 --- a/playbooks/engine/infrastructure.yml +++ /dev/null @@ -1,25 +0,0 @@ ---- -- name: Set up infrastructure - hosts: all - become: true - - tasks: - - name: Load OTA engine config - ansible.builtin.include_vars: - name: app_config - file: "{{ inventory_dir }}/{{ ota_engine_config_path | default('../config/production.json') }}" - tags: always - - - ansible.builtin.include_role: - name: infrastructure/mongo - when: - - (app_config.recorder.versions.storage.type is defined and app_config.recorder.versions.storage.type == 'mongo') or - (app_config.recorder.snapshots.storage.type is defined and app_config.recorder.snapshots.storage.type == 'mongo') - # Skip Debian 11 with ARM architecture as it is not currently supported by MongoDB. - # See https://www.mongodb.com/docs/manual/installation/#supported-platforms - - ansible_distribution != 'Debian' or (ansible_distribution == 'Debian' and ansible_facts['architecture'] != 'aarch64') - - roles: - - role: infrastructure/node - - role: infrastructure/chromium - - role: infrastructure/nginx diff --git a/playbooks/engine_and_federated_api/all.yml b/playbooks/engine_and_federated_api/all.yml deleted file mode 100644 index 952cef7..0000000 --- a/playbooks/engine_and_federated_api/all.yml +++ /dev/null @@ -1,6 +0,0 @@ ---- -- name: Set up infrastructure and deploy the Open Terms Archive engine and federated API - hosts: all - -- import_playbook: infrastructure.yml -- import_playbook: application.yml diff --git a/playbooks/engine_and_federated_api/application.yml b/playbooks/engine_and_federated_api/application.yml deleted file mode 100644 index dc3b46b..0000000 --- a/playbooks/engine_and_federated_api/application.yml +++ /dev/null @@ -1,43 +0,0 @@ ---- -- name: Deploy the Open Terms Archive engine and federated API - hosts: all - vars: - ota_reverse_proxy_engine_path: "/collection-api" - ota_reverse_proxy_federated_api_path: "/federation-api" - tasks: - - block: - - name: Load the engine production config - ansible.builtin.include_vars: - name: ota_engine_app_config - file: "{{ inventory_dir }}/{{ ota_engine_config_path | default('../config/production.json') }}" - - - ansible.builtin.include_role: - name: engine - - - ansible.builtin.include_role: - name: federated_api - tags: always - - - block: - - name: Add conf in NGINX sites-available - ansible.builtin.template: - src: nginx-conf.j2 - dest: '/etc/nginx/sites-available/ota' - force: true - mode: "644" - - - name: Link conf from sites-available to sites-enabled - ansible.builtin.file: - src: '/etc/nginx/sites-available/ota' - dest: '/etc/nginx/sites-enabled/ota' - state: link - force: true - become: true - notify: Restart NGINX - - handlers: - - name: Restart NGINX - become: true - ansible.builtin.service: - name: nginx - state: restarted diff --git a/playbooks/engine_and_federated_api/infrastructure.yml b/playbooks/engine_and_federated_api/infrastructure.yml deleted file mode 100644 index 52b6fd6..0000000 --- a/playbooks/engine_and_federated_api/infrastructure.yml +++ /dev/null @@ -1,7 +0,0 @@ ---- -- name: Set up infrastructure - hosts: all - become: true - -- ansible.builtin.import_playbook: ../engine/infrastructure.yml -- ansible.builtin.import_playbook: ../federated_api/infrastructure.yml diff --git a/playbooks/federated_api/all.yml b/playbooks/federated_api/all.yml deleted file mode 100644 index d4a58bc..0000000 --- a/playbooks/federated_api/all.yml +++ /dev/null @@ -1,6 +0,0 @@ ---- -- name: Set up infrastructure and deploy the Open Terms Archive federated API - hosts: all - -- import_playbook: infrastructure.yml -- import_playbook: application.yml diff --git a/playbooks/federated_api/application.yml b/playbooks/federated_api/application.yml deleted file mode 100644 index 559efdb..0000000 --- a/playbooks/federated_api/application.yml +++ /dev/null @@ -1,6 +0,0 @@ ---- -- name: Deploy the Open Terms Archive federated API - hosts: all - - roles: - - role: federated_api diff --git a/playbooks/federated_api/infrastructure.yml b/playbooks/federated_api/infrastructure.yml deleted file mode 100644 index f67c8fe..0000000 --- a/playbooks/federated_api/infrastructure.yml +++ /dev/null @@ -1,8 +0,0 @@ ---- -- name: Set up infrastructure - hosts: all - become: true - - roles: - - role: infrastructure/node - - role: infrastructure/nginx diff --git a/playbooks/engine_and_federated_api/templates/nginx-conf.j2 b/playbooks/templates/nginx-reverse-proxy-conf.j2 similarity index 67% rename from playbooks/engine_and_federated_api/templates/nginx-conf.j2 rename to playbooks/templates/nginx-reverse-proxy-conf.j2 index b356620..5a452fb 100644 --- a/playbooks/engine_and_federated_api/templates/nginx-conf.j2 +++ b/playbooks/templates/nginx-reverse-proxy-conf.j2 @@ -3,20 +3,22 @@ server { listen 80; server_name {{ inventory_hostname }}; - - location {{ ota_reverse_proxy_engine_path }} { + +{% if collection_api_basePath and collection_api_port %} + location {{ collection_api_basePath }} { # Allowing for a `burst` of up to 5 requests beyond the specified rate limit. The `nodelay` parameter ensures that excessive requests beyond the burst limit are immediately rejected with a 429 error response instead of being queued. See https://www.nginx.com/blog/rate-limiting-nginx/. limit_req zone=limited burst=5 nodelay; - rewrite ^{{ ota_reverse_proxy_engine_path }}/(.*)$ /$1 break; - proxy_pass http://localhost:{{ ota_engine_app_config.api.port }}; + proxy_pass http://localhost:{{ collection_api_port }}; proxy_redirect off; } +{% endif %} - location {{ ota_reverse_proxy_federated_api_path }} { +{% if federation_api_basePath and federation_api_port %} + location {{ federation_api_basePath }} { # Allowing for a `burst` of up to 5 requests beyond the specified rate limit. The `nodelay` parameter ensures that excessive requests beyond the burst limit are immediately rejected with a 429 error response instead of being queued. See https://www.nginx.com/blog/rate-limiting-nginx/. limit_req zone=limited burst=5 nodelay; - rewrite ^{{ ota_reverse_proxy_federated_api_path }}/(.*)$ /$1 break; - proxy_pass http://localhost:{{ ota_federated_api_app_config.port }}; + proxy_pass http://localhost:{{ federation_api_port }}; proxy_redirect off; } +{% endif %} } diff --git a/roles/infrastructure/nginx/templates/nginx.conf.j2 b/playbooks/templates/nginx.conf.j2 similarity index 100% rename from roles/infrastructure/nginx/templates/nginx.conf.j2 rename to playbooks/templates/nginx.conf.j2 diff --git a/roles/chromium/meta/main.yml b/roles/chromium/meta/main.yml new file mode 100644 index 0000000..6b4fff8 --- /dev/null +++ b/roles/chromium/meta/main.yml @@ -0,0 +1,3 @@ +--- +dependencies: + - { role: common } diff --git a/roles/infrastructure/chromium/tasks/main.yml b/roles/chromium/tasks/main.yml similarity index 81% rename from roles/infrastructure/chromium/tasks/main.yml rename to roles/chromium/tasks/main.yml index 527444d..d40fef3 100644 --- a/roles/infrastructure/chromium/tasks/main.yml +++ b/roles/chromium/tasks/main.yml @@ -15,7 +15,7 @@ state: latest when: ansible_distribution == 'Ubuntu' - # See https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md#recommended-enable-user-namespace-cloning +# See https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md#recommended-enable-user-namespace-cloning - name: Enable user namespace cloning to allow running Chromium in a sandbox ansible.builtin.command: sysctl -w kernel.unprivileged_userns_clone=1 when: ansible_facts['architecture'] != 'aarch64' diff --git a/roles/infrastructure/common/tasks/main.yml b/roles/common/tasks/main.yml similarity index 93% rename from roles/infrastructure/common/tasks/main.yml rename to roles/common/tasks/main.yml index dc39076..b53c389 100644 --- a/roles/infrastructure/common/tasks/main.yml +++ b/roles/common/tasks/main.yml @@ -10,3 +10,4 @@ - zip update_cache: true state: latest + become: true diff --git a/roles/engine/README.md b/roles/engine/README.md deleted file mode 100644 index efebbdb..0000000 --- a/roles/engine/README.md +++ /dev/null @@ -1,2 +0,0 @@ -# Engine - diff --git a/roles/engine/defaults/main.yml b/roles/engine/defaults/main.yml deleted file mode 100644 index 3b0782b..0000000 --- a/roles/engine/defaults/main.yml +++ /dev/null @@ -1,5 +0,0 @@ -ota_engine_declarations_directory: "{{ ota_engine_app_config.name }}" -ota_engine_declarations_branch: main -ota_engine_snapshots_branch: main -ota_engine_versions_branch: main -ota_engine_restart_delay: 10800000 # 3 hours in ms diff --git a/roles/engine/handlers/main.yml b/roles/engine/handlers/main.yml deleted file mode 100644 index 347479f..0000000 --- a/roles/engine/handlers/main.yml +++ /dev/null @@ -1,6 +0,0 @@ ---- -- name: Restart NGINX - become: true - ansible.builtin.service: - name: nginx - state: restarted diff --git a/roles/engine/tasks/database.yml b/roles/engine/tasks/database.yml deleted file mode 100644 index d9047a3..0000000 --- a/roles/engine/tasks/database.yml +++ /dev/null @@ -1,40 +0,0 @@ -- name: Check if {{ engine_database_name }} base data has already been obtained from {{ engine_database_repository }} - ansible.builtin.git: - repo: '{{ engine_database_repository }}' - version: '{{ engine_database_branch }}' - dest: '{{ engine_database_directory }}' - clone: false - update: false - key_file: '/home/{{ ansible_user }}/.ssh/ota-github-bot-key' - # the `before` property of the return value can tell us if the repository has been cloned already or not, - # see - register: existing_repository - -- name: Obtain {{ engine_database_name }} initial data from branch {{ engine_database_branch }} of {{ engine_database_repository }} - ansible.builtin.git: - repo: '{{ engine_database_repository }}' - version: '{{ engine_database_branch }}' - dest: '{{ engine_database_directory }}' - key_file: '/home/{{ ansible_user }}/.ssh/ota-github-bot-key' - when: existing_repository.before is defined and not existing_repository.before # if existing_repository.before is null, then the repository is new - -- name: Configure {{ engine_database_name }} repository to use the GitHub Bot SSH Key - ansible.builtin.git_config: - name: core.sshCommand - value: ssh -i '/home/{{ ansible_user }}/.ssh/ota-github-bot-key' - scope: local - repo: '{{ engine_database_directory }}' - -- name: Remove existing locks in {{ engine_database_name }} - ansible.builtin.file: - path: '{{ engine_database_directory }}/.git/index.lock' - state: absent - -- name: Get latest data from {{ engine_database_repository }} - ansible.builtin.git: - repo: '{{ engine_database_repository }}' - version: '{{ engine_database_branch }}' - dest: '{{ engine_database_directory }}' - update: true - force: true # discard any modified files - key_file: '/home/{{ ansible_user }}/.ssh/ota-github-bot-key' diff --git a/roles/engine/tasks/main.yml b/roles/engine/tasks/main.yml deleted file mode 100644 index d52fe15..0000000 --- a/roles/engine/tasks/main.yml +++ /dev/null @@ -1,104 +0,0 @@ -- name: Add GitHub Bot SSH key - ansible.builtin.copy: - content: '{{ ota_engine_github_bot_private_key }}' - dest: '/home/{{ ansible_user }}/.ssh/ota-github-bot-key' - owner: '{{ ansible_user }}' - group: '{{ ansible_user }}' - mode: "600" - -- name: Add GitHub SSH key to known_hosts - ansible.builtin.known_hosts: - name: github.com - key: "{{ item }}" - path: ~/.ssh/known_hosts - with_items: # GitHub's SSH key fingerprints can be found here: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/githubs-ssh-key-fingerprints - - "github.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOMqqnkVzrm0SdG6UOoqKLsabgH5C9okWi0dh2l9GKJl" - - "github.com ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEmKSENjQEezOmxkZMy7opKgwFB9nkt5YRrYMjNuG5N87uRgg6CLrbo5wAdT/y6v0mKV0U2w0WZ2YB/++Tpockg=" - - "github.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCj7ndNxQowgcQnjshcLrqPEiiphnt+VTTvDP6mHBL9j1aNUkY4Ue1gvwnGLVlOhGeYrnZaMgRK6+PKCUXaDbC7qtbW8gIkhL7aGCsOr/C56SJMy/BCZfxd1nWzAOxSDPgVsmerOBYfNqltV9/hWCqBywINIR+5dIg6JTJ72pcEpEjcYgXkE2YEFXV1JHnsKgbLWNlhScqb2UmyRkQyytRLtL+38TGxkxCflmO+5Z8CSSNY7GidjMIZ7Q4zMjA2n1nGrlTDkzwDCsw+wqFPGQA179cnfGWOWRVruj16z6XyvxvjJwbz0wQZ75XK5tKSb7FNyeIEs4TT4jk+S4dhPeAUC5y+bDYirYgM4GC7uEnztnZyaVWQ7B381AK4Qdrwt51ZqExKbQpTUNn+EjqoTwvqNj4kqx5QUCI0ThS/YkOxJCXmPUWZbhjpCg56i+2aB6CmK2JGhn57K5mj0MNdBXA4/WnwH6XoPWJzK5Nyu2zB3nAZp+S5hpQs+p1vN1/wsjk=" - -- name: Install services declarations - ansible.builtin.git: - repo: '{{ ota_engine_app_config.services.repository }}' - dest: '/home/{{ ansible_user }}/{{ ota_engine_declarations_directory }}' - version: '{{ ota_engine_declarations_branch }}' - force: true - key_file: '/home/{{ ansible_user }}/.ssh/ota-github-bot-key' - depth: 1 - tags: - - update-declarations - -- name: Install services declarations dependencies - ansible.builtin.command: - cmd: npm ci - chdir: '/home/{{ ansible_user }}/{{ ota_engine_declarations_directory }}' - tags: - - update-declarations - -- name: Add .env file - ansible.builtin.template: - src: .env - dest: '/home/{{ ansible_user }}/{{ ota_engine_declarations_directory }}/.env' - force: true - mode: "644" - -- name: Add pm2 config file - ansible.builtin.template: - src: pm2.config.j2 - dest: '/home/{{ ansible_user }}/{{ ota_engine_declarations_directory }}/pm2.config.cjs' - force: true - mode: "644" - -- name: Stop Open Terms Archive schedulers - ansible.builtin.command: - cmd: pm2 stop pm2.config.cjs - chdir: '/home/{{ ansible_user }}/{{ ota_engine_declarations_directory }}' - tags: - - stop - - update-declarations - -- name: Add conf in NGINX sites-available - become: true - ansible.builtin.template: - src: nginx-conf.j2 - dest: '/etc/nginx/sites-available/ota' - force: true - mode: "644" - notify: Restart NGINX - -- name: Link conf from sites-available to sites-enabled - become: true - ansible.builtin.file: - src: '/etc/nginx/sites-available/ota' - dest: '/etc/nginx/sites-enabled/ota' - state: link - force: true - notify: Restart NGINX - -- name: Setup snapshots git repository - ansible.builtin.include_tasks: database.yml - when: ota_engine_app_config.recorder.snapshots.storage.git.repository is defined - vars: - engine_database_name: snapshots - engine_database_repository: '{{ ota_engine_app_config.recorder.snapshots.storage.git.repository }}' - engine_database_branch: '{{ ota_engine_snapshots_branch }}' - engine_database_directory: '/home/{{ ansible_user }}/{{ ota_engine_declarations_directory }}/{{ ota_engine_app_config.recorder.snapshots.storage.git.path }}' - -- name: Setup versions git repository - ansible.builtin.include_tasks: database.yml - when: ota_engine_app_config.recorder.versions.storage.git.repository is defined - vars: - engine_database_name: versions - engine_database_repository: '{{ ota_engine_app_config.recorder.versions.storage.git.repository }}' - engine_database_branch: '{{ ota_engine_versions_branch }}' - engine_database_directory: '/home/{{ ansible_user }}/{{ ota_engine_declarations_directory }}/{{ ota_engine_app_config.recorder.versions.storage.git.path }}' - -- name: Start Open Terms Archive schedulers - ansible.builtin.command: - cmd: pm2 startOrRestart pm2.config.cjs - chdir: '/home/{{ ansible_user }}/{{ ota_engine_declarations_directory }}' - environment: - NODE_ENV: production - tags: - - restart - - start - - update-declarations diff --git a/roles/engine/templates/.env b/roles/engine/templates/.env deleted file mode 100644 index 85472f9..0000000 --- a/roles/engine/templates/.env +++ /dev/null @@ -1,11 +0,0 @@ -{{ ansible_managed | comment }} - -{% if ota_engine_sendinblue_api_key is defined %} -SENDINBLUE_API_KEY={{ ota_engine_sendinblue_api_key }} -{% endif %} -{% if ota_engine_smtp_password is defined %} -SMTP_PASSWORD={{ ota_engine_smtp_password }} -{% endif %} -{% if ota_engine_github_token is defined %} -GITHUB_TOKEN={{ ota_engine_github_token }} -{% endif %} diff --git a/roles/engine/templates/nginx-conf.j2 b/roles/engine/templates/nginx-conf.j2 deleted file mode 100644 index 654911d..0000000 --- a/roles/engine/templates/nginx-conf.j2 +++ /dev/null @@ -1,13 +0,0 @@ -{{ ansible_managed | comment }} - -server { - listen 80; - server_name {{ inventory_hostname }}; - - location / { - # Allowing for a `burst` of up to 5 requests beyond the specified rate limit. The `nodelay` parameter ensures that excessive requests beyond the burst limit are immediately rejected with a 429 error response instead of being queued. See https://www.nginx.com/blog/rate-limiting-nginx/. - limit_req zone=limited burst=5 nodelay; - proxy_pass http://localhost:{{ ota_engine_app_config.api.port }}; - proxy_redirect off; - } -} diff --git a/roles/engine/templates/pm2.config.j2 b/roles/engine/templates/pm2.config.j2 deleted file mode 100644 index cc994e0..0000000 --- a/roles/engine/templates/pm2.config.j2 +++ /dev/null @@ -1,25 +0,0 @@ -module.exports = { - apps: [ - { - name: 'ota', - script: 'npm', - args: 'run start:schedule', - max_restarts: 2, - min_uptime: '1h', // Set a relatively high duration (more than the longest run) so that restarts that occur before this duration has elapsed are considered unstable. - restart_delay: {{ ota_engine_restart_delay }}, - }, - { - name: 'ota-api', - script: 'npm', - args: 'run start:api', - }, - { - name: 'ota-release', - script: 'npm', - args: 'run dataset:schedule', - max_restarts: 2, - min_uptime: '1h', // Set a relatively high duration (more than the longest run) so that restarts that occur before this duration has elapsed are considered unstable. - restart_delay: 15 * 60 * 1000, - } - ], -}; diff --git a/roles/federated_api/README.md b/roles/federated_api/README.md deleted file mode 100644 index 4320fc0..0000000 --- a/roles/federated_api/README.md +++ /dev/null @@ -1,3 +0,0 @@ -# Federated API - -See https://github.com/OpenTermsArchive/federated-api/ diff --git a/roles/federated_api/defaults/main.yml b/roles/federated_api/defaults/main.yml deleted file mode 100644 index 28f3bea..0000000 --- a/roles/federated_api/defaults/main.yml +++ /dev/null @@ -1,3 +0,0 @@ -ota_federated_api_repo: https://github.com/OpenTermsArchive/federated-api -ota_federated_api_directory: federated-api -ota_federated_api_branch: main diff --git a/roles/federated_api/files/pm2.config.cjs b/roles/federated_api/files/pm2.config.cjs deleted file mode 100644 index 65de2c1..0000000 --- a/roles/federated_api/files/pm2.config.cjs +++ /dev/null @@ -1,9 +0,0 @@ -module.exports = { - apps: [ - { - name: 'ota-federated-api', - script: 'npm', - args: 'run start', - } - ], -}; diff --git a/roles/federated_api/handlers/main.yml b/roles/federated_api/handlers/main.yml deleted file mode 100644 index 347479f..0000000 --- a/roles/federated_api/handlers/main.yml +++ /dev/null @@ -1,6 +0,0 @@ ---- -- name: Restart NGINX - become: true - ansible.builtin.service: - name: nginx - state: restarted diff --git a/roles/federated_api/tasks/main.yml b/roles/federated_api/tasks/main.yml deleted file mode 100644 index ee40c2f..0000000 --- a/roles/federated_api/tasks/main.yml +++ /dev/null @@ -1,87 +0,0 @@ -- name: Add GitHub Bot SSH key - ansible.builtin.copy: - content: '{{ ota_engine_github_bot_private_key }}' - dest: '/home/{{ ansible_user }}/.ssh/ota-github-bot-key' - owner: '{{ ansible_user }}' - group: '{{ ansible_user }}' - mode: "600" - -- name: Add GitHub SSH key to known_hosts - ansible.builtin.known_hosts: - name: github.com - key: "{{ item }}" - path: ~/.ssh/known_hosts - with_items: # GitHub's SSH key fingerprints can be found here: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/githubs-ssh-key-fingerprints - - "github.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOMqqnkVzrm0SdG6UOoqKLsabgH5C9okWi0dh2l9GKJl" - - "github.com ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEmKSENjQEezOmxkZMy7opKgwFB9nkt5YRrYMjNuG5N87uRgg6CLrbo5wAdT/y6v0mKV0U2w0WZ2YB/++Tpockg=" - - "github.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCj7ndNxQowgcQnjshcLrqPEiiphnt+VTTvDP6mHBL9j1aNUkY4Ue1gvwnGLVlOhGeYrnZaMgRK6+PKCUXaDbC7qtbW8gIkhL7aGCsOr/C56SJMy/BCZfxd1nWzAOxSDPgVsmerOBYfNqltV9/hWCqBywINIR+5dIg6JTJ72pcEpEjcYgXkE2YEFXV1JHnsKgbLWNlhScqb2UmyRkQyytRLtL+38TGxkxCflmO+5Z8CSSNY7GidjMIZ7Q4zMjA2n1nGrlTDkzwDCsw+wqFPGQA179cnfGWOWRVruj16z6XyvxvjJwbz0wQZ75XK5tKSb7FNyeIEs4TT4jk+S4dhPeAUC5y+bDYirYgM4GC7uEnztnZyaVWQ7B381AK4Qdrwt51ZqExKbQpTUNn+EjqoTwvqNj4kqx5QUCI0ThS/YkOxJCXmPUWZbhjpCg56i+2aB6CmK2JGhn57K5mj0MNdBXA4/WnwH6XoPWJzK5Nyu2zB3nAZp+S5hpQs+p1vN1/wsjk=" - -- name: Clone federated API repository - ansible.builtin.git: - repo: '{{ ota_federated_api_repo }}' - dest: '/home/{{ ansible_user }}/{{ ota_federated_api_directory }}' - version: '{{ ota_federated_api_branch }}' - force: true - key_file: '/home/{{ ansible_user }}/.ssh/ota-github-bot-key' - depth: 1 - -- name: Read the production config - shell: cat '/home/{{ ansible_user }}/{{ ota_federated_api_directory }}/config/production.json' - register: read_config - -- name: Save production config data into a variable - set_fact: - ota_federated_api_app_config: "{{ read_config.stdout | from_json }}" - -- name: Install dependencies - ansible.builtin.command: - cmd: npm ci - chdir: '/home/{{ ansible_user }}/{{ ota_federated_api_directory }}' - -- name: Add .env file - ansible.builtin.template: - src: .env - dest: '/home/{{ ansible_user }}/{{ ota_federated_api_directory }}/.env' - force: true - mode: "644" - -- name: Add pm2 config file - ansible.builtin.copy: - src: pm2.config.cjs - dest: '/home/{{ ansible_user }}/{{ ota_federated_api_directory }}/pm2.config.cjs' - mode: "644" - -- name: Stop federated API - ansible.builtin.command: - cmd: pm2 stop pm2.config.cjs - chdir: '/home/{{ ansible_user }}/{{ ota_federated_api_directory }}' - tags: - - stop - -- name: Add conf in NGINX sites-available - become: true - ansible.builtin.template: - src: nginx-conf.j2 - dest: '/etc/nginx/sites-available/ota' - force: true - mode: "644" - notify: Restart NGINX - -- name: Link conf from sites-available to sites-enabled - become: true - ansible.builtin.file: - src: '/etc/nginx/sites-available/ota' - dest: '/etc/nginx/sites-enabled/ota' - state: link - force: true - notify: Restart NGINX - -- name: Start federated API - ansible.builtin.command: - cmd: pm2 startOrRestart pm2.config.cjs - chdir: '/home/{{ ansible_user }}/{{ ota_federated_api_directory }}' - environment: - NODE_ENV: production - tags: - - restart - - start diff --git a/roles/federated_api/templates/.env b/roles/federated_api/templates/.env deleted file mode 100644 index 9daa89c..0000000 --- a/roles/federated_api/templates/.env +++ /dev/null @@ -1,5 +0,0 @@ -{{ ansible_managed | comment }} - -{% if ota_federated_api_smtp_password is defined %} -SMTP_PASSWORD={{ ota_federated_api_smtp_password }} -{% endif %} diff --git a/roles/federated_api/templates/nginx-conf.j2 b/roles/federated_api/templates/nginx-conf.j2 deleted file mode 100644 index 8ac6285..0000000 --- a/roles/federated_api/templates/nginx-conf.j2 +++ /dev/null @@ -1,13 +0,0 @@ -{{ ansible_managed | comment }} - -server { - listen 80; - server_name {{ inventory_hostname }}; - - location / { - # Allowing for a `burst` of up to 5 requests beyond the specified rate limit. The `nodelay` parameter ensures that excessive requests beyond the burst limit are immediately rejected with a 429 error response instead of being queued. See https://www.nginx.com/blog/rate-limiting-nginx/. - limit_req zone=limited burst=5 nodelay; - proxy_pass http://localhost:{{ ota_federated_api_app_config.port }}; - proxy_redirect off; - } -} diff --git a/roles/github/tasks/main.yml b/roles/github/tasks/main.yml new file mode 100644 index 0000000..949086d --- /dev/null +++ b/roles/github/tasks/main.yml @@ -0,0 +1,26 @@ +--- +- name: Check if github-bot-private-key file is provided (on local machine) + ansible.builtin.stat: + path: "{{ inventory_dir }}/github-bot-private-key" + register: github_bot_private_key_file + delegate_to: localhost + +- name: Add GitHub Bot SSH key + ansible.builtin.copy: + src: "{{ inventory_dir }}/github-bot-private-key" + dest: /home/{{ ansible_user }}/.ssh/ota-github-bot-key + owner: "{{ ansible_user }}" + group: "{{ ansible_user }}" + mode: "600" + when: github_bot_private_key_file.stat.exists + +- name: Add GitHub SSH key to known_hosts + ansible.builtin.known_hosts: + name: github.com + key: "{{ item }}" + path: ~/.ssh/known_hosts + with_items: # GitHub's SSH key fingerprints can be found here: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/githubs-ssh-key-fingerprints + - github.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOMqqnkVzrm0SdG6UOoqKLsabgH5C9okWi0dh2l9GKJl + - github.com ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEmKSENjQEezOmxkZMy7opKgwFB9nkt5YRrYMjNuG5N87uRgg6CLrbo5wAdT/y6v0mKV0U2w0WZ2YB/++Tpockg= + - github.com ssh-rsa + AAAAB3NzaC1yc2EAAAADAQABAAABgQCj7ndNxQowgcQnjshcLrqPEiiphnt+VTTvDP6mHBL9j1aNUkY4Ue1gvwnGLVlOhGeYrnZaMgRK6+PKCUXaDbC7qtbW8gIkhL7aGCsOr/C56SJMy/BCZfxd1nWzAOxSDPgVsmerOBYfNqltV9/hWCqBywINIR+5dIg6JTJ72pcEpEjcYgXkE2YEFXV1JHnsKgbLWNlhScqb2UmyRkQyytRLtL+38TGxkxCflmO+5Z8CSSNY7GidjMIZ7Q4zMjA2n1nGrlTDkzwDCsw+wqFPGQA179cnfGWOWRVruj16z6XyvxvjJwbz0wQZ75XK5tKSb7FNyeIEs4TT4jk+S4dhPeAUC5y+bDYirYgM4GC7uEnztnZyaVWQ7B381AK4Qdrwt51ZqExKbQpTUNn+EjqoTwvqNj4kqx5QUCI0ThS/YkOxJCXmPUWZbhjpCg56i+2aB6CmK2JGhn57K5mj0MNdBXA4/WnwH6XoPWJzK5Nyu2zB3nAZp+S5hpQs+p1vN1/wsjk= diff --git a/roles/infrastructure/chromium/README.md b/roles/infrastructure/chromium/README.md deleted file mode 100644 index 7e1f097..0000000 --- a/roles/infrastructure/chromium/README.md +++ /dev/null @@ -1,3 +0,0 @@ -# Chromium - -Used by the engine as a [puppeteer](https://pptr.dev) dependency. diff --git a/roles/infrastructure/chromium/meta/main.yml b/roles/infrastructure/chromium/meta/main.yml deleted file mode 100644 index e5e65fc..0000000 --- a/roles/infrastructure/chromium/meta/main.yml +++ /dev/null @@ -1,2 +0,0 @@ -dependencies: - - { role: infrastructure/common } diff --git a/roles/infrastructure/common/README.md b/roles/infrastructure/common/README.md deleted file mode 100644 index 019b25f..0000000 --- a/roles/infrastructure/common/README.md +++ /dev/null @@ -1 +0,0 @@ -# Common diff --git a/roles/infrastructure/mongo/README.md b/roles/infrastructure/mongo/README.md deleted file mode 100644 index e679fb7..0000000 --- a/roles/infrastructure/mongo/README.md +++ /dev/null @@ -1 +0,0 @@ -# Mongo diff --git a/roles/infrastructure/mongo/meta/main.yml b/roles/infrastructure/mongo/meta/main.yml deleted file mode 100644 index e5e65fc..0000000 --- a/roles/infrastructure/mongo/meta/main.yml +++ /dev/null @@ -1,2 +0,0 @@ -dependencies: - - { role: infrastructure/common } diff --git a/roles/infrastructure/nginx/README.md b/roles/infrastructure/nginx/README.md deleted file mode 100644 index b30386d..0000000 --- a/roles/infrastructure/nginx/README.md +++ /dev/null @@ -1 +0,0 @@ -# Nginx diff --git a/roles/infrastructure/nginx/meta/main.yml b/roles/infrastructure/nginx/meta/main.yml deleted file mode 100644 index e5e65fc..0000000 --- a/roles/infrastructure/nginx/meta/main.yml +++ /dev/null @@ -1,2 +0,0 @@ -dependencies: - - { role: infrastructure/common } diff --git a/roles/infrastructure/nginx/tasks/main.yml b/roles/infrastructure/nginx/tasks/main.yml deleted file mode 100644 index df9abd3..0000000 --- a/roles/infrastructure/nginx/tasks/main.yml +++ /dev/null @@ -1,21 +0,0 @@ ---- -- name: Install NGINX package - ansible.builtin.apt: - name: nginx - update_cache: true - state: latest - -- name: Clean default NGINX site - ansible.builtin.file: - state: absent - path: /etc/nginx/sites-enabled/default - notify: Restart NGINX - -- name: Setup NGINX conf - ansible.builtin.template: - src: nginx.conf.j2 - dest: /etc/nginx/conf.d/ota.conf - owner: root - group: root - mode: "644" - notify: Restart NGINX diff --git a/roles/infrastructure/node/README.md b/roles/infrastructure/node/README.md deleted file mode 100644 index 6d9836e..0000000 --- a/roles/infrastructure/node/README.md +++ /dev/null @@ -1 +0,0 @@ -# Node diff --git a/roles/infrastructure/node/meta/main.yml b/roles/infrastructure/node/meta/main.yml deleted file mode 100644 index e5e65fc..0000000 --- a/roles/infrastructure/node/meta/main.yml +++ /dev/null @@ -1,2 +0,0 @@ -dependencies: - - { role: infrastructure/common } diff --git a/roles/infrastructure/mongo/files/mongod.conf b/roles/mongo/configure/files/mongod.conf similarity index 100% rename from roles/infrastructure/mongo/files/mongod.conf rename to roles/mongo/configure/files/mongod.conf diff --git a/roles/mongo/configure/handlers/main.yml b/roles/mongo/configure/handlers/main.yml new file mode 100644 index 0000000..47b3720 --- /dev/null +++ b/roles/mongo/configure/handlers/main.yml @@ -0,0 +1,5 @@ +--- +- name: Restart MongoDB + ansible.builtin.service: + name: mongod + state: restarted diff --git a/roles/mongo/configure/meta/main.yml b/roles/mongo/configure/meta/main.yml new file mode 100644 index 0000000..6b4fff8 --- /dev/null +++ b/roles/mongo/configure/meta/main.yml @@ -0,0 +1,3 @@ +--- +dependencies: + - { role: common } diff --git a/roles/mongo/configure/tasks/main.yml b/roles/mongo/configure/tasks/main.yml new file mode 100644 index 0000000..35991cd --- /dev/null +++ b/roles/mongo/configure/tasks/main.yml @@ -0,0 +1,22 @@ +--- +- name: Configure MongoDB + notify: Restart MongoDB + block: + - name: Add mongod.conf + ansible.builtin.copy: + src: mongod.conf + dest: /etc/mongod.conf + mode: "644" + + - name: Create data directory + ansible.builtin.file: + path: /mnt/disk/mongodb + state: directory + mode: "744" + + - name: Set database files permissions + ansible.builtin.file: + path: /mnt/disk/mongodb + owner: mongodb + group: mongodb + recurse: true diff --git a/roles/mongo/install/meta/main.yml b/roles/mongo/install/meta/main.yml new file mode 100644 index 0000000..6b4fff8 --- /dev/null +++ b/roles/mongo/install/meta/main.yml @@ -0,0 +1,3 @@ +--- +dependencies: + - { role: common } diff --git a/roles/infrastructure/mongo/tasks/main.yml b/roles/mongo/install/tasks/main.yml similarity index 69% rename from roles/infrastructure/mongo/tasks/main.yml rename to roles/mongo/install/tasks/main.yml index fe2a47c..88598f6 100644 --- a/roles/infrastructure/mongo/tasks/main.yml +++ b/roles/mongo/install/tasks/main.yml @@ -26,27 +26,3 @@ name: mongodb-org update_cache: true state: latest - -- name: Add mongod.conf - ansible.builtin.copy: - src: mongod.conf - dest: '/etc/mongod.conf' - mode: "644" - -- name: Create data directory - ansible.builtin.file: - path: /mnt/disk/mongodb - state: directory - mode: "744" - -- name: Set database files permissions - ansible.builtin.file: - path: /mnt/disk/mongodb - owner: mongodb - group: mongodb - recurse: true - -- name: Start MongoDB service - ansible.builtin.service: - name: mongod - state: restarted diff --git a/roles/infrastructure/nginx/handlers/main.yml b/roles/nginx/configure/handlers/main.yml similarity index 100% rename from roles/infrastructure/nginx/handlers/main.yml rename to roles/nginx/configure/handlers/main.yml diff --git a/roles/nginx/configure/meta/main.yml b/roles/nginx/configure/meta/main.yml new file mode 100644 index 0000000..6b4fff8 --- /dev/null +++ b/roles/nginx/configure/meta/main.yml @@ -0,0 +1,3 @@ +--- +dependencies: + - { role: common } diff --git a/roles/nginx/configure/tasks/main.yml b/roles/nginx/configure/tasks/main.yml new file mode 100644 index 0000000..c7a0363 --- /dev/null +++ b/roles/nginx/configure/tasks/main.yml @@ -0,0 +1,33 @@ +--- +- name: Configure NGINX + notify: Restart NGINX + block: + - name: Clean default NGINX site + ansible.builtin.file: + state: absent + path: /etc/nginx/sites-enabled/default + + - name: Setup NGINX conf + ansible.builtin.template: + src: "{{ ota_nginx_config_template }}" + dest: /etc/nginx/conf.d/ota.conf + owner: root + group: root + mode: "644" + +- name: Configure NGINX Reverse Proxy + notify: Restart NGINX + block: + - name: Add conf in NGINX sites-available + ansible.builtin.template: + src: "{{ ota_nginx_reverse_proxy_config_template }}" + dest: /etc/nginx/sites-available/ota + force: true + mode: "644" + + - name: Link conf from sites-available to sites-enabled + ansible.builtin.file: + src: /etc/nginx/sites-available/ota + dest: /etc/nginx/sites-enabled/ota + state: link + force: true diff --git a/roles/nginx/install/meta/main.yml b/roles/nginx/install/meta/main.yml new file mode 100644 index 0000000..6b4fff8 --- /dev/null +++ b/roles/nginx/install/meta/main.yml @@ -0,0 +1,3 @@ +--- +dependencies: + - { role: common } diff --git a/roles/nginx/install/tasks/main.yml b/roles/nginx/install/tasks/main.yml new file mode 100644 index 0000000..c48b5af --- /dev/null +++ b/roles/nginx/install/tasks/main.yml @@ -0,0 +1,6 @@ +--- +- name: Install NGINX package + ansible.builtin.apt: + name: nginx + update_cache: true + state: latest diff --git a/roles/node/meta/main.yml b/roles/node/meta/main.yml new file mode 100644 index 0000000..6b4fff8 --- /dev/null +++ b/roles/node/meta/main.yml @@ -0,0 +1,3 @@ +--- +dependencies: + - { role: common } diff --git a/roles/infrastructure/node/tasks/main.yml b/roles/node/tasks/main.yml similarity index 65% rename from roles/infrastructure/node/tasks/main.yml rename to roles/node/tasks/main.yml index a667e4f..f275b09 100644 --- a/roles/infrastructure/node/tasks/main.yml +++ b/roles/node/tasks/main.yml @@ -11,8 +11,7 @@ executable: /bin/bash - name: Create deb repository - ansible.builtin.shell: - set -o pipefail && echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_20.x nodistro main" | tee /etc/apt/sources.list.d/nodesource.list # Remember to update the major version of NPM when updating the major version of Node + ansible.builtin.shell: set -o pipefail && echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_20.x nodistro main" | tee /etc/apt/sources.list.d/nodesource.list # Remember to update the major version of NPM when updating the major version of Node args: executable: /bin/bash @@ -24,6 +23,3 @@ - name: Update NPM to latest version 10 ansible.builtin.command: npm install -g npm@10 - -- name: Install PM2 to latest version 5 - ansible.builtin.command: npm install -g pm2@5 --production=true diff --git a/roles/ota/apps/defaults/main.yml b/roles/ota/apps/defaults/main.yml new file mode 100644 index 0000000..979421f --- /dev/null +++ b/roles/ota/apps/defaults/main.yml @@ -0,0 +1,4 @@ +--- +ota_apps_read_config_only: false +ota_source_repository_branch: main +ota_directory: "{{ ota_source_repository | ansible.builtin.urlsplit('path') | ansible.builtin.basename | ansible.builtin.splitext | first }}" # extract the repository name from the URL diff --git a/roles/ota/apps/meta/main.yml b/roles/ota/apps/meta/main.yml new file mode 100644 index 0000000..2f3426c --- /dev/null +++ b/roles/ota/apps/meta/main.yml @@ -0,0 +1,4 @@ +--- +dependencies: + - { role: common } + - { role: github } diff --git a/roles/ota/apps/tasks/main.yml b/roles/ota/apps/tasks/main.yml new file mode 100644 index 0000000..c719695 --- /dev/null +++ b/roles/ota/apps/tasks/main.yml @@ -0,0 +1,39 @@ +--- +- name: Clone repository + ansible.builtin.git: + repo: "{{ ota_source_repository }}" + dest: /home/{{ ansible_user }}/{{ ota_directory }} + version: "{{ ota_source_repository_branch }}" + force: true + key_file: /home/{{ ansible_user }}/.ssh/ota-github-bot-key + depth: 1 + +- name: Read the production config + ansible.builtin.command: cat '/home/{{ ansible_user }}/{{ ota_directory }}/config/production.json' + register: read_config + +- name: Save production config data into a variable + ansible.builtin.set_fact: + ota_apps_config: "{{ read_config.stdout | from_json }}" + +- name: Set up environment variables and install dependencies + when: not ota_apps_read_config_only | bool + block: + - name: Install dependencies + ansible.builtin.command: + cmd: npm ci + chdir: /home/{{ ansible_user }}/{{ ota_directory }} + + - name: Check if .env file is provided (on local machine) + ansible.builtin.stat: + path: "{{ inventory_dir }}/.env" + register: env_file + delegate_to: localhost + + - name: Add .env file + ansible.builtin.copy: + src: "{{ inventory_dir }}/.env" + dest: /home/{{ ansible_user }}/{{ ota_directory }}/.env + force: true + mode: "644" + when: env_file.stat.exists diff --git a/roles/ota/git-database/meta/main.yml b/roles/ota/git-database/meta/main.yml new file mode 100644 index 0000000..2f3426c --- /dev/null +++ b/roles/ota/git-database/meta/main.yml @@ -0,0 +1,4 @@ +--- +dependencies: + - { role: common } + - { role: github } diff --git a/roles/ota/git-database/tasks/main.yml b/roles/ota/git-database/tasks/main.yml new file mode 100644 index 0000000..c0d4754 --- /dev/null +++ b/roles/ota/git-database/tasks/main.yml @@ -0,0 +1,41 @@ +--- +- name: Check if base data has already been obtained from {{ ota_git_database_repository }} + ansible.builtin.git: + repo: "{{ ota_git_database_repository }}" + version: "{{ ota_git_database_branch }}" + dest: "{{ ota_git_database_directory }}" + clone: false + update: false + key_file: /home/{{ ansible_user }}/.ssh/ota-github-bot-key + # the `before` property of the return value can tell us if the repository has been cloned already or not, + # see + register: existing_repository + +- name: Obtain initial data from branch {{ ota_git_database_branch }} of {{ ota_git_database_repository }} + ansible.builtin.git: + repo: "{{ ota_git_database_repository }}" + version: "{{ ota_git_database_branch }}" + dest: "{{ ota_git_database_directory }}" + key_file: /home/{{ ansible_user }}/.ssh/ota-github-bot-key + when: existing_repository.before is defined and not existing_repository.before # if existing_repository.before is null, then the repository is new + +- name: Configure {{ ota_git_database_directory }} repository to use the GitHub Bot SSH Key + community.general.git_config: + name: core.sshCommand + value: ssh -i '/home/{{ ansible_user }}/.ssh/ota-github-bot-key' + scope: local + repo: "{{ ota_git_database_directory }}" + +- name: Remove existing locks in {{ ota_git_database_directory }} + ansible.builtin.file: + path: "{{ ota_git_database_directory }}/.git/index.lock" + state: absent + +- name: Get latest data from {{ ota_git_database_repository }} + ansible.builtin.git: + repo: "{{ ota_git_database_repository }}" + version: "{{ ota_git_database_branch }}" + dest: "{{ ota_git_database_directory }}" + update: true + force: true # discard any modified files + key_file: /home/{{ ansible_user }}/.ssh/ota-github-bot-key diff --git a/roles/pm2/install/meta/main.yml b/roles/pm2/install/meta/main.yml new file mode 100644 index 0000000..3b80e22 --- /dev/null +++ b/roles/pm2/install/meta/main.yml @@ -0,0 +1,4 @@ +--- +dependencies: + - { role: common } + - { role: node } diff --git a/roles/pm2/install/tasks/main.yml b/roles/pm2/install/tasks/main.yml new file mode 100644 index 0000000..5c4637b --- /dev/null +++ b/roles/pm2/install/tasks/main.yml @@ -0,0 +1,3 @@ +--- +- name: Install PM2 to latest version 5 + ansible.builtin.command: npm install --global pm2@5 --production=true diff --git a/roles/pm2/manage/meta/main.yml b/roles/pm2/manage/meta/main.yml new file mode 100644 index 0000000..3b80e22 --- /dev/null +++ b/roles/pm2/manage/meta/main.yml @@ -0,0 +1,4 @@ +--- +dependencies: + - { role: common } + - { role: node } diff --git a/roles/pm2/manage/tasks/main.yml b/roles/pm2/manage/tasks/main.yml new file mode 100644 index 0000000..a978952 --- /dev/null +++ b/roles/pm2/manage/tasks/main.yml @@ -0,0 +1,29 @@ +- name: Check if pm2.config.cjs file is provided + ansible.builtin.stat: + path: /home/{{ ansible_user }}/{{ ota_directory }}/pm2.config.cjs + register: pm2_config + +- name: Stop Open Terms Archive applications + ansible.builtin.command: + cmd: pm2 stop pm2.config.cjs + chdir: /home/{{ ansible_user }}/{{ ota_directory }} + tags: + - stop + when: pm2_config.stat.exists + +- name: Copy pm2.config.cjs file + ansible.builtin.copy: + src: "{{ inventory_dir }}/pm2.config.cjs" + dest: /home/{{ ansible_user }}/{{ ota_directory }}/pm2.config.cjs + force: true + mode: "644" + +- name: Start Open Terms Archive applications + ansible.builtin.command: + cmd: pm2 startOrRestart pm2.config.cjs + chdir: /home/{{ ansible_user }}/{{ ota_directory }} + environment: + NODE_ENV: production + tags: + - restart + - start diff --git a/tests/Vagrantfile b/tests/Vagrantfile index a8e6433..f1fd94a 100644 --- a/tests/Vagrantfile +++ b/tests/Vagrantfile @@ -1,15 +1,15 @@ # -*- mode: ruby -*- # vi: set ft=ruby : +guest_port = 80 +host_port = ENV["PORT"] || 8080 + Vagrant.configure("2") do |config| + config.vm.define "opentermsarchive_deployment" config.vm.hostname = "vagrant" config.vm.box = "debian/bullseye64" # Unable to locate package mongodb-org - - # in order to have the same config for both Docker and VirtualBox providers, we load the key manually - # if necessary, create the key with `ssh-keygen -f ~/.ssh/ota-vagrant -q -N ""` - # CAUTION: use of `~` in path causes problems with ssh - config.vm.provision "file", source: File.join(ENV['HOME'], ".ssh", "ota-vagrant.pub"), destination: "/home/vagrant/.ssh/authorized_keys" + config.vm.network "forwarded_port", guest: guest_port, host: host_port # based on https://github.com/rofrano/vagrant-docker-provider#example-vagrantfile config.vm.provider :docker do |docker, override| @@ -20,19 +20,12 @@ Vagrant.configure("2") do |config| docker.privileged = true docker.volumes = ["/sys/fs/cgroup:/sys/fs/cgroup:rw"] docker.create_args = ["--cgroupns=host"] - - # python is not installed by default in the vagrant-provider image - # and deploying results in /bin/sh: 1: /usr/bin/python: not found - # use a provision to fix that - # only with debian, no need with ubuntu - # Also need to name the provisioner, so that it runs only once https://github.com/hashicorp/vagrant/issues/7685#issuecomment-308281283 - config.vm.provision "install_python3", type: "shell", inline: $installPython3 end -end -$installPython3 = <<-SCRIPT -echo Updating apt... -sudo apt-get update --fix-missing # Needed to fix "No package matching 'chromium' is available" -echo Installing python... -sudo apt-get --assume-yes install python3 python3-pip -SCRIPT + # Provisioning step to install Python 3 and make it the default + config.vm.provision "shell", inline: <<-SHELL + apt-get update + apt-get install --assume-yes python3 python3-pip + update-alternatives --install /usr/bin/python python /usr/bin/python3 1 + SHELL +end diff --git a/tests/engine_config.json b/tests/engine_config.json deleted file mode 100644 index f78d16d..0000000 --- a/tests/engine_config.json +++ /dev/null @@ -1,32 +0,0 @@ -{ - "name": "vagrant", - "services": { - "repository": "https://github.com/OpenTermsArchive/sandbox-declarations.git" - }, - "recorder": { - "versions": { - "storage": { - "git": { - "repository": "git@github.com:OpenTermsArchive/sandbox-versions.git", - "path": "./data/versions", - "publish": false - } - } - }, - "snapshots": { - "storage": { - "git": { - "repository": "git@github.com:OpenTermsArchive/sandbox-snapshots.git", - "path": "./data/snapshots", - "publish": false - } - } - } - }, - "logger": { - "sendMailOnError": false - }, - "api": { - "port": 3000 - } -} diff --git a/tests/github-bot-private-key b/tests/github-bot-private-key new file mode 100644 index 0000000..f1a3ccd --- /dev/null +++ b/tests/github-bot-private-key @@ -0,0 +1,26 @@ +$ANSIBLE_VAULT;1.1;AES256 +32383732393936323334373030666366633637306438373563613266366138333837346135666265 +3935326632373536326331393331356637376233343535620a616637343063346437393062636439 +66333464396232343462353637356236663337643838623866353166326535643934306638666164 +3061616138363064640a353335643837366564346136313636323331316539623961313963336336 +34343633336137636532356261646663383636666339336634653232653262353538373732346366 +65356534373838316234353330306536313265663638663262303437636537333139373365316433 +36343139376265646263386364336164343565356336373930353133636633316138366565643263 +65666366353766376134363230303964373163636538373364646164343962303166366337343137 +61613961396334653536333336303134663366623663633633656130306637376164346230373135 +64616361333462646334613132366330326634313638313134666237386638393333393662373861 +33343839363731633336653130396662326134613236353031396431666636646236643937323038 +33383038623365616130633234333331313531383436393465316164633335663163626634643738 +31663231613031313331323739313833353039373639396336336639623664346533386539666238 +64336635376564363766343965306535613838353135643734653535633264343166393939613932 +33326435343265343863636338653135333566366239623538373665646131613432653962303766 +66643433666161636631643130643935663761633337663839663338646461356237353562376638 +63366439653932383635313138353336666362626231663133393232613633386639323631613363 +30633762333235333637616565396562666436353435626534366661633265613530383538306338 +64366338396566323531366534353164326463646135363237353864376630633065363163356335 +34613964313566643366313562343664666666326361303364626234326162343362383764646366 +30323530316238656536643730363930303533653464383936373630323232386264373163376265 +64646665626534663133303937316365383935666336353666306435303634393664306539656265 +66333366393038343566326237353063326337356361643663326339663633333263326666363238 +31626231303533373065313662353132623035626638346331303238336331383035316465376437 +66643339653036363438323062326234623865643063613530366335373535646363 diff --git a/tests/inventory.yml b/tests/inventory.yml index 61caf9b..4940463 100644 --- a/tests/inventory.yml +++ b/tests/inventory.yml @@ -1,35 +1,7 @@ vagrant: hosts: - 127.0.0.1: + vagrant: # Name the host instead of using 127.0.0.1 to ensure `delegate_to` distinguishes between the local machine and the locally running Vagrant VM ansible_user: vagrant ansible_port: 2222 - ansible_python_interpreter: /usr/bin/python3 - ansible_ssh_private_key_file: ~/.ssh/ota-vagrant - ota_engine_config_path: ./engine_config.json - ota_engine_github_bot_private_key: !vault | - $ANSIBLE_VAULT;1.1;AES256 - 63366161663933343964333231666663356335616231393562366266353139653032376635373036 - 3233356266633764663963666564366633346561633633300a333865393538363665383936323336 - 63326432396664643962303737656430363534326466336238613432343231626333303637616137 - 3065333331626234370a653962326564646534616238626538383436396234653234333838633064 - 30326166376664356437386366313162396634313562383564316166663038306361396365303833 - 38316161326432643735376236333532386162383662656130313430313338323236626132646136 - 30643232353765376664303265343836363330373062373535333665383230656662376362636436 - 37393135393162396665616234666137356539363663373130306136346135666133666162333463 - 31303133386663366662303137326435383432303932353863636131373732313737306232306330 - 35303732316338343035316332353262386533333335653732343230393562623766356263653562 - 61343131343532346633323538636366323633396534663735333333653139386335636632663664 - 62626166613130396163373039666630383231653461323138396366373065346335313837643563 - 35353139373564343165623063386338343639333363336466356261396436376266323861303938 - 38333339386164343261316161633131303833363762333733393865626163373236313364656464 - 33346335656233333064653733666332656337626665646161306566356235616438363265316136 - 39663363393332396264323831643037363237303938353235636165633937663964346139633364 - 38626632343139366464613734313334646335386364666438303336306562333337306639333637 - 61396564353430653932643535616461393562346633643830303766376332323333643235343135 - 34323333616466663736323164313036623038376134623662386431306163666430393132623361 - 66346265383435383164313335303262316238623530613737346661373837623939636262653233 - 64363764613537633538316531303063633132393362323034346166306361653661653236633133 - 33316437376364656130323936356662326162323565663433326530343064643866373331616534 - 63313139393761646335366464386638666430346136373561643761633130663736633132633333 - 65636336376633393030333133633733306231666638613034313639666462653563656634633739 - 66383964306235666233363665633333373766353663653237636332306330393130 + ansible_ssh_host: 127.0.0.1 + ota_source_repository: https://github.com/OpenTermsArchive/demo-declarations.git diff --git a/tests/pm2.config.cjs b/tests/pm2.config.cjs new file mode 100644 index 0000000..05ebc5c --- /dev/null +++ b/tests/pm2.config.cjs @@ -0,0 +1,39 @@ +module.exports = { + apps: [ + { + name: 'ota', + script: 'npm', + args: 'run start:schedule', + max_restarts: 2, + min_uptime: '1h', // Set a relatively high duration (more than the longest run) so that restarts that occur before this duration has elapsed are considered unstable. + restart_delay: 3 * 60 * 60 * 1000, // likely related to a connectivity problem that will take some time to be fixed + }, + { + name: 'ota-api', + script: 'npm', + args: 'run start:api', + min_uptime: '10s', + max_restarts: 10, + restart_delay: 1000, + exponential_backoff_restart_delay: true + }, + { + name: 'ota-release', + script: 'npm', + args: 'run dataset:schedule', + min_uptime: '10s', + max_restarts: 10, + restart_delay: 60 * 60 * 1000, // likely related to a GitHub availability problem that will take some time to be fixed + exponential_backoff_restart_delay: true + }, + { + name: 'ota-federation-api', + script: 'npm', + args: 'run start:federation-api', + min_uptime: '10s', + max_restarts: 10, + restart_delay: 1000, + exponential_backoff_restart_delay: true + } + ], +};