Fix Creation of AWS Sandboxes + Containerize

[fridim]

The creation of new AWS sandboxes is currently broken and is a manual process that is run from different places.

• ansible playbooks to create the accounts
• cypress automation to enable the accounts in access.redhat.com
• playbook again to validate the sandboxes and make sure the GOLD images there

Instead, make it possible to run it as a single OpenShift job.

Tasks:

• Create a container image that has all the software needed for the creation of a new sandbox

• Fix locales in Containerfile.admin

• Create a wrapper script for automation => Python

• Automatically guess the next sandbox number from all different DB (prod and dev)

• Add profiling callback to ansible creation playbook runs.

• Enable Gold images by using the new HCC (console.redhat.com) APIs instead of Cypress + access.redhat.com
  There is a transition from access.redhat.com web page to HCC (console.redhat.com)
  Advantages:

  • much much faster
  • less dependencies in the images (roughly -500MB)

• Status script creation_status.py – list creation in progress freshly created sandbox

• Add a --retry sandbox123 capability

• Add alternate contact to the account

• Toggle ansible-playbook output ON|OFF

• Add an Org Policy to p protect anything that is required by HCC (role, …)

• Performance: make slow task async in the playbook (low priority)

• New feature: provide the reservation name, by default new sandboxes end up in a new reservation

• sandboxes are created in a untested reservation first. After the functional tests, if successful, we move the new sandboxes to the target reservation (default new)

• Create monitoring dashboard or at least scripts for the creation

• Add a test to ensure Vault value is correct. Try to read one key with the passed vault secret. If it doesn't work, exit. That will prevent accidentally creating sandboxes with a vault different that the one currently in use for the target DB;

• document

• Package everything for OpenShift: use OpenShift job to run the creation

[jkupferer]

This random delay does not work as expected. Unfortunately Ansible only evaluates the delay value once when starting the task rather than for each retry. So if it gets 5 it will be a 5 second delay it will be the same delay for each retry.