Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set up local database docker container for lightweight testing, eliminate Sandbox #585

Open
maxachis opened this issue Jan 20, 2025 · 2 comments
Labels
fixed_in_dev This is merged into the dev environment and waiting to be merged into main

Comments

@maxachis
Copy link
Contributor

maxachis commented Jan 20, 2025

One of the lessons learned from the Data Source Identification Repo is that a local database would enable ue to test database changes more simply, quickly and reliably compared to a standalone sandbox database.

This would also considerably simplify our database migration logic, as we could eliminate having to account for separate sandbox database logic.

  • In a pinch, we may also be able to eliminate stage, but that's much less certain

Additionally, this may help with #584 , if we wish to go in that direction.

Steps would be as follows:

  1. Set up a local_database repository within the repo which can be used to spin up a local postgres database
  2. Local environment variables would then need modified to point to the local database (as opposed to a sandbox database)
  3. Set up a README that instructs users to do exactly that
  4. Set up logic such that the SQLAlchemy model is used to construct the database structure
  5. Run existing tests to ensure they still perform as expected in the new environment
  6. If tests fail, that indicates the SQLAlchemy model needs modified to properly reflect the sandbox database -- only models.py should be modified for these to pass.
  7. The Github Runner could then also be modified to use a local database for tests, much like how it's currently done in the data source identification app
@maxachis
Copy link
Contributor Author

maxachis commented Jan 22, 2025

Now that a local database is set up, there are a few additional steps which would need to be done to bring the sandbox database offline:

  • Modify the existing Jenkins job Prod to Stage and Sandbox Migration to no longer migrate to sandbox
  • Validate that Stage and Prod migration jobs continue to function
  • Double-check that no one else is using the sandbox database (no one should be, so this should be fine)
  • Destroy the Sandbox database.

@maxachis maxachis added the fixed_in_dev This is merged into the dev environment and waiting to be merged into main label Jan 22, 2025
@josh-chamberlain
Copy link
Contributor

@maxachis love this !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fixed_in_dev This is merged into the dev environment and waiting to be merged into main
Projects
None yet
Development

No branches or pull requests

2 participants