Postgresql parameters for db query investigations

[alismx]

DEVOPS/DATABASE PULL REQUEST

Related Issue

• resolves Explore database configurations to help with investigations and tuning query usage and performance #4318

Changes Proposed

• Installs pg_stat_statement extension ( this will be installed manually )
• Sets up terraform configuration for the following PostgreSQL parameters
  • log_min_duration_statement
  • track_io_timing
  • pg_stat_statements.track
  • auto_explain.log_analyze
  • auto_explain.log_min_duration
• Adds auto_explain to our databases list of shared_preload_libraries (previously was pg_cron,pg_stat_statements)
• Limits our azure.extensions to extensions we use. This was manually set up and had way more than we needed or used.

Additional Information

• Manual rollout steps:

  • Create a db backup
  • install pg_stat_statements
  • All databases will need to be restarted to add auto_explain to our list of shared_preload_libraries
  • merge this pr
  • Smoke test environment
  • delete db backup

• I tried to add the pg_stat_statements extension with a backend migration, but our migrations user doesn't have permission

• All these parameters have been tested to work but have been disabled because these options can cause performance issues, so we only want to enable them when we believe we need them.

• This is work to enable future devs to investigate query issues.

• I'd like to run an exercise using these parameters to gather baseline data about our queries.

Wiki Page: https://github.com/CDCgov/prime-simplereport/wiki/Database-investigation-tools

Testing

• How should reviewers verify this PR?

• Check the terraform plan.

Checklist for Primary Reviewer

Database

• Only database changes are included in this PR
• Any new tables or columns that do not contain PII are accompanied by a GRANT SELECT to the no-PHI user
• Any changes to tables that have custom no-PHI views are accompanied by changes to those views (including re-granting permission to the no-PHI user if need be)
• Each new changeset has a corresponding tag
• Rollback has been verifed locally and in a deployed environment
• Any changes to the startup configuration have been documented in the README

Infrastructure

• Consult the results of the terraform-plan job inside the "Terraform Checks" workflow run for this PR. Confirm that there are no unexpected changes!

Security

• Changes with security implications have been approved by a security engineer (changes to authentication, encryption, handling of PII, etc.)
• Any dependencies introduced have been vetted and discussed

Cloud

• Oncall has been notified if this change is going in after-hours
• If there are changes that cannot be tested locally, this has been deployed to our Azure test, dev, or pentest environment for verification

Documentation

• Any changes to the startup configuration have been documented in the README

[emyl3]

Question for my own learning: is there a good value to start off with when turning this on? Is every minute too much too little? 🤔

[alismx]

The answer to this depends very much on the context, but I expect most queries to complete well under 1 second, so in most cases, setting it to something between 2 and 5 seconds would probably be fine.

[emyl3]

Left one non-blocking question! Thank you for adding docs on how to use this 😲

[BobanL]

Changes look good! Thanks for adding the documentation 😄

[rin-skylight]

Quick note: Let's make sure we take a backup of the prod DB before we roll this out. Just in case!

[alismx]

I'll deploy this and perform the manual DB updates on the week of July 26th.

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information