feat: caching improvements for stats endpoints #420
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tazlin
changed the title
If the cron string (`p_schedule`) is different than the currently scheduled cron job, `schedule_cron_job` now makes sure that the schedule is updated.
tazlin
force-pushed
the
caching-improvements
branch
from
ec63577
to
fb2d30b
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Being victims of our own success, the increase in the size of the stats tables has led to the
/stats/*endpoints becoming slower and slower over time. To remedy this, I've introduced some new tables, new stored procedures, and pg_cron into the mix so these endpoints are always precomputed and that the backends have fewer long-running threads. See thesql_statements/README.mdfor some additional information.This also creates the
known_image_modelstable which is populated with the entries from the model reference. This serves to allow us to one day refactor themodel_referenceshared module object out in preference for this table.