Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Persistent cache #6589

Closed
wants to merge 20 commits into from
Closed

Persistent cache #6589

wants to merge 20 commits into from

Conversation

fantix
Copy link
Member

@fantix fantix commented Dec 11, 2023

Closes #6349

  • Add LRUMapping.gc() we can use stmt_cache.pyx
  • Pass over the user schema version in DB
  • Replace dbver with the non-local schema version
  • Create cache registry table
  • Rewrite QueryUnits to use SQL functions
  • Implement persistent cache
  • Implement cache eviction
  • Adopt Lock concurrent compiles of the same query #6349
  • Add tests

Queries after DDL in a transaction have been running without cache for
some time, dropping the dead code for now; maybe we will add it back in
the future properly.
During a query compilation, if the schema is changed, the server was
caching the compiled query under a wrong dbver, causing future queries
to fail. See the test for issue reproduction.
@fantix fantix force-pushed the persistent-cache branch 3 times, most recently from d184c52 to f617fe1 Compare December 12, 2023 15:44
@@ -535,7 +535,13 @@ cdef class DatabaseConnectionView:
raise errors.InternalServerError(
'no need to serialize state while in transaction')

ver = self._db.schema_version
ver = hash(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will ver ever end up in Postgres or in another Python process? hash() as you know is randomized

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Negative. In Postgres, it's in the state temporary table and never used.

@fantix fantix force-pushed the persistent-cache branch 2 times, most recently from 15a04d4 to 83f17ec Compare December 28, 2023 18:28
@fantix fantix changed the title [WIP] Persistent cache Persistent cache Jan 31, 2024
@fantix
Copy link
Member Author

fantix commented Mar 13, 2024

Close in favor of #6881 and many others.

@fantix fantix closed this Mar 13, 2024
@fantix fantix deleted the persistent-cache branch March 13, 2024 15:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants