Persistent compile cache

[fantix]

Objective

• Compiled EdgeQL cache (SQLs) should survive server restarts, so that restarts won't cause potentially large drops in quality of service (like a slow query) when rebuilding the cache.
• (secondary) Reduce server memory footprint and backend network load.

Proposal

Dynamically create individual SQL functions for each compiled EdgeQL (named/indexed by query / argument types hash and schema version) in order to replace the current in-memory LRUMapping. The EdgeDB server should keep an up-to-date list of cached queries (names of functions) to avoid extra round-trip to the backend checking the existence of the cache of a certain query, so that the server could either just execute that function (cache hit), or compile + create function + execute function (cache miss) with minimal I/O to the backend, when handling an EdgeQL query request.

Action items

• Test how well Postgres handles concurrent changes to the functions table under load
• Design for multi-backend-tenancy (multiple EdgeDB servers on one backend) - basically a notification for other tenants to reload what's cached in the backend
• Design the cleanup of the "cache" (functions table) when hitting maxsize
• Make sure prepared statements work fine
• Benchmark to see how the design performs
• Use CALL Postgres protocol message to call the cached funcs

[msullivan]

We have merged a working persistent cache, but need some additional work to get functions working.

[msullivan]

Current issues:

• Test failures when using Add EDGEDB_TEST_REPEATS env var for testing function cache #7497
• Increased log spew because of DROP FUNCTION IF EXISTS
• I once observed "function does not exist" failures trying to call the cache function. It seemed nondeterministic. Need to try to reproduce.