storage controller: don't hold detached projects in memory #9712
Labels
c/storage/controller
Component: Storage Controller
t/feature
Issue type: feature, for new features or requests
Some use cases for Neon involve storing large numbers of long-term-idle tenants. We already safe resources by detaching them from pageservers, but these tenants remain resident in memory on the storage controller.
Our motivation for storing tenants in-memory (rather than always going to the database) is to enable fast scheduling decisions, but detached tenants are not involved in scheduling.
We should skip these tenants when loading from the database on startup, drop them from the in-memory map of shards when setting them to detached, and then on-demand re-instantiate them when we receive a configuration request that can move them from detached to another state (i.e. a request for /location_conf or to set the tenant's placement policy directly).
The text was updated successfully, but these errors were encountered: