Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature] avoid repopulating table from scratch #934

Open
exp-table opened this issue Jun 11, 2024 · 1 comment
Open

[feature] avoid repopulating table from scratch #934

exp-table opened this issue Jun 11, 2024 · 1 comment

Comments

@exp-table
Copy link

We use a ponder to track quests completion for our project which involves different actions taken with different contracts.
To add a quest, we basically add a new contract, a new schema and listen for the proper events with a new listener.

Once the new code is pushed, it triggers a new deployment which results in ponder having to restart from scratch (it wipes the tables, create new ones and repopulate them).

I will note that we are not using Railway's DB system but Supabase's.

This results in two issues ;

  • unavailable or incomplete data until complete sync (which can take some time for some big tables)
  • I/O to the database which results in more usage and thus cost

Perhaps there could be a flag (which would be false by default) that stops ponder to populate the tables from scratch and pick up from where it left with the use of checkpoints?

@zkSoju
Copy link

zkSoju commented Jul 21, 2024

Following up here because have been in talks with Kyle on this, but having difficulties coming up with a long term solution. It takes a very long time for each deployment to become healthy (several hours+ depending on what is indexed) which effects production usage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants