Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run fork tests on multiple chains #599

Closed
PaulRBerg opened this issue Jul 2, 2023 · 5 comments
Closed

Run fork tests on multiple chains #599

PaulRBerg opened this issue Jul 2, 2023 · 5 comments
Labels
effort: epic Multi-stage task that may require multiple PRs. priority: 3 Nice-to-have. Willing to ship without this. type: test Adding, updating, or removing tests. work: complicated Sense-analyze-respond. The relationship between cause and effect requires analysis or expertise.

Comments

@PaulRBerg
Copy link
Member

PaulRBerg commented Jul 2, 2023

The fork tests are currently run against a single chain.

Ref: foundry-rs/book#872

@PaulRBerg PaulRBerg added effort: epic Multi-stage task that may require multiple PRs. type: test Adding, updating, or removing tests. work: complicated Sense-analyze-respond. The relationship between cause and effect requires analysis or expertise. priority: 3 Nice-to-have. Willing to ship without this. and removed backlog labels Jan 1, 2024
@andreivladbrg
Copy link
Member

andreivladbrg commented Apr 4, 2024

I was thinking to address this issue. Since forge does not provide helpers I am thinking to apply this methodology:

  • add a new foundry profile (fork multi chain - or any other name)
  • this profile is going to be used only in ci-fork
  • implement a util function to check this profile, similar to isTestOptimizedProfile
  • if the profile is specified run the fork tests against all endpoints in foundry.toml, if any other profile is used, run only on mainnet
  • implement a modifier to select all forks and run the same tests for them
  • introduce new datatypes for all information needed

I am currently working on this locally, and it's going well. I will add the branch here once I push any commit.

Reading the code will also make it easier to understand my methodology from above.

@PaulRBerg tagging you to get notification

@andreivladbrg
Copy link
Member

I have to admit, this is taking much longer than expected. One factor that influences the speed of the process is that it takes forever to run fork tests against multiple assets and chains.

I have successfully implemented this on this branch, but only for Linear.

@smol-ninja
Copy link
Member

Wouldn't it burn RPC endpoints for fork tests? I am afraid we might be seeing more frequent stalled/failed tests due to rate limiting. And don't forget it already takes ~1 hr currently and with new fork tests, it may take ~6-7 hours or may be longer.

Andrei, how long it takes to run all fork tests (i.e. 20 runs) on your machines?

I may be wrong here but I think its overkill to run fork tests on all chains (after all they all are EVMs) 😅.

@PaulRBerg
Copy link
Member Author

I agree with Shub, it's overkill to run cross-chain forks tests.

We're not a bridge so we are not interested in cross-chain communication. The Sablier protocol is siloed.

after all they all are EVMs

There are some important differences in opcode pricing and gas calculations, but we at Sablier have the privilege of being able to ignore them.

@PaulRBerg
Copy link
Member Author

Closing. Thank you for your efforts thus far @andreivladbrg.

@PaulRBerg PaulRBerg closed this as not planned Won't fix, can't repro, duplicate, stale Apr 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
effort: epic Multi-stage task that may require multiple PRs. priority: 3 Nice-to-have. Willing to ship without this. type: test Adding, updating, or removing tests. work: complicated Sense-analyze-respond. The relationship between cause and effect requires analysis or expertise.
Projects
None yet
Development

No branches or pull requests

3 participants