Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Profiler probes for non-sequence steps #15518

Merged
merged 1 commit into from
Oct 17, 2024

Conversation

cameel
Copy link
Member

@cameel cameel commented Oct 16, 2024

I realized that the only steps covered by our profiler are the ones that go through runSequence(). Disambiguator, NameSimplifier, StackCompressor, StackLimitEvader, NameSimplifier, VarNameCleaner are not included, which undercounts the actual impact of the optimizer by quite a bit (33% in case of Uniswap). This PR adds probes for these elements.

@cameel
Copy link
Member Author

cameel commented Oct 16, 2024

Here's how it looks in benchmarks. The tables show the timing of the added probes (in italics), with total optimization time and top 3 steps included for comparison.

Looks like it's closer to 15%. I assumed 33% based on the fact that for Uniswap I used to see ~60 s of optimization time and now that's up to ~80 s, but apparently this run was just particularly slow.

The impact of NameSimplifier being 4-6% is a lot more than I expected.

OpenZeppelin

Compilation time: 33 s.
Time spent in added probes: 4 s (16% of optimization time).

Time % Time Calls Scope
0.1% 0.031 s 274 ConstantOptimiser
0.1% 0.033 s 274 VarNameCleaner
1.1% 0.261 s 274 Disambiguator
3.8% 0.890 s 274 StackLimitEvader
4.3% 0.998 s 274 StackCompressor
6.3% 1.470 s 548 NameSimplifier
7.5% 1.750 s 2740 UnusedPruner
7.9% 1.834 s 1370 UnusedAssignEliminator
16.8% 3.917 s 3294 CommonSubexpressionEliminator
100.0% 23.259 s 40588 TOTAL

Uniswap

Compilation time: 130 s.
Time spent in added probes: 12 s (15% of optimization time).

Time % Time Calls Scope
0.1% 0.051 s 270 ConstantOptimiser
0.3% 0.261 s 270 VarNameCleaner
0.8% 0.612 s 270 Disambiguator
4.4% 3.508 s 540 NameSimplifier
4.5% 3.575 s 270 StackLimitEvader
4.9% 3.965 s 270 StackCompressor
7.6% 6.086 s 1350 UnusedAssignEliminator
8.2% 6.570 s 1391 ExpressionSimplifier
17.2% 13.785 s 3281 CommonSubexpressionEliminator
100.0% 80.193 s 40206 TOTAL

Eigenlayer

Compilation time: 556 s.
Time spent in added probes: 48 s (15% of optimization time).

Time % Time Calls Scope
0.1% 0.272 s 376 ConstantOptimiser
0.6% 1.875 s 376 VarNameCleaner
0.7% 2.163 s 376 Disambiguator
3.5% 11.434 s 752 NameSimplifier
4.8% 15.680 s 376 StackLimitEvader
5.1% 16.761 s 376 StackCompressor
7.9% 26.112 s 1972 ExpressionSimplifier
9.3% 30.850 s 3852 LiteralRematerialiser
17.0% 56.100 s 4604 CommonSubexpressionEliminator
100.0% 329.952 s 56200 TOTAL

Comment on lines -126 to +133
NameSimplifier::run(suite.m_context, astRoot);
{
PROFILER_PROBE("NameSimplifier", probe);
NameSimplifier::run(suite.m_context, astRoot);
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder why we're running NameSimplifier twice. It has been that way ever since it was added in #9591. I don't see any specific justification for this in that PR though.

Removing it could shave off 2-3% of the optimizer running time.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking that perhaps it yields nicer names in some cases, but at least for our test suite it doesn't even change anything.

Copy link
Member Author

@cameel cameel Oct 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. You could submit that as a PR then. We should run it by Daniel just to make sure there wasn't any obscure reason he's aware of but if it's having no effect, we should really get rid of it. It's too expensive to run it without a good reason.

@cameel cameel merged commit cb576b1 into develop Oct 17, 2024
74 checks passed
@cameel cameel deleted the profiler-probes-for-non-sequence-steps branch October 17, 2024 17:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants