Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Start benchmarking with tango #15

Merged
merged 3 commits into from
Oct 15, 2024
Merged

Start benchmarking with tango #15

merged 3 commits into from
Oct 15, 2024

Conversation

kubouch
Copy link
Collaborator

@kubouch kubouch commented Oct 13, 2024

Sets up some benchmarks using tango-bench.

Apart from total compile time, each stage is also benchmarked separately.

Result on my machine (AMD Ryzen 9 5950X)

> target/benchmarks/compiler/benchmarks solo -s 100 --warmup true
def_parse                                           [   3.8 us ...   3.8 us ...   3.9 us ]  stddev:  22.7 ns
def_resolve                                         [ 854.0 ns ... 863.0 ns ... 880.5 ns ]  stddev:   5.6 ns
def_typecheck                                       [ 686.6 ns ... 725.7 ns ... 753.4 ns ]  stddev:  13.5 ns
def_resolve_merge                                   [   1.0 us ...   1.0 us ...   1.1 us ]  stddev:   7.1 ns
def_typecheck_merge                                 [ 692.3 ns ... 765.9 ns ... 789.9 ns ]  stddev:  15.0 ns
def_compile                                         [   4.9 us ...   5.0 us ...   5.0 us ]  stddev:  19.0 ns
def_nu_old                                          [   3.2 us ...   3.3 us ...   3.4 us ]  stddev:  29.6 ns
if_parse                                            [   4.8 us ...   4.8 us ...   4.9 us ]  stddev:  28.0 ns
if_resolve                                          [ 445.6 ns ... 450.4 ns ... 458.1 ns ]  stddev:   2.5 ns
if_typecheck                                        [ 402.8 ns ... 408.8 ns ... 416.7 ns ]  stddev:   3.8 ns
if_resolve_merge                                    [ 567.8 ns ... 575.0 ns ... 605.2 ns ]  stddev:   5.4 ns
if_typecheck_merge                                  [ 426.1 ns ... 433.1 ns ... 442.9 ns ]  stddev:   3.9 ns
if_compile                                          [   5.5 us ...   5.6 us ...   5.7 us ]  stddev:  28.5 ns
if_nu_old                                           [   4.8 us ...   4.9 us ...   5.0 us ]  stddev:  40.5 ns
nu_old_empty                                        [ 317.7 ns ... 325.0 ns ... 333.8 ns ]  stddev:   3.0 ns

@kubouch kubouch marked this pull request as ready for review October 13, 2024 15:15
@fdncred
Copy link
Contributor

fdncred commented Oct 13, 2024

This looks cool. It would be even cooler if we could run the same tests against nu-parser to see how new-nu-parser compares.

@kubouch
Copy link
Collaborator Author

kubouch commented Oct 14, 2024

I can try to add nu-parser benchmarks, good idea.

@kubouch
Copy link
Collaborator Author

kubouch commented Oct 14, 2024

Oops, so the new compiler appears slower, even in its incomplete implementation. I wonder why. Clearly, the parsing stage is much slower than everything else, that might be due to the constant peek() re-lexing.

@fdncred
Copy link
Contributor

fdncred commented Oct 14, 2024

Another item, while we're talking about benchmarks, might be memory consumption since one of the points is to remove spans. Not sure how to get there exactly but it's something to think about.

@kubouch
Copy link
Collaborator Author

kubouch commented Oct 15, 2024

Good idea. I'll merge this at least, but let's keep digging.

@kubouch kubouch merged commit 8a6aa37 into nushell:main Oct 15, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants