Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add note of caution to performance ratios and uncomment tests #174

Merged
merged 2 commits into from
May 28, 2024

Conversation

willtebbutt
Copy link
Member

@willtebbutt willtebbutt commented May 28, 2024

The performance ratios will differ a lot depending on where you run them. Add a note to tell people to be cautious.

edit: additionally, in a previous PR, I had accidentally deactivated a large number of tests that really ought not to have been deactivated. Revert that.

@willtebbutt willtebbutt changed the title Add note of caution to performance ratios Add note of caution to performance ratios and uncomment tests May 28, 2024
Copy link
Contributor

Performance Ratio:
Warning: results are very approximate!

┌────────────────────────────┬────────┬─────────┬─────────────┬─────────┐
│                      Label │  Tapir │  Zygote │ ReverseDiff │  Enzyme │
│                     String │ String │  String │      String │  String │
├────────────────────────────┼────────┼─────────┼─────────────┼─────────┤
│                        sum │   29.7 │   0.324 │        2.27 │   0.494 │
│                       _sum │   6.64 │   483.0 │        27.6 │   0.121 │
│                   kron_sum │   75.6 │    3.45 │       189.0 │    21.9 │
│              kron_view_sum │   84.2 │    9.84 │       224.0 │     7.1 │
│      naive_map_sin_cos_exp │   4.16 │ missing │         8.7 │    2.88 │
│            map_sin_cos_exp │   4.67 │    1.73 │        7.61 │    3.41 │
│      broadcast_sin_cos_exp │   4.41 │    2.65 │        1.67 │    2.83 │
│                 simple_mlp │   9.16 │    3.27 │        11.4 │    3.23 │
│                     gp_lml │   17.1 │    4.36 │     missing │ missing │
│ turing_broadcast_benchmark │   6.33 │ missing │        39.9 │ missing │
└────────────────────────────┴────────┴─────────┴─────────────┴─────────┘

@willtebbutt willtebbutt merged commit a0a67e2 into main May 28, 2024
16 checks passed
@willtebbutt willtebbutt deleted the wct/performance-note branch May 28, 2024 18:58
@willtebbutt willtebbutt mentioned this pull request May 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant