Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update benchmark base #2788

Merged

Conversation

officialasishkumar
Copy link
Member

@officialasishkumar officialasishkumar commented Aug 9, 2024

📝 Description

Type: 🪲 bugfix

  • Pass tardis-refdata via env variable
  • update get_relative_path

📌 Resources

Examples, notebooks, and links to useful references.

🚦 Testing

How did you test these changes?

  • Testing pipeline
  • Other method (describe)
  • My changes can't be tested (explain why)

☑️ Checklist

  • I requested two reviewers for this pull request
  • I updated the documentation according to my changes
  • I built the documentation by applying the build_docs label

Note: If you are not allowed to perform any of these actions, ping (@) a contributor.

@tardis-bot
Copy link
Contributor

tardis-bot commented Aug 9, 2024

*beep* *bop*
Hi human,
I ran ruff on the latest commit (3e09abf).
Here are the outputs produced.
Results can also be downloaded as artifacts here.
Summarised output:

1	B018  	[ ] Found useless expression. Either assign it to a variable or remove it.
1	RET506	[ ] Unnecessary `else` after `raise` statement
1	I001  	[*] Import block is un-sorted or un-formatted
1	W292  	[*] No newline at end of file

Complete output(might be large):

asv.conf.json:1:1: B018 Found useless expression. Either assign it to a variable or remove it.
asv.conf.json:32:2: W292 [*] No newline at end of file
benchmarks/benchmark_base.py:1:1: I001 [*] Import block is un-sorted or un-formatted
benchmarks/benchmark_base.py:75:9: RET506 Unnecessary `else` after `raise` statement
Found 4 errors.
[*] 2 fixable with the `--fix` option.

Copy link

codecov bot commented Aug 9, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 69.32%. Comparing base (2df54e5) to head (3e09abf).
Report is 7 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2788      +/-   ##
==========================================
- Coverage   69.87%   69.32%   -0.55%     
==========================================
  Files         205      205              
  Lines       15440    15440              
==========================================
- Hits        10788    10704      -84     
- Misses       4652     4736      +84     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@officialasishkumar officialasishkumar changed the title Update get_relative_path with Pathlib in benchmark_base Update benchmark base Aug 9, 2024
"html_dir": ".asv/html",
"matrix": {
"env": {
"TARDIS_REF_PATH": "tardis-refdata/atom_data"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does the atom_data needs to be passed by a different env variable?

@tardis-bot
Copy link
Contributor

tardis-bot commented Aug 9, 2024

*beep* *bop*
Hi human,
I ran benchmarks as you asked comparing master (2df54e5) and the latest commit (3e09abf).
Here are the logs produced by ASV.
Results can also be downloaded as artifacts here.

Significantly changed benchmarks:

All benchmarks:

Benchmarks that have stayed the same:

| Change   | Before [2df54e5d] <master>   | After [3e09abf2]    | Ratio   | Benchmark (Parameter)                                                                                                               |
|----------|------------------------------|---------------------|---------|-------------------------------------------------------------------------------------------------------------------------------------|
|          | 4.84±0.02ms                  | 7.19±0.01ms         | ~1.49   | opacities_opacity_state.BenchmarkOpacitiesOpacityState.time_opacity_state_initialize('macroatom')                                   |
|          | 2.57±0.5ms                   | 2.95±0.7ms          | ~1.15   | transport_montecarlo_single_packet_loop.BenchmarkTransportMontecarloSinglePacketLoop.time_single_packet_loop                        |
|          | 3.07±0.8μs                   | 3.43±0.7μs          | ~1.12   | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_bad_vpacket                                       |
|          | 6.75±1μs                     | 6.05±1μs            | ~0.90   | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_vpacket                                           |
|          | 51.9±20μs                    | 46.0±20μs           | ~0.89   | transport_montecarlo_interaction.BenchmarkTransportMontecarloInteraction.time_line_scatter                                          |
|          | 56.3±10μs                    | 43.2±20μs           | ~0.77   | transport_montecarlo_interaction.BenchmarkTransportMontecarloInteraction.time_line_emission                                         |
|          | 2.62±0.01ms                  | 2.87±0ms            | 1.09    | opacities_opacity_state.BenchmarkOpacitiesOpacityState.time_opacity_state_initialize('scatter')                                     |
|          | 541±100ns                    | 570±200ns           | 1.05    | opacities_opacity.BenchmarkMontecarloMontecarloNumbaOpacities.time_pair_creation_opacity_calculation                                |
|          | 2.00±1μs                     | 2.09±0.8μs          | 1.04    | transport_montecarlo_estimators_radfield_estimator_calcs.BenchmarkMontecarloMontecarloNumbaPacket.time_update_line_estimators       |
|          | 7.69±2μs                     | 8.01±2μs            | 1.04    | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_vpacket_volley                                    |
|          | 30.6±0.03μs                  | 31.4±0.05μs         | 1.03    | transport_montecarlo_packet_trackers.BenchmarkTransportMontecarloPacketTrackers.time_generate_rpacket_tracker_list                  |
|          | 521±200ns                    | 531±200ns           | 1.02    | opacities_opacity.BenchmarkMontecarloMontecarloNumbaOpacities.time_compton_opacity_calculation                                      |
|          | 206±0.2ns                    | 209±0.3ns           | 1.02    | spectrum_formal_integral.BenchmarkTransportMontecarloFormalIntegral.time_intensity_black_body                                       |
|          | 1.34±0.4μs                   | 1.37±0.4μs          | 1.02    | transport_geometry_calculate_distances.BenchmarkTransportGeometryCalculateDistances.time_calculate_distance_line                    |
|          | 62.0±0.2ms                   | 63.3±0.03ms         | 1.02    | transport_montecarlo_packet_trackers.BenchmarkTransportMontecarloPacketTrackers.time_rpacket_trackers_to_dataframe                  |
|          | 1.65±0ms                     | 1.66±0.01ms         | 1.01    | transport_montecarlo_main_loop.BenchmarkTransportMontecarloMontecarloMainLoop.time_montecarlo_main_loop                             |
|          | 39.0±0.01s                   | 38.8±0.03s          | 1.00    | run_tardis.BenchmarkRunTardis.time_run_tardis                                                                                       |
|          | 1.05±0m                      | 1.05±0m             | 1.00    | run_tardis.BenchmarkRunTardis.time_run_tardis_rpacket_tracking                                                                      |
|          | 2.06±0m                      | 2.06±0m             | 1.00    | spectrum_formal_integral.BenchmarkTransportMontecarloFormalIntegral.time_FormalIntegrator_functions                                 |
|          | 1.20±0μs                     | 1.21±0μs            | 1.00    | transport_geometry_calculate_distances.BenchmarkTransportGeometryCalculateDistances.time_calculate_distance_boundary                |
|          | 726±0.3ns                    | 721±0.3ns           | 0.99    | transport_montecarlo_interaction.BenchmarkTransportMontecarloInteraction.time_thomson_scatter                                       |
|          | 19.7±4μs                     | 19.5±4μs            | 0.99    | transport_montecarlo_packet_trackers.BenchmarkTransportMontecarloPacketTrackers.time_generate_rpacket_last_interaction_tracker_list |
|          | 3.38±0.6μs                   | 3.34±0.6μs          | 0.99    | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_vpacket_within_shell                              |
|          | 581±200ns                    | 532±200ns           | 0.92    | opacities_opacity.BenchmarkMontecarloMontecarloNumbaOpacities.time_photoabsorption_opacity_calculation                              |

If you want to see the graph of the results, you can check it here

andrewfullard
andrewfullard previously approved these changes Aug 12, 2024
@andrewfullard andrewfullard enabled auto-merge (squash) August 12, 2024 13:59
@@ -1,6 +1,6 @@
import functools
import os
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from os get environ perhaps

Signed-off-by: Asish Kumar <[email protected]>
auto-merge was automatically disabled August 12, 2024 14:01

Head branch was pushed to by a user without write access

DeerWhale
DeerWhale previously approved these changes Aug 12, 2024
@andrewfullard andrewfullard self-requested a review August 12, 2024 14:02
andrewfullard
andrewfullard previously approved these changes Aug 12, 2024
jvshields
jvshields previously approved these changes Aug 12, 2024
@jvshields jvshields enabled auto-merge (squash) August 12, 2024 14:02
Signed-off-by: Asish Kumar <[email protected]>
auto-merge was automatically disabled August 12, 2024 14:09

Head branch was pushed to by a user without write access

jvshields
jvshields previously approved these changes Aug 12, 2024
andrewfullard
andrewfullard previously approved these changes Aug 12, 2024
@andrewfullard andrewfullard enabled auto-merge (squash) August 12, 2024 14:15
@andrewfullard andrewfullard merged commit fad3987 into tardis-sn:master Aug 16, 2024
14 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

6 participants