Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add script to make graphs from problem size sweeps #245

Draft
wants to merge 43 commits into
base: develop
Choose a base branch
from

Conversation

MrBurmark
Copy link
Member

@MrBurmark MrBurmark commented May 12, 2022

Simplify creating graphs from problem size sweeps

Add a script that can create graphs from the output of our problem size sweep script.

  • This PR is a feature
  • It does the following:
    • Adds graphing script at the request of people who like graphs

Example workflow

Collecting data with sweep_size.sh

The sweep_size.sh script lets you run the perf suite and collect data into a file layout that the plotting script expects.

Here is an example invocation that I made on rzvernal, an ElCap early access machine with 8 gpus per node. This sweep ran with problem sizes 1024, 2048, …, 83886008 and output to a rzvernal_4.5.2 directory because I was testing the rocm 4.5.2 compiler. The arguments after – are passed directly to raja perf, I excluded the ATOMIC and INDEXLIST kernels due to long runtimes.

salloc -N1 -t480 ../scripts/sweep_size.sh -x rzvernal_4.5.2 "srun -n8 $(pwd)/bin/raja-perf.exe" --size-min 1024 --size-max 10000000 --size-ratio 2 -- --npasses 4 --npasses-combiners Minimum Average Maximum -ek PI_ATOMIC INDEXLIST -sp

Plotting data with sweep_graph.py

The sweep_graph.py script lets you graph the data collected by sweep_size.sh.

Here is an example of making line graphs of bandwidth vs problem size for every kernel.

../../../scripts/sweep_graph.py ../../rzvernal ../../rzansel -slg "bandwidth(GB/s)" -o graph_bandwidth

Here is an example of making a bar graph of bandwidth for all kernels and variants at the 8M problem size. Here the max operator us used to pick the max bandwidth of all tunings, collapsing out the tuning dimension.

../../../scripts/sweep_graph.py ../../rzvernal ../../rzansel -bg "max<bandwidth(GB/s)[run_size::8388608],tuning_index>" -o graph_bandwidth

Here is an example of making a bar graph of speedup for all kernels and variants at the 8M problem size vs the RAJA_CUDA variant on rzansel. The min here is used to pick the min runtime of all tunings, collapsing out the tuning dimension.

../../../scripts/sweep_graph.py ../../rzvernal ../../rzansel -bg "div<min<time(s)[run_size::8388608],tuning_index>[sweep_dir_name::rzansel][variant_index::RAJA_CUDA],min<time(s)[run_size::8388608],tuning_index>>" -ev Lambda_HIP Lambda_CUDA -ek Basic_INDEXLIST -o graph_speedup

@artv3
Copy link
Member

artv3 commented May 13, 2022

YES! If its ready do you want to mark it ready for review?

Copy link
Member

@rhornung67 rhornung67 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Please add a paragraph in the README.md file about running sweeps and the fact that these scripts exist to make plots.

@MrBurmark
Copy link
Member Author

There are still some rough edges in this script and I want to add some more comments.

This allows you to make new kinds of data from other kinds
without having to write every kind out beforehand, like
avg<time(s)> and avg<FLOPS>.
Now plotting multiple kinds of data will appear on the same graph
Added support for linear regression and segmented linear regression
@jonesholger jonesholger mentioned this pull request Aug 2, 2022
@rhornung67 rhornung67 mentioned this pull request Jul 10, 2023
24 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants