Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bottleneck Characterization #379

Draft
wants to merge 12 commits into
base: amd-staging
Choose a base branch
from
Draft

Bottleneck Characterization #379

wants to merge 12 commits into from

Conversation

coleramos425
Copy link
Collaborator

@coleramos425 coleramos425 commented Jun 17, 2024

This PR builds off of #242 which added "Bottleneck Characterization" to Omniperf, consisting of

  • End-to-end Runtime Classification plot
  • GPU Kernel Runtime Breakdown plot

I have modified the code so that the characterization and plotting modules match Omniperf's class based structure. See original PR for the feature's methodology and docs

Usage

To use the "Bottleneck Characterization" feature, just ensure omnitrace (specifically omnitrace-instrument) is available on your PATH. Omniperf will automatically run instrumentation as part of its profiling pipeline (disable using --no-trace). You can find your .proto output file in the standard Omniperf output directory.

Use analyze mode to generate the output plots (.pdf versions will also be saved by default). If for whatever reason you'd like to explicitly specify a .proto trace file you can do so in analysis using the --trace flag.

omniperf analyze -p workloads/dummy/MI200 --trace dummy_app.proto --gui

You'll notice the standalone GUI view now highlights these two plots in the previously empty top section of the webpage
image

Todo

benrichard-amd and others added 12 commits June 12, 2024 10:27
…376)

* Fix crash in multi-GPU scenario

Exclude -o option when invoking rocprof so that each rocprof process
writes to a different .csv file. Concatenate into a single .csv file
when finished.

Signed-off-by: benrichard-amd <[email protected]>

* Only combine csv files when using rocprofv2

rocprofv1 does not have separate csv files

Signed-off-by: benrichard-amd <[email protected]>

* Fix indices in combined CSV file

Use ignore_index flag to ensure there are no duplicate indices.

Signed-off-by: benrichard-amd <[email protected]>

* Fix Dispatch_ID column and remove unnamed column

-Pandas was inserting an unnamed column (index column)
-Overwrite the Dispatch_ID column so that every row is unique, starting at 0
-Remove fixup_rocprofv2_dispatch_ids as no longer needed

Signed-off-by: benrichard-amd <[email protected]>

* Fix code formatting

Signed-off-by: benrichard-amd <[email protected]>

* Fix code formatting (for real this time)

Signed-off-by: benrichard-amd <[email protected]>

---------

Signed-off-by: benrichard-amd <[email protected]>
* Workload Characterization Scripts

Adding three new workload characterization python scripts.

Signed-off-by: dwchang79 <[email protected]>

* Create Workload_Characterization_Manual.pdf

Documentation on how to use the Workload Characterization Tools

Signed-off-by: dwchang79 <[email protected]>

* Update Workload_Characterization_Manual.pdf

Minor additions to the documentation

---------

Signed-off-by: dwchang79 <[email protected]>
Signed-off-by: coleramos425 <[email protected]>
Signed-off-by: coleramos425 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants