Updated Doc for Intel XPU Profile #3013

louie-tsai · 2024-08-26T22:13:50Z

Description

PyTorch Profiling changes for XPU.

Landing page:
https://pytorch.org/tutorials/recipes/recipes/profiler_recipe.html
https://pytorch.org/tutorials/recipes/profile_with_itt.html

Checklist

The issue that is being fixed is referred in the description (see above "Fixes #ISSUE_NUMBER")
Only one issue is addressed in this pull request
Labels from the issue that this PR is fixing are added to this pull request
No unnecessary issues are included into this pull request.

cc @gujinghui @EikanWang @fengyuan14 @guangyey

pytorch-bot · 2024-08-26T22:13:53Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/tutorials/3013

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

[PREEMTIVE] Experimenting with new runners linux.aws.a100 on inductor-perf-compare.yml

✅ No Failures

As of commit a5408b1 with merge base b725032 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2024-08-26T22:13:56Z

Hi @louie-tsai!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at [email protected]. Thanks!

recipes_source/recipes/profiler_recipe.py

recipes_source/profile_with_itt.rst

recipes_source/recipes/profiler_recipe.py

malfet

Changes to profile_with_itt looks fine to me, but changes to profiler_recpie makes it really hard to read. Perhaps one should try to find an easier to use interface for collecting/analyzing profiler information from accelerators?

louie-tsai · 2024-09-16T23:42:43Z

Changes to profile_with_itt looks fine to me, but changes to profiler_recpie makes it really hard to read. Perhaps one should try to find an easier to use interface for collecting/analyzing profiler information from accelerators?

@malfet
The profiler_recipe contents will also be populated into a html file : https://pytorch.org/tutorials/recipes/recipes/profiler_recipe.html
Hope that html file format addresses your concern for "hard to read".

malfet · 2024-09-16T23:46:13Z

The profiler_recipe contents will also be populated into a html file : https://pytorch.org/tutorials/recipes/recipes/profiler_recipe.html Hope that html file format addresses your concern for "hard to read".

Sorry, I was not talking about this one, but rather about the following code:

if device == 'cuda':
   activities.append(ProfilerActivity.CUDA)
elif device == 'xpu':
   activities.append(ProfilerActivity.XPU)

As one can not compile PyTorch with CUDA and XPU at the same time, why introducing new enum rather than creating an alias(i.e. ProfilerActivity.XPU == ProfilerActivity.CUDA), so that users do not have to rewrite their programs when migrating from one accelerator to another?

jingxu10 · 2024-09-18T03:01:54Z

The profiler_recipe contents will also be populated into a html file : https://pytorch.org/tutorials/recipes/recipes/profiler_recipe.html Hope that html file format addresses your concern for "hard to read".

Sorry, I was not talking about this one, but rather about the following code:
if device == 'cuda':
   activities.append(ProfilerActivity.CUDA)
elif device == 'xpu':
   activities.append(ProfilerActivity.XPU)
As one can not compile PyTorch with CUDA and XPU at the same time, why introducing new enum rather than creating an alias(i.e. ProfilerActivity.XPU == ProfilerActivity.CUDA), so that users do not have to rewrite their programs when migrating from one accelerator to another?

Hi @malfet , it doesn't seem to be something can be fixed in this tutorial. It involves code changes. We will see how to address this concern in 2.6 with a separate PR to change profiler code.

dvrogozh · 2024-09-18T14:26:00Z

As one can not compile PyTorch with CUDA and XPU at the same time

@malfet : why? this should be technically possible if someone will install both CUDA and XPU stacks on the system. And might be convenient for some people (mostly for debug I guess) to build pytorch with both path enabled in a single build.

louie-tsai · 2024-10-02T18:02:44Z

As one can not compile PyTorch with CUDA and XPU at the same time

@malfet : why? this should be technically possible if someone will install both CUDA and XPU stacks on the system. And might be convenient for some people (mostly for debug I guess) to build pytorch with both path enabled in a single build.

@malfet
Any feedback? Ex: some laptops also have both Intel iGPU and NVidia dGPU at the same time, so it might be common to have both XPU and CUDA.

recipes_source/recipes/profiler_recipe.py

malfet · 2024-10-05T01:10:14Z

recipes_source/recipes/profiler_recipe.py

+######################################################################
+# (Note: the first use of XPU profiling may bring an extra overhead.)
+
+######################################################################
+# The resulting table output (omitting some columns):
+#
+# .. code-block:: sh
+#
+#-------------------------------------------------------  ------------  ------------  ------------  ------------  ------------
+#                                                   Name    Self XPU    Self XPU %     XPU total  XPU time avg    # of Calls
+#  -------------------------------------------------------   ------------  ------------  ------------  ------------  ------------
+#                                        model_inference      0.000us         0.00%       2.567ms       2.567ms             1
+#                                           aten::conv2d      0.000us         0.00%       1.871ms      93.560us            20
+#                                      aten::convolution      0.000us         0.00%       1.871ms      93.560us            20
+#                                     aten::_convolution      0.000us         0.00%       1.871ms      93.560us            20
+#                         aten::convolution_overrideable      1.871ms        72.89%       1.871ms      93.560us            20
+#                                               gen_conv      1.484ms        57.82%       1.484ms      74.216us            20
+#                                       aten::batch_norm      0.000us         0.00%     432.640us      21.632us            20
+#                           aten::_batch_norm_impl_index      0.000us         0.00%     432.640us      21.632us            20
+#                                aten::native_batch_norm      432.640us      16.85%     432.640us      21.632us            20
+#                                           conv_reorder      386.880us      15.07%     386.880us       6.448us            60
+#  -------------------------------------------------------   ------------  ------------  ------------  ------------  ------------
+#  Self CPU time total: 712.486ms
+#  Self XPU time total: 2.567ms


What value does this extra table brings to the user?

there are indeed just minor changes including some different operators and also XPU instead of GPU in this table. just want people to understand how it might look like for the output.

recipes_source/recipes/profiler_recipe.py

louie-tsai · 2024-10-09T18:58:05Z

@malfet @jingxu10 @dvrogozh @guangyey
please help to merge it if no concern for the change.

thanks!

louie-tsai · 2024-10-15T22:29:43Z

@malfet
updated accordingly. hope it works for you.

svekars · 2024-10-16T16:19:06Z

To fix spelling, please add these to the en-wordlist.txt:

_batch_norm_impl_index
convolution_overrideable
aten
XPU

Update profiler_recipe.py to unify the accelerators python codes

louie-tsai · 2024-10-22T18:20:37Z

To fix spelling, please add these to the en-wordlist.txt:

_batch_norm_impl_index

convolution_overrideable

aten

XPU

addressed them accordingly. please also help to approve it if all look good to you.

malfet

Feels a bit verbose to me, but looks fine.

svekars added the module: xpu XPU related issues label Aug 26, 2024

dvrogozh suggested changes Aug 26, 2024

View reviewed changes

recipes_source/recipes/profiler_recipe.py Outdated Show resolved Hide resolved

recipes_source/recipes/profiler_recipe.py Outdated Show resolved Hide resolved

louie-tsai requested review from guangyey and dvrogozh August 29, 2024 00:27

facebook-github-bot added the cla signed label Aug 29, 2024

guangyey reviewed Aug 29, 2024

View reviewed changes

recipes_source/recipes/profiler_recipe.py Outdated Show resolved Hide resolved

louie-tsai force-pushed the xpu_profile branch 3 times, most recently from 5692f57 to 9eb8734 Compare September 4, 2024 23:50

dvrogozh suggested changes Sep 5, 2024

View reviewed changes

recipes_source/profile_with_itt.rst Outdated Show resolved Hide resolved

recipes_source/recipes/profiler_recipe.py Outdated Show resolved Hide resolved

malfet requested changes Sep 16, 2024

View reviewed changes

louie-tsai force-pushed the xpu_profile branch from 49771be to d2a7183 Compare September 16, 2024 23:37

louie-tsai requested review from malfet, dvrogozh and guangyey September 17, 2024 23:57

louie-tsai force-pushed the xpu_profile branch from d2a7183 to c382e5d Compare October 2, 2024 18:02

malfet reviewed Oct 5, 2024

View reviewed changes

recipes_source/recipes/profiler_recipe.py Outdated Show resolved Hide resolved

malfet reviewed Oct 5, 2024

View reviewed changes

recipes_source/recipes/profiler_recipe.py Outdated Show resolved Hide resolved

malfet reviewed Oct 5, 2024

View reviewed changes

recipes_source/recipes/profiler_recipe.py Outdated Show resolved Hide resolved

louie-tsai force-pushed the xpu_profile branch from c382e5d to b0fcfdd Compare October 9, 2024 18:55

louie-tsai requested a review from malfet October 9, 2024 18:57

louie-tsai force-pushed the xpu_profile branch from 7fe0506 to a66e75f Compare October 15, 2024 22:28

louie-tsai force-pushed the xpu_profile branch from 2f062bf to c3df122 Compare October 22, 2024 17:52

add xpu profiling files and recipes_source/profile_with_itt.rst

ef7d0d3

Update profiler_recipe.py to unify the accelerators python codes

louie-tsai force-pushed the xpu_profile branch from c3df122 to ef7d0d3 Compare October 22, 2024 18:00

Update en-wordlist.txt

a5408b1

malfet approved these changes Oct 24, 2024

View reviewed changes

malfet merged commit 2f3f3fa into pytorch:main Oct 24, 2024
20 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updated Doc for Intel XPU Profile #3013

Updated Doc for Intel XPU Profile #3013

louie-tsai commented Aug 26, 2024 •

edited by pytorch-bot bot

Loading

pytorch-bot bot commented Aug 26, 2024 •

edited

Loading

facebook-github-bot commented Aug 26, 2024

malfet left a comment

louie-tsai commented Sep 16, 2024

malfet commented Sep 16, 2024

jingxu10 commented Sep 18, 2024

dvrogozh commented Sep 18, 2024

louie-tsai commented Oct 2, 2024 •

edited

Loading

malfet Oct 5, 2024

louie-tsai Oct 15, 2024

louie-tsai commented Oct 9, 2024

louie-tsai commented Oct 15, 2024

svekars commented Oct 16, 2024 •

edited

Loading

louie-tsai commented Oct 22, 2024

malfet left a comment

Updated Doc for Intel XPU Profile #3013

Updated Doc for Intel XPU Profile #3013

Conversation

louie-tsai commented Aug 26, 2024 • edited by pytorch-bot bot Loading

Description

Checklist

pytorch-bot bot commented Aug 26, 2024 • edited Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/tutorials/3013

❗ 1 Active SEVs

✅ No Failures

facebook-github-bot commented Aug 26, 2024

Action Required

Process

malfet left a comment

Choose a reason for hiding this comment

louie-tsai commented Sep 16, 2024

malfet commented Sep 16, 2024

jingxu10 commented Sep 18, 2024

dvrogozh commented Sep 18, 2024

louie-tsai commented Oct 2, 2024 • edited Loading

malfet Oct 5, 2024

Choose a reason for hiding this comment

louie-tsai Oct 15, 2024

Choose a reason for hiding this comment

louie-tsai commented Oct 9, 2024

louie-tsai commented Oct 15, 2024

svekars commented Oct 16, 2024 • edited Loading

louie-tsai commented Oct 22, 2024

malfet left a comment

Choose a reason for hiding this comment

louie-tsai commented Aug 26, 2024 •

edited by pytorch-bot bot

Loading

pytorch-bot bot commented Aug 26, 2024 •

edited

Loading

louie-tsai commented Oct 2, 2024 •

edited

Loading

svekars commented Oct 16, 2024 •

edited

Loading