Add compiled autograd tutorial #3026
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/tutorials/3026
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 45acca1 with merge base 7490332 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
xmfan
force-pushed
the
xmfan/ca_tutorial
branch
from
b86d2c5
to
ffa0a81
Compare
xmfan
changed the title
|
|
||
| ###################################################################### | ||
| # Compiled Autograd addresses certain limitations of AOTAutograd | ||
| # ------------ |
There was a problem hiding this comment.
should this be a title or just a paragraph?
There was a problem hiding this comment.
paragraph to clarify that it's a new section after "Compiling the forward and backward pass using different flags"
xmfan
force-pushed
the
xmfan/ca_tutorial
branch
from
4b696bb
to
a9d3bb3
Compare
xmfan
force-pushed
the
xmfan/ca_tutorial
branch
from
a9d3bb3
to
271b8f2
Compare
|
There is still some issue on line 54 that prevents the tutorial from building. Can you check? |
| * Added runtime overhead at the start of the backward for cache lookup | ||
| * More prone to recompiles and graph breaks in dynamo due to the larger capture | ||
|
|
||
| .. note:: Compiled Autograd is under active development and is not yet compatible with all existing PyTorch features. For the latest status on a particular feature, refer to `Compiled Autograd Landing Page <https://docs.google.com/document/d/11VucFBEewzqgkABIjebZIzMvrXr3BtcY1aGKpX61pJY>`_. |
There was a problem hiding this comment.
I'm not sure why it's called Landing Page. Should it be called a Compiled Autograd Roadmap? Also, it feels like it would be better to point to issues on github with a specific label, perhaps, or a Github Project even?
There was a problem hiding this comment.
I'm expecting the content of this doc to change rapidly at the moment, so I'll keep it as a google doc for now. It's also something we did for dynamic shapes and custom operators.
The end goal once the development is more stable, is to move it into the docs
svekars
added
2.5
Fixes #3034
Description
Add tutorial for a PyTorch 2.4 feature
Checklist