Print delegation info in export_llama in verbose #7803
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/7803
Note: Links to docs will display an error until the docs builds have been completed. This comment was automatically generated by Dr. CI and updates every 15 minutes. |
facebook-github-bot
added
the
CLA Signed
tarun292
reviewed
Jan 22, 2025
| @@ -816,6 +818,13 @@ def _export_llama(args) -> LLMEdgeManager: # noqa: C901 | |||
|
|
|||
| builder = builder.to_executorch(passes=additional_passes) | |||
|
|
|||
| if args.verbose: | |||
| graph_module = builder.edge_manager.exported_program().graph_module | |||
There was a problem hiding this comment.
The EdgeManager is modified inplace, so after to_executorch it inserts memory.alloc nodes for out variant ops. These memory.alloc nodes are not generated in the final PTE file but only used for memory allocation for out tensor of out-variant ops, so the delegation count given here will be inaccurate. I think we should print this right after to_backend above.
There was a problem hiding this comment.
That makes sense. Let me update it. Thanks @tarun292 !
iseeyuan
force-pushed
the
delegate_info
branch
from
020b0f6 to
ce4c066
Compare
tarun292
approved these changes
Jan 22, 2025
|
@pytorchbot label "topic: not user facing" |
iseeyuan
force-pushed
the
delegate_info
branch
from
ce4c066 to
48b6c76
Compare
iseeyuan
force-pushed
the
delegate_info
branch
from
48b6c76 to
0e27a3b
Compare
|
@pytorchbot merge |
|
Mergebot is not configured for this repository. Please use the merge button provided by GitHub. |
SS-JIA
added a commit
that referenced
this pull request
Jan 27, 2025
## Context #7803 added an import to export_llama but did not add it to the buck target. Differential Revision: [D68716129](https://our.internmc.facebook.com/intern/diff/D68716129/) [ghstack-poisoned]
SS-JIA
added a commit
that referenced
this pull request
Jan 27, 2025
## Context #7803 added an import to export_llama but did not add it to the buck target. Differential Revision: [D68716129](https://our.internmc.facebook.com/intern/diff/D68716129/) ghstack-source-id: 263234883 Pull Request resolved: #7963
kirklandsign
pushed a commit
that referenced
this pull request
Jan 27, 2025
## Context #7803 added an import to export_llama but did not add it to the buck target. Differential Revision: [D68716129](https://our.internmc.facebook.com/intern/diff/D68716129/) ghstack-source-id: 263234883 Pull Request resolved: #7963 Co-authored-by: Stephen Jia <[email protected]>
YIWENX14
pushed a commit
that referenced
this pull request
Jan 28, 2025
Co-authored-by: Martin Yuan <[email protected]>
YIWENX14
pushed a commit
that referenced
this pull request
Jan 28, 2025
## Context #7803 added an import to export_llama but did not add it to the buck target. Differential Revision: [D68716129](https://our.internmc.facebook.com/intern/diff/D68716129/) ghstack-source-id: 263234883 Pull Request resolved: #7963 Co-authored-by: Stephen Jia <[email protected]>
zonglinpeng
pushed a commit
to zonglinpeng/executorch
that referenced
this pull request
Jan 30, 2025
Co-authored-by: Martin Yuan <[email protected]>
zonglinpeng
pushed a commit
to zonglinpeng/executorch
that referenced
this pull request
Jan 30, 2025
## Context pytorch#7803 added an import to export_llama but did not add it to the buck target. Differential Revision: [D68716129](https://our.internmc.facebook.com/intern/diff/D68716129/) ghstack-source-id: 263234883 Pull Request resolved: pytorch#7963 Co-authored-by: Stephen Jia <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
For better dev and debugging experience, add delegation info prints in verbose in export_llama. It only shows up with
-voption.Test plan