Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tuner] Add direct TD spec generation for candidates #606

Merged
merged 1 commit into from
Dec 16, 2024

Conversation

Max191
Copy link
Contributor

@Max191 Max191 commented Nov 25, 2024

This PR adds direct transform dialect spec generation for candidate configurations. This is the first part of the large refactoring described in #577. The way TD specs are generated is by matching against certain types of operations, and then creating a named sequence with transform.iree.match.cast_compatible_dag_from_root based on the matched operation. This is done for each configuration found, and the specs are saved to the temporary tuning directory to be used later in tuning.

One main difference in the flow of candidate generation is that state is no longer tracked by saving files to a temporary directory. Instead, ir modules are passed to each function, and only at the very end of candidate generation are the transform dialect specs written to files. This makes things cleaner, since there no longer needs to be a coordination of file paths.

@Max191
Copy link
Contributor Author

Max191 commented Nov 25, 2024

This PR is broken right now because of an issue tracked here iree-org/iree#19269. Reviews are appreciated, but it won't be able to land until after this issue is resolved.

tuner/tuner/libtuner.py Outdated Show resolved Hide resolved
tuner/tuner/op_matchers.py Outdated Show resolved Hide resolved
@Max191 Max191 requested a review from kuhar November 25, 2024 20:56
tuner/pyproject.toml Outdated Show resolved Hide resolved
tuner/tuner/candidate_gen.py Outdated Show resolved Hide resolved
tuner/tuner/candidate_gen.py Outdated Show resolved Hide resolved
tuner/tuner/candidate_gen.py Outdated Show resolved Hide resolved
tuner/tuner/candidate_gen.py Outdated Show resolved Hide resolved
tuner/tuner/candidate_gen.py Outdated Show resolved Hide resolved
tuner/tuner/candidate_gen.py Outdated Show resolved Hide resolved
tuner/tuner/candidate_gen.py Outdated Show resolved Hide resolved
tuner/tuner/spec_builder.py Outdated Show resolved Hide resolved
tuner/tuner/spec_builder.py Outdated Show resolved Hide resolved
tuner/tuner/libtuner.py Outdated Show resolved Hide resolved
@Max191 Max191 marked this pull request as ready for review December 12, 2024 19:07
@Max191 Max191 requested a review from kuhar December 12, 2024 19:07
tuner/examples/test/conv_benchmark.mlir Outdated Show resolved Hide resolved
tuner/examples/test/mmt_benchmark.mlir Outdated Show resolved Hide resolved
tuner/tuner/candidate_gen.py Outdated Show resolved Hide resolved
tuner/tuner/candidate_gen.py Outdated Show resolved Hide resolved
tuner/tuner/candidate_gen.py Show resolved Hide resolved
Comment on lines +658 to +664
# Index 0 is reserved for default config, so it gets no td spec.
with ir.Location.unknown() as loc:
empty_module = ir.Module.create(loc)
config_specs: list[ir.Module] = [empty_module]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid this special case, we could also look up the lowering_config and translation_info from the original module and re-materialize it here, but I don't think it's worth it...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or we could generate a placeholder spec that never matches anything.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kind of like the empty module for now. It seems relatively clean to me, and it makes it clear to someone looking through the specs that it is a placeholder. Having an explicit placeholder spec would also be good.

tuner/tuner/candidate_gen_test.py Outdated Show resolved Hide resolved
tuner/tuner/candidate_gen_test.py Show resolved Hide resolved
tuner/tuner/dispatch_parser.py Outdated Show resolved Hide resolved
tuner/tuner/libtuner.py Outdated Show resolved Hide resolved
@Max191 Max191 force-pushed the tuner-candidate-spec-gen branch 2 times, most recently from 872be87 to 119833b Compare December 12, 2024 20:54
@Max191 Max191 requested a review from kuhar December 12, 2024 20:55
tuner/tuner/libtuner.py Outdated Show resolved Hide resolved
@Max191 Max191 requested a review from kuhar December 13, 2024 14:32
@Max191 Max191 force-pushed the tuner-candidate-spec-gen branch 2 times, most recently from 083018d to 41bb86d Compare December 13, 2024 17:06
Copy link
Member

@kuhar kuhar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Make sure you drop the benchmark mlir files before landing.

tuner/tuner/dispatch_parser.py Outdated Show resolved Hide resolved
tuner/tuner/dispatch_parser.py Outdated Show resolved Hide resolved
@Max191 Max191 force-pushed the tuner-candidate-spec-gen branch 2 times, most recently from f37d82f to e435510 Compare December 16, 2024 16:49
@Max191 Max191 force-pushed the tuner-candidate-spec-gen branch from 259bd92 to 37a349c Compare December 16, 2024 18:12
@Max191 Max191 merged commit 4f542ac into nod-ai:main Dec 16, 2024
14 checks passed
IanNod pushed a commit to IanNod/SHARK-Platform that referenced this pull request Dec 17, 2024
This PR adds direct transform dialect spec generation for candidate
configurations. This is the first part of the large refactoring
described in nod-ai#577. The way TD
specs are generated is by matching against certain types of operations,
and then creating a named sequence with
`transform.iree.match.cast_compatible_dag_from_root` based on the
matched operation. This is done for each configuration found, and the
specs are saved to the temporary tuning directory to be used later in
tuning.

One main difference in the flow of candidate generation is that state is
no longer tracked by saving files to a temporary directory. Instead, ir
modules are passed to each function, and only at the very end of
candidate generation are the transform dialect specs written to files.
This makes things cleaner, since there no longer needs to be a
coordination of file paths.

Signed-off-by: Max Dawkins <[email protected]>
Max191 added a commit that referenced this pull request Jan 7, 2025
This PR removes the old code paths for the tuner following the large
refactoring done in #606 and
#704.
- The old compilation and benchmarking logic for models and dispatches
are now gone, and the unified `benchmark` and `compile` functions should
be used instead.
- Much of the dispatch parsing logic is removed, and dispatch parsing is
now done by 2 DispatchParser implementations for contraction ops and
convolution ops.
- The example tuning clients in `tuner/examples` for dispatches and
punet are removed, since they use the old path. The new example to
follow is in `tuner/examples/test`.
- The candidate generation `tune` function is removed, and the
`generate_configs_and_td_specs` should be used to generate candidates
instead.
- Many utility functions, structs, and struct fields are now removed,
since they are no longer used.
- All tests testing the old path are removed.

---------

Signed-off-by: Max Dawkins <[email protected]>
monorimet pushed a commit that referenced this pull request Jan 8, 2025
This PR adds direct transform dialect spec generation for candidate
configurations. This is the first part of the large refactoring
described in #577. The way TD
specs are generated is by matching against certain types of operations,
and then creating a named sequence with
`transform.iree.match.cast_compatible_dag_from_root` based on the
matched operation. This is done for each configuration found, and the
specs are saved to the temporary tuning directory to be used later in
tuning.

One main difference in the flow of candidate generation is that state is
no longer tracked by saving files to a temporary directory. Instead, ir
modules are passed to each function, and only at the very end of
candidate generation are the transform dialect specs written to files.
This makes things cleaner, since there no longer needs to be a
coordination of file paths.

Signed-off-by: Max Dawkins <[email protected]>
monorimet pushed a commit that referenced this pull request Jan 8, 2025
This PR removes the old code paths for the tuner following the large
refactoring done in #606 and
#704.
- The old compilation and benchmarking logic for models and dispatches
are now gone, and the unified `benchmark` and `compile` functions should
be used instead.
- Much of the dispatch parsing logic is removed, and dispatch parsing is
now done by 2 DispatchParser implementations for contraction ops and
convolution ops.
- The example tuning clients in `tuner/examples` for dispatches and
punet are removed, since they use the old path. The new example to
follow is in `tuner/examples/test`.
- The candidate generation `tune` function is removed, and the
`generate_configs_and_td_specs` should be used to generate candidates
instead.
- Many utility functions, structs, and struct fields are now removed,
since they are no longer used.
- All tests testing the old path are removed.

---------

Signed-off-by: Max Dawkins <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants