Use Attach TPC to Framework as part of MCT main track #1308

ofirgo · 2025-01-01T08:33:46Z

Pull Request Description:

This PR changes the TPC mechanism in MCT APIs.
All the main API functions now get a framework-agnostic target platform description and use the Attach2Fw module to generate a framework-dependent quantization capabilities description.

The main changes include:

Alignment of the existing TP models with the new schema (v1).
Remove TP models v2-4 for IMX500 (moved to tests for simplicity of testing).
Remove existing fw TPCs (no need for them anymore).
Addition of attach2fw usage in all API functions.
- To prevent breaking changes for users using the default TPC, the get_target_platform_capabilities API function is kept with a reduced functionality that allows to only obtain the default v1 TPC.
Minor changes to the built-in OperatorSetsNames enum to support more required operators.
Massive tests modification and alignment to comply with the new TPC mechanism.

Checklist before requesting a review:

I set the appropriate labels on the pull request.
I have added/updated the release note draft (if necessary).
I have updated the documentation to reflect my changes (if necessary).
All function and files are well documented.
All function and classes have type hints.
There is a licenses in all file.
The function and variable names are informative.
I have checked for code duplications.
I have added new unittest (if necessary).

Replace schema classes from dataclass to pydantic 'BaseModel'. Fix tests to support pydantic schema classes. Add test for exporting tp model to json

All Pytorch Feature model tests pass except few special cases to solve later.

…al tpc and dedicated tests for the TPC versions

…l error

- enforce matching opsets from tp model to fw - modify attach2fw classes names - remove unused static tpc initialization

ofirgo · 2025-01-01T08:34:47Z

@lior-dikstein @irenaby
Please start going over the main code and API changes.
Note that the work on the tests is still not done, so don't go over the tests yet.
Thanks

model_compression_toolkit/core/common/quantization/quantization_config.py

model_compression_toolkit/target_platform_capabilities/schema/v1.py

...pression_toolkit/target_platform_capabilities/tpc_models/get_target_platform_capabilities.py

model_compression_toolkit/target_platform_capabilities/tpc_models/imx500_tpc/v1/tp_model.py

model_compression_toolkit/target_platform_capabilities/schema/v1.py

...oolkit/target_platform_capabilities/target_platform/targetplatform2framework/attach2keras.py

...lkit/target_platform_capabilities/target_platform/targetplatform2framework/attach2pytorch.py

...arget_platform_capabilities/target_platform/targetplatform2framework/operations_to_layers.py

model_compression_toolkit/core/keras/resource_utilization_data_facade.py

model_compression_toolkit/core/pytorch/resource_utilization_data_facade.py

...n_toolkit/target_platform_capabilities/target_platform/targetplatform2framework/attach2fw.py

model_compression_toolkit/target_platform_capabilities/schema/v1.py

model_compression_toolkit/xquant/keras/keras_report_utils.py

tests/common_tests/helpers/tpcs_for_tests/v1/tp_model.py

tests/keras_tests/feature_networks_tests/feature_networks/mixed_precision_bops_test.py

tests/keras_tests/function_tests/test_hmse_error_method.py

model_compression_toolkit/gptq/pytorch/quantization_facade.py

…emoved OPSET prefix)

model_compression_toolkit/core/common/quantization/quantization_config.py

model_compression_toolkit/target_platform_capabilities/schema/v1.py

...n_toolkit/target_platform_capabilities/target_platform/targetplatform2framework/attach2fw.py

ofirgo and others added 21 commits December 25, 2024 12:33

update all TP models to the new schema and removed pytorch fw tpcs

0af006d

Refactor Target Platform Capabilities - Phase 4

ca7c104

Replace schema classes from dataclass to pydantic 'BaseModel'. Fix tests to support pydantic schema classes. Add test for exporting tp model to json

Add pydantic to requirements file

20fa29f

Modify pytorch API with attach2fw.

33dbe52

All Pytorch Feature model tests pass except few special cases to solve later.

All Pytorch function tests pass except few special cases to solve later.

53fe1d8

Aligned all pytorch APIs, all other pytorch tests pass successfully

80f0d56

merge main into branch

b3d4125

merge tpc phase 4 branch changes

266ebb3

Modify Keras APIs and TPCs and removed fw TPCs

7760dc2

finised tpcs modifications to start running keras tests

349e68c

Move existing tp models to tests

1439e8a

All keras feature tests pass except some related to const and 16 bit

f129054

most of function tests pass except weired issue and 16bit tests

edcaecd

All other keras tests pass except one in exporter that requires speci…

c68f477

…al tpc and dedicated tests for the TPC versions

All torch tests pass except tp model tests

d7a4526

All keras tests pass except tp model tests and some weired maybe loca…

530e40f

…l error

Fix some keras tests

b5285f4

- extend opset fw mapping

93224a1

- enforce matching opsets from tp model to fw - modify attach2fw classes names - remove unused static tpc initialization

verify default tpc version

6e13aa0

fix pytorch custom layer opset

417a3d3

merge main

1c909b9

ofirgo requested review from lior-dikstein and irenaby January 1, 2025 08:33

github-actions bot added auto:core auto:gptq auto:ptq auto:qat auto:target_platform_capabilities auto:tests labels Jan 1, 2025

ofirgo added 3 commits January 1, 2025 13:43

fixes to built-in opset

e6f2ec5

fix tests

905b190

fix import

d0e6fd6

irenaby reviewed Jan 1, 2025

View reviewed changes

ofirgo added 3 commits January 1, 2025 17:00

fix keras input layer import in tests

42be1cb

fix tests

683d95a

fix some tests

8b4503e

lior-dikstein reviewed Jan 2, 2025

View reviewed changes

ofirgo added 5 commits January 2, 2025 13:14

remove tpc from graph object saved in the output pytorch model

290ef92

Merge branch 'remove-tpc-from-torch-model' into use-attach2fw

1d6f511

merge fix to export test

34985e9

one more test fix

20e25a0

fix import in keras tests

cebef33

haihabi reviewed Jan 2, 2025

View reviewed changes

model_compression_toolkit/gptq/pytorch/quantization_facade.py Show resolved Hide resolved

ofirgo added 4 commits January 5, 2025 11:56

fix import in keras tests

620abd1

fix const quantization test tpc

7dcedd2

change enum to str-enum and remove "value" in usage

151522d

fix PR comments and modify opset names enum (change to str enum and r…

37c09bb

…emoved OPSET prefix)

ofirgo added the pr: refactoring/code cleanup label Jan 5, 2025

ofirgo added 3 commits January 5, 2025 17:27

align tests and use CustomOpsetLayers

26ae10e

align tests and use CustomOpsetLayers

c280dd8

align tests and use CustomOpsetLayers

c2016ea

lior-dikstein reviewed Jan 6, 2025

View reviewed changes

model_compression_toolkit/core/common/quantization/quantization_config.py Show resolved Hide resolved

lior-dikstein reviewed Jan 6, 2025

View reviewed changes

model_compression_toolkit/target_platform_capabilities/schema/v1.py Outdated Show resolved Hide resolved

lior-dikstein reviewed Jan 6, 2025

View reviewed changes

...n_toolkit/target_platform_capabilities/target_platform/targetplatform2framework/attach2fw.py Show resolved Hide resolved

lior-dikstein approved these changes Jan 6, 2025

View reviewed changes

ofirgo added 3 commits January 6, 2025 10:57

minor last fixes

8040d69

minor last fixes

7e757b7

minor last fixes

51f8470

ofirgo merged commit d895518 into sony:main Jan 6, 2025
40 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use Attach TPC to Framework as part of MCT main track #1308

Use Attach TPC to Framework as part of MCT main track #1308

ofirgo commented Jan 1, 2025 •

edited

Loading

ofirgo commented Jan 1, 2025

Use Attach TPC to Framework as part of MCT main track #1308

Use Attach TPC to Framework as part of MCT main track #1308

Conversation

ofirgo commented Jan 1, 2025 • edited Loading

Pull Request Description:

Checklist before requesting a review:

ofirgo commented Jan 1, 2025

ofirgo commented Jan 1, 2025 •

edited

Loading