[Oneshot Refactor] Main refactor #1110

horheynm · 2025-01-28T19:35:53Z

ORDER OF REVIEWS:

SUMMARY:

Create a class to decouple dependency to main. Class Oneshot consists of pre-processing, carrying out oneshot logic and post-processing
Move the oneshot class and method under llmcompressor/entrypoints/oneshot.py.
Add ReadMe in /llmcompressor/entrypoints for info on oneshot
Delete oneshot logic from /finetune directory, add deprecation warning
Remove apply used only for stagerunner oneshot pathway in session.py and session_function.py
Add oneshot only calibration dataloader logic
Add a return variable of model: PretrainedModel for def oneshot.
Make oneshot carryout logic independent of TrainingArguments
remove overwrite_output_dir as oneshot input arg -> only used for TrainingArguments
Update README on /finetune path. Remove oneshot logic and oneshot with fsdp
Update wrap_save_pretrained logic to run only if not updated already -> used for stage runner to avoid double wrapping

Entrypoints:

from llmcompressor import oneshot
oneshot(**kwargs) # calls Oneshot

or

from llmcompressor import Oneshot
oneshot = Oneshot(**kwargs)
oneshot() # preprocesss, carries out logic and post process

TEST PLAN:
Pass all tests and examples.
Verified https://github.com/vllm-project/llm-compressor/blob/main/examples/quantization_2of4_sparse_w4a16/llama7b_sparse_w4a16.py works as expected.

FOLLOW UPS:

Stage runner removal
Update entrypoints folder with train, eval, predict, etc.

github-actions · 2025-01-28T19:36:06Z

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed.

src/llmcompressor/transformers/calibration/oneshot.py

src/llmcompressor/transformers/finetune/text_generation.py

dsikka · 2025-02-03T18:56:06Z

Blocked on #1103

SUMMARY:

Create a class to decouple dependency to main. Class Oneshot consists of pre-processing, carrying out oneshot logic and post-processing

Entrypoints:
oneshot(**kwargs) # calls Oneshot
or
oneshot = Oneshot(**kwargs)
oneshot.run() # preprocesss, carries out logic and post process
TEST PLAN: Contingent on #1103, should pass all tests

Can you update the PR description so it clearly lays out what PRs we should be reviewing and in what order, so that it is easier to understand the changes?

dsikka

Have we verified example testing cases? E.g
https://github.com/vllm-project/llm-compressor/blob/main/examples/quantization_2of4_sparse_w4a16/llama7b_sparse_w4a16.py

src/llmcompressor/transformers/calibration/oneshot.py

src/llmcompressor/transformers/sparsification/compressed_tensors_utils.py

src/llmcompressor/transformers/finetune/session_mixin.py

src/llmcompressor/transformers/finetune/runner.py

…ame_from_model (#1108) ORDER OF REVIEWS: 1. #1108 <- current PR 2. #1103 3. #1109 4. #1110 SUMMARY: * Rename `get_shared_processor_src` to `get_processor_from_model` * Appropriate signature on `initialize_processor_from_path`, where `teacher` should be optinal TEST PLAN: * Pass all existing tests * Search `get_shared_processor_src` using pygrep ```bash 3 function pygrep() { 2 local search_term="$1" 1 shift 126 local search_dirs="${*:-src examples tests}" 1 grep -rn --include="*.py" -E "$search_term" $search_dirs 2 } ``` --------- Co-authored-by: Dipika Sikka <[email protected]>

src/llmcompressor/transformers/finetune/data/data_helpers.py

ORDER OF REVIEWS: 1. #1108 2. #1103 <- current PR 3. #1109 4. #1110 SUMMARY: Refactor dataclass used for llm-compressor entrypoints (oneshot, train, apply) to decouple non-relevant attributes from the existing dataclass. Ex. recipe in training_args. Recipe is contained in a session, not in the trainer that training_args govern. Dataclass refactor details are in https://docs.google.com/document/d/1YbR1dTQmCzqhGk74m5msBzqoPHQgB6dVxDtf6cTmetc/edit?usp=sharing Note: #1110 takes care of using a new entrypoint that will prohibit the post_train / oneshot call to use training_args. Current entrypoint will need training_args for oneshot to function - this PR is just for refactoring the dataclass. Before: ModelArguments: https://github.com/vllm-project/llm-compressor/blob/6fa5a5eecc7d363ec73474d011d40135b6374179/src/llmcompressor/transformers/finetune/model_args.py#L6 DataTrainingArguments: https://github.com/vllm-project/llm-compressor/blob/6fa5a5eecc7d363ec73474d011d40135b6374179/src/llmcompressor/transformers/finetune/data/data_args.py#L70 TrainingArguments: https://github.com/vllm-project/llm-compressor/blob/6fa5a5eecc7d363ec73474d011d40135b6374179/src/llmcompressor/transformers/finetune/training_args.py#L10 After: ModelArguments: https://github.com/vllm-project/llm-compressor/pull/1103/files#diff-58fd0f7ae4564317960ae0d4d4b2cdb97b9588c1915f062915e74ecf51b5502cR6 DatasetArguments: https://github.com/vllm-project/llm-compressor/pull/1103/files#diff-5e43f74ba5d8327b937adada3c7f30a7efb13f9a44cb3fdb5e1a2a12b8b8ea27R70 RecipeArguments: https://github.com/vllm-project/llm-compressor/pull/1103/files#diff-0ff9c048a4deb55e5459054bdc61a5d8c81da9c94588ec2355e6b2c2ec8675d1R6 TrainingArguments: https://github.com/vllm-project/llm-compressor/pull/1103/files#diff-249ee96763dd50956a7309f898eda4bcaa91c6af653474568fbda10b5a39c817R12 TEST PLAN: * Pass all existing tests * Search dataclass arguments using `pygrep` ```bash 3 function pygrep() { 2 local search_term="$1" 1 shift 126 local search_dirs="${*:-src examples tests}" 1 grep -rn --include="*.py" -E "$search_term" $search_dirs 2 } ``` --------- Co-authored-by: Rahul Tuli <[email protected]>

horheynm · 2025-02-11T14:05:57Z

Closing and making changes to
#1136

ORDER OF REVIEWS: 1. #1108 2. #1103 3. #1109 <- current PR 4. #1110 SUMMARY: Refactor `initialize_model_from_path` to decouple `training_args` dependent logic and oneshot (non-training_args) logic. TEST PLAN: * Pass all tests * search `initialize_model_from_path` using `grep`

…ressor into oneshot-refac-main

horheynm · 2025-02-11T23:02:51Z

Have we verified example testing cases? E.g https://github.com/vllm-project/llm-compressor/blob/main/examples/quantization_2of4_sparse_w4a16/llama7b_sparse_w4a16.py

Yes!

Signed-off-by: George <[email protected]>

…ressor into oneshot-refac-main

brian-dellabetta

approved with a comment

README.md

examples/quantization_2of4_sparse_w4a16/llama7b_sparse_w4a16.py

kylesayrs

Good job

src/llmcompressor/entrypoints/__init__.py

src/llmcompressor/entrypoints/oneshot.py

src/llmcompressor/transformers/finetune/runner.py

Signed-off-by: George Ohashi <[email protected]>

dsikka

LGTM pending lm_eval testing.
Can confirm performance testing and staged recipes look good.

oneshot refac

26bcbb0

dsikka requested changes Feb 3, 2025

View reviewed changes

src/llmcompressor/transformers/calibration/oneshot.py Outdated Show resolved Hide resolved

src/llmcompressor/transformers/finetune/text_generation.py Outdated Show resolved Hide resolved

src/llmcompressor/transformers/finetune/text_generation.py Outdated Show resolved Hide resolved

This was referenced Feb 3, 2025

[Oneshot Refactor] Rename get_shared_processor_src to get_processor_name_from_model #1108

Merged

[Oneshot Refactor] dataclass Arguments #1103

Merged

[Oneshot refactor] Refactor initialize_model_from_path #1109

Merged

dsikka requested changes Feb 4, 2025

View reviewed changes

horheynm added 2 commits February 4, 2025 15:10

comments

8c0a255

Merge branch 'main' into oneshot-refac-main

468a714

dsikka requested changes Feb 8, 2025

View reviewed changes

src/llmcompressor/transformers/finetune/data/data_helpers.py Show resolved Hide resolved

horheynm closed this Feb 11, 2025

dsikka reopened this Feb 11, 2025

horheynm added 10 commits February 11, 2025 14:32

merge main

428a2c7

stashed changes

58b3d6a

run examples pass

380d164

pass tests

0ae9ade

Merge branch 'main' into oneshot-refac-main

9be9174

add entrypoints

95bfaa1

Merge branch 'oneshot-refac-main' of github.com:vllm-project/llm-comp…

c4dd9cc

…ressor into oneshot-refac-main

udpate read me on /finetune

d2ffc4a

pass tests

fc4c42f

pass tests

09942b7

add readme and remove breakpoint

00dd629

horheynm added the ready When a PR is ready for review label Feb 12, 2025

horheynm added 4 commits February 14, 2025 09:46

fix self attr population

58a7a5a

fix test, get torch model not stub

ca7da03

use non-gated model

103fd71

Merge branch 'main' into oneshot-refac-main

7cf5f1a

horheynm mentioned this pull request Feb 14, 2025

[StageRunner] Add stage for stage recipe #1154

Closed

horheynm added 8 commits February 17, 2025 17:42

lint

333dc42

Merge branch 'main' into oneshot-refac-main

e7e838f

update stages

5a1dccf

Signed-off-by: George <[email protected]>

Merge branch 'main' into oneshot-refac-main

fb2af8d

revert output_dir name

ca9f295

Signed-off-by: George <[email protected]>

Merge branch 'oneshot-refac-main' of github.com:vllm-project/llm-comp…

a9e8597

…ressor into oneshot-refac-main

Merge branch 'main' into oneshot-refac-main

d2e6274

Merge branch 'main' into oneshot-refac-main

5f1d383

brian-dellabetta previously approved these changes Feb 20, 2025

View reviewed changes

README.md Outdated Show resolved Hide resolved

kylesayrs reviewed Feb 20, 2025

View reviewed changes

examples/quantization_2of4_sparse_w4a16/llama7b_sparse_w4a16.py Outdated Show resolved Hide resolved

kylesayrs previously approved these changes Feb 20, 2025

View reviewed changes

comments

eb3094c

horheynm dismissed stale reviews from kylesayrs and brian-dellabetta via eb3094c February 20, 2025 20:51

dsikka reviewed Feb 20, 2025

View reviewed changes

src/llmcompressor/transformers/finetune/runner.py Outdated Show resolved Hide resolved

remove if condition

b3a09a4

Signed-off-by: George Ohashi <[email protected]>

dsikka previously approved these changes Feb 24, 2025

View reviewed changes

dsikka added ready When a PR is ready for review and removed ready When a PR is ready for review labels Feb 24, 2025

Merge branch 'main' into oneshot-refac-main

fe6b797

horheynm dismissed dsikka’s stale review via fe6b797 February 24, 2025 15:21

kylesayrs approved these changes Feb 24, 2025

View reviewed changes

dsikka approved these changes Feb 24, 2025

View reviewed changes

dsikka merged commit 865364f into main Feb 24, 2025
7 checks passed

dsikka deleted the oneshot-refac-main branch February 24, 2025 20:04

[Oneshot Refactor] Main refactor #1110

[Oneshot Refactor] Main refactor #1110

Uh oh!

Conversation

horheynm commented Jan 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jan 28, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dsikka commented Feb 3, 2025

Uh oh!

dsikka left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

horheynm commented Feb 11, 2025

Uh oh!

horheynm commented Feb 11, 2025

Uh oh!

brian-dellabetta left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

kylesayrs left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dsikka left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

horheynm commented Jan 28, 2025 •

edited

Loading