[WIP] Llama3 weight analysis and visualization using safetensors #33
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for he contribution @pythonLoader, I'm going to see if there's a bit more we can do to generalize this further over the next few weeks once I can find sometime. I'm worried about landing specific scripts under utils or the code base. If you'd like to get this in, then we can package it as an example but we'll need some more things around it such as a readme fo rusage. |
|
Hi @markurtz, sure! I can add an argparser and a README for this. Is there anything else that I need to do for making it more generalized? |
|
That should be fine for now provided we can add I as a dedicated directory under /examples! |
markmc
pushed a commit
to markmc/llm-compressor
that referenced
this pull request
Nov 13, 2024
* Compressed lifecycle implementation (INT8 only) * Apply suggestions from code review * small fixes for runtime * Quantization Compressor Support (vllm-project#45) * add classes * WIP * moving around classes * code complete * tests passing * unit test bugs * fill out int decompression * docstrings * allow repeat frozens * int compressor unit tests * PR comments * fix device issue * fixing leaf checker * initial commit * Revert "Merge branch 'main' into compressed-lifecycle" This reverts commit 8dcdde51bb153036b551a5a0252208f1ff9275e1, reversing changes made to bb369367bd74f7b18ddd3bfdc32db2902b604f92. * update version * fix test --------- Co-authored-by: Sara Adkins <[email protected]> Co-authored-by: dbogunowicz <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
SUMMARY:
Weight analysis and visualization for llama3 models loaded from safetensors.
TEST PLAN:
No automatic tests written yet. It's a work in progress. Will add test after review.