-
-
Notifications
You must be signed in to change notification settings - Fork 5.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Bamba Model #10909
Open
fabianlim
wants to merge
65
commits into
vllm-project:main
Choose a base branch
from
fabianlim:bamba-pr
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+3,563
−95
Open
Add Bamba Model #10909
Changes from 60 commits
Commits
Show all changes
65 commits
Select commit
Hold shift + click to select a range
62181d5
initial pr without tp fix
fabianlim 51bc78c
fix casting in rms norm gated
fabianlim 81b93b4
TP fix
fabianlim 0f93e4a
fix mamba scan invalid address
fabianlim 742ae79
some fixes and remove unused kernels
fabianlim b2dc5ca
fmt + lint
fabianlim 9ad9e20
more comments
fabianlim 25bf381
initial fix for chunked prefill (incomplete)
fabianlim 43ce07c
improve comments
fabianlim 80f14b5
do not attach seq_idx to attn_metadata
fabianlim 6b8ac49
activate initial states for chunked prefill
fabianlim d788db6
reuse softplus and remove triton2 remark
fabianlim 400db27
add comment on weight loader and format
fabianlim bda8ea7
rename test_jamba to test_hybrid and got rid of test_bamba
fabianlim 66078d6
Merge remote-tracking branch 'upstream/main' into bamba-pr
fabianlim a74de9f
update bamba to ishybrid and support pp
fabianlim b44caa7
lint
fabianlim 8cf3644
add unit test for mamba ssd
fabianlim e375b40
fix lint
fabianlim dcbae7b
full chunked-prefill fix (sans unit tests)
fabianlim 2597105
format and add cont batch unit tests (will need more cases)
fabianlim db5eea5
fix kernel tests and add more chunked prefill cases
fabianlim dfbcb16
bound adjustment
fabianlim 7913009
bound adjustment
fabianlim 9c5d045
lint errors
fabianlim 6bc9dac
Add permalink correction from @tlrmchlsmth
fabianlim 6d02e85
improved comment for segsum, add more sizes for test_mamba_chunk_scan…
fabianlim e5882f2
rename and comment functions, add more sizes for test_mamba_chunk_sca…
fabianlim 6d6fa86
addressed comments on mamba_mixer2.py
fabianlim 773dd80
replace with get_rope
fabianlim 63f5340
rope scaling
fabianlim 89e36d8
fixes
fabianlim 7a4ae96
zero out ssm states
fabianlim a9e149c
fix tests (sans updating dev checkpoint)
fabianlim 5c9f48d
not replacing dev model for now
fabianlim 55647b1
update requirements
fabianlim 2342bc0
remove extraneous comment
fabianlim 011c141
update test
fabianlim 503bc42
fix lint
fabianlim 312cf1d
fix lint
fabianlim c1db743
fix requirements-test
fabianlim c956a30
Mamba2 changes from #10909
tlrmchlsmth 17923ad
Get Mamba2 working!
tlrmchlsmth 4183d45
Add integration test -- something is wrong!!
tlrmchlsmth 5377644
format
tlrmchlsmth 39f55d1
fixes
tlrmchlsmth dd31f19
update test registry, fixes
fabianlim e2e5aac
Fix for conv state shape and update placeholder_attn
tlrmchlsmth bc1b8af
back out placeholder_attn changes
tlrmchlsmth 9db0dd5
make seq_idx to chunk indices more efficient
fabianlim cd89283
WIP debugging, restore local mamba and placeholder_attn changes
tlrmchlsmth 9a838a3
Integration tests are now green
tlrmchlsmth be8318e
remove bamba-specific files
tlrmchlsmth f34d434
Merge branch 'main' into tms/mamba2
tlrmchlsmth a65e2cb
Handle grouping in Mixer2RMSNormGated
tlrmchlsmth 0d4bb0f
debug cruft
tlrmchlsmth 74f6088
Remove codestral integration test
tlrmchlsmth 95583b8
Merge branch 'tms/mamba2' into bamba-pr
fabianlim b72389c
update mamba_cache
fabianlim 10d75eb
remove changes to requirements
fabianlim 5aea1e6
revert changes
fabianlim 2ee8d07
Merge remote-tracking branch 'upstream/main' into bamba-pr
fabianlim 043e006
fix lint
fabianlim 7e4ce4f
fix lint
fabianlim 8219480
more reverts
fabianlim File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. same as above, the version on the left is old. the right is from latest main |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you merge in latest main? We've already landed this change
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ya actually yest i reverted this file and took it from latest main, but somehow the diff shows up in github. The version on the left shown by github is actually old
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok if i merge in latest main it seems fine..