Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] qwen2 vl support the turbomind engine #2774

Open
DexterGuo opened this issue Nov 20, 2024 · 5 comments
Open

[Feature] qwen2 vl support the turbomind engine #2774

DexterGuo opened this issue Nov 20, 2024 · 5 comments
Assignees
@irexyc

Comments

@DexterGuo
Copy link

Motivation

1、The qwen2vl effect is the sota level in the open source model
2、lmdeploy is an excellent inference framework
3、So it's important to support turbomind

Related resources

No response

Additional context

No response
@lvhan028
Copy link
Collaborator

#2720 is working on it.

@ciwang
Copy link

ciwang commented Nov 26, 2024

Hi @lvhan028, do you have an estimate of when that PR will be merged? Thank you in advance!

@lvhan028
Copy link
Collaborator

Not yet. We are trying to refactor the VLM inference in PR #2810
TM engine will not support qwen2-vl until that PR is merged.

@songlinlibit
Copy link

@lvhan028 Hello, I noticed that this PR has been merged. Is it confirmed that versions after #2810 already support the use of the tm engine for Qwen2-VL inference?

@comeby
Copy link

comeby commented Dec 23, 2024

Is there any plan to merge this feature?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@irexyc irexyc

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@comeby @lvhan028 @ciwang @irexyc @DexterGuo @songlinlibit