Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mixtral adapters returning broadcast shape error #301

Open
2 tasks
noyoshi opened this issue Mar 4, 2024 · 4 comments
Open
2 tasks

mixtral adapters returning broadcast shape error #301

noyoshi opened this issue Mar 4, 2024 · 4 comments
Assignees
@ajtejankar @geoffreyangus
Labels
bug Something isn't working

Comments

@noyoshi
Copy link
Collaborator

noyoshi commented Mar 4, 2024 •
edited by ohho
Loading

System Info

lorax version: 4c39e8a

Information

  • When prompting Mixtral with adapter got the following error: Request failed during generation: Server error: output with shape [1, 32000] doesn't match the broadcast shape [227, 32000]

Tasks

  • An officially supported command
  • My own modifications

Reproduction

Load adapter into mixtral and try to prompt

Expected behavior

There should be no error
@tgaddair
Copy link
Contributor

tgaddair commented Mar 4, 2024

Same issue caused by #231.

We can disable LM_HEAD in Mixtral for now until we fix.

@tgaddair tgaddair added the bug Something isn't working label Mar 5, 2024
@abhibst
Copy link

abhibst commented Mar 5, 2024

we do face this issues . hope it will be fixed with current workaround .

@tgaddair
Copy link
Contributor

tgaddair commented Apr 4, 2024

@geoffreyangus will be picking this up second half of April with plan to land no later than end of April.

@tgaddair
Copy link
Contributor

tgaddair commented May 7, 2024

Related #163

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ajtejankar ajtejankar

@geoffreyangus geoffreyangus

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@tgaddair @ajtejankar @noyoshi @geoffreyangus @abhibst