Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

getting a loop with lmstudio local server "Object not generated according to the schema, fallback to manual parsing" #49

Open
ForgetMeAI opened this issue Feb 8, 2025 · 3 comments

Comments

@ForgetMeAI
Copy link

Log:
Step 1 / Budget used 0.00%
Gaps: [ 'what is the capital of France?' ]
Object not generated according to the schema, fallback to manual parsing
undefined <- [search, answer, reflect]
{ answer: 'Paris' }
Waiting 1s...
Step 2 / Budget used 0.08%
Gaps: []
Object not generated according to the schema, fallback to manual parsing
undefined <- [search, answer, reflect]
{ answer: 'Paris' }
Waiting 1s...

model is qwen2.5-7b-instruct, identifier is same as in .env file. i think i need some example json schema for structured output, but idk where i can find it, help pls

@PureRocketsDev
Copy link

I'm getting the same error with LMStudio LM-Studio-0.3.9-6 on Linux and [email protected] when calling npm run dev "1+1=" . In LMStudio I'm using deepseek-r1-distill-qwen-7b. I also tried with llama-3.2-1b-instruct, same error.

@hanxiao
Copy link
Member

hanxiao commented Feb 9, 2025

local llm's structured output is not fully tested. only gemini (1.5flash, 2.0flash) and openai (gpt-4o/-mini) are tested

2 things we still need to figure out: vercel ai-sdk's structured output on local llm, is it good? local llm's own structured output handling, is it good?

@ForgetMeAI
Copy link
Author

local llm's structured output is not fully tested. only gemini (1.5flash, 2.0flash) and openai (gpt-4o/-mini) are tested

2 things we still need to figure out: vercel ai-sdk's structured output on local llm, is it good? local llm's own structured output handling, is it good?

is there any example of json schema that i can test?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants