Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add a model config option to disable system prompts #1432

Open
nsarrazin opened this issue Aug 25, 2024 · 0 comments
Open

Add a model config option to disable system prompts #1432

nsarrazin opened this issue Aug 25, 2024 · 0 comments
Labels
back This issue is related to the Svelte backend or the DB enhancement New feature or request models This issue is related to model performance/reliability

Comments

@nsarrazin
Copy link
Collaborator

Describe your feature request

Some models don't support system prompts and if you try to use the system role in their chat template they throw an error. We should add a config option in the model definition to specify that system prompts are not supported. In that case we

Implementation idea

add a systemPrompt: true|false in the modelConfig which will trigger logic to filter out role: 'system' messages and preprend the content of the first one to the first user message

Solves #1386
@nsarrazin nsarrazin added enhancement New feature or request back This issue is related to the Svelte backend or the DB models This issue is related to model performance/reliability labels Aug 25, 2024
@nsarrazin nsarrazin changed the title Add a config option to disable system prompts Add a model config option to disable system prompts Aug 25, 2024
@nsarrazin nsarrazin mentioned this issue Aug 25, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
back This issue is related to the Svelte backend or the DB enhancement New feature or request models This issue is related to model performance/reliability
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@nsarrazin