Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cohere endpoint type doesn't support tools #1404

Open
nsarrazin opened this issue Aug 12, 2024 · 2 comments
Open

Cohere endpoint type doesn't support tools #1404

nsarrazin opened this issue Aug 12, 2024 · 2 comments
Labels
bug Something isn't working models This issue is related to model performance/reliability

Comments

@nsarrazin
Copy link
Collaborator

Bug description

The cohere endpoint type doesn't work well with our current tools setup and errors with cannot specify both message and tool_results in multistep mode

Steps to reproduce

  1. Set a .env.local with a model using cohere endpoint type, and set tools: true on that model.
  2. Start a conversation with tool enabled

Context

Logs

 Error: {"message":"invalid request: cannot specify both message and tool_results in multistep mode"}
at eval (C:/Users/achar/Documents/aditya/chat-ui/src/lib/server/endpoints/cohere/endpointCohere.ts:93:17)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async eval (C:/Users/achar/Documents/aditya/chat-ui/src/lib/server/endpoints/cohere/endpointCohere.ts:70:18)
at async Module.generate (C:/Users/achar/Documents/aditya/chat-ui/src/lib/server/textGeneration/generate.ts:8:20)
at async textGenerationWithoutTitle (C:/Users/achar/Documents/aditya/chat-ui/src/lib/server/textGeneration/index.ts:54:3)
at async Module.mergeAsyncGenerators (C:/Users/achar/Documents/aditya/chat-ui/src/lib/utils/mergeAsyncGenerators.ts:13:34)
at async Module.textGeneration (C:/Users/achar/Documents/aditya/chat-ui/src/lib/server/textGeneration/index.ts:24:3)
at async Object.start (C:/Users/achar/Documents/aditya/chat-ui/src/routes/conversation/[id]/+server.ts:329:26) {
[cause]: BadRequestError: BadRequestError
Status code: 400
Body: {}
at CohereClient. (C:\Users\achar\Documents\aditya\chat-ui\node_modules\cohere-ai\Client.js:112:31)
at Generator.next ()
at fulfilled (C:\Users\achar\Documents\aditya\chat-ui\node_modules\cohere-ai\Client.js:31:58)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
statusCode: 400,
body: ReadableStream { locked: false, state: 'closed', supportsBYOB: false }

Config

MODELS=`[
  {
    "name": "command-r-plus",
    "tools": true,
    "displayName": "Command R+",
    "endpoints": [{
      "type": "cohere",
    }]
  }
]`
@nsarrazin nsarrazin added bug Something isn't working models This issue is related to model performance/reliability labels Aug 12, 2024
@dvirmor
Copy link
Contributor

dvirmor commented Sep 25, 2024

Hi, I'm new. I don't know if custom pull requests are welcome here.
I also encounter this issues and I've found a solution:
add force_single_step: true to cohere.chatStream calling in the endpointCohere.ts file.

This is my pull request: #1490

I hope this is fine

@nsarrazin
Copy link
Collaborator Author

PRs are very welcome! thanks for looking at this @dvirmor i'll review 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working models This issue is related to model performance/reliability
Projects
None yet
Development

No branches or pull requests

2 participants