Skip to content

improve: increase default context length and add tests for detecting context length #6214

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

uinstinct
Copy link
Contributor

Description

  • update the default context length from 8192 to 32,768
  • add tests for all providers in llm-info's for testing their context length whenever present

resolves CON-2295

Checklist

  • [] I've read the contributing guide
  • [] The relevant docs, if any, have been updated or created
  • [] The relevant tests, if any, have been updated or created

Screenshots

[ For visual changes, include screenshots. Screen recordings are particularly helpful, and appreciated! ]

Tests

[ What tests were added or updated to ensure the changes work as expected? ]

@uinstinct uinstinct requested a review from a team as a code owner June 20, 2025 11:30
@uinstinct uinstinct requested review from Patrick-Erichsen and removed request for a team June 20, 2025 11:30
Copy link

netlify bot commented Jun 20, 2025

👷 Deploy request for continuedev pending review.

Visit the deploys page to approve it

Name Link
🔨 Latest commit 8a79d55

@dosubot dosubot bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Jun 20, 2025
@@ -8,7 +8,6 @@ class Anthropic extends BaseLLM {
static providerName = "anthropic";
static defaultOptions: Partial<LLMOptions> = {
model: "claude-3-5-sonnet-latest",
contextLength: 200_000,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed this from the class because it was already present in anthropic's llm info

Copy link

recurseml bot commented Jun 20, 2025

✨ No issues found! Your code is sparkling clean! ✨

@@ -48,7 +48,6 @@ class Bedrock extends BaseLLM {
static defaultOptions: Partial<LLMOptions> = {
region: "us-east-1",
model: "anthropic.claude-3-sonnet-20240229-v1:0",
contextLength: 200_000,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here, it was already present in bedrock's llm info

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size:M This PR changes 30-99 lines, ignoring generated files.
Projects
Status: Todo
Development

Successfully merging this pull request may close these issues.

1 participant