Skip to content

fix(config-yaml): prompt caching and reasoning #5202

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

chezsmithy
Copy link
Contributor

@chezsmithy chezsmithy commented Apr 17, 2025

Description

Could also resolve: #4918

  • Enabled prompt caching and reasoning support again after the split out to use the config-yaml package to load the yaml configuration.

Checklist

  • The relevant docs, if any, have been updated or created
  • The relevant tests, if any, have been updated or created

Screenshots

[ For visual changes, include screenshots. Screen recordings are particularly helpful, and appreciated! ]

Testing instructions

Validated by enabling prompt caching and thinking with the AWS bedrock claude sonet 3.7 model. With this change I am now seeing prompt caching and thinking enabled.

@chezsmithy chezsmithy requested a review from a team as a code owner April 17, 2025 04:46
@chezsmithy chezsmithy requested review from RomneyDa and removed request for a team April 17, 2025 04:46
Copy link

netlify bot commented Apr 17, 2025

Deploy Preview for continuedev ready!

Name Link
🔨 Latest commit 2f85661
🔍 Latest deploy log https://app.netlify.com/sites/continuedev/deploys/6803339d22d49600084e2517
😎 Deploy Preview https://deploy-preview-5202--continuedev.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@chezsmithy chezsmithy force-pushed the fix-yaml-reasoning-caching branch 2 times, most recently from 3f65c48 to 5bbd42e Compare April 18, 2025 23:57
@chezsmithy
Copy link
Contributor Author

@RomneyDa appears that with the latest preview and production builds this problem exists in the yaml config loader so it's no longer possible to activate thinking and caching unless the user reverts back to json. Hope you can check and if possible get this merged soon?

@chezsmithy chezsmithy force-pushed the fix-yaml-reasoning-caching branch from 5bbd42e to 2f85661 Compare April 19, 2025 05:24
@chezsmithy chezsmithy changed the title fix: Enable prompt caching and reasoning with YAML and config-yaml split out fix(config-yaml): Enable prompt caching and reasoning Apr 19, 2025
@chezsmithy chezsmithy changed the title fix(config-yaml): Enable prompt caching and reasoning fix(config-yaml): prompt caching and reasoning Apr 19, 2025
@chezsmithy
Copy link
Contributor Author

@RomneyDa and @Patrick-Erichsen curious if we could get a review on this PR and the one also referenced. For those of us who have moved to YAML (I just moved my whole organization) the ability to use prompt caching and reasoning is broken. Thanks!

@Patrick-Erichsen
Copy link
Collaborator

Apologies for the delay here @chezsmithy , thanks for submitting the fix!

@Patrick-Erichsen Patrick-Erichsen merged commit cfef134 into continuedev:main Apr 28, 2025
31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

support reasoning and reasoningBudgetTokens completion option in YAML
2 participants