idea: Add Claude Prompt Caching Support to Jan

[kvn8888]

Problem Statement

Using an API to access Claude without requiring a $20/month Claude Pro subscription saves users money in the long run. However, this benefit diminishes whenever users enter long context multi-turn conversations with Claude, especially with file uploads

Feature Idea

By integrating Anthropic's newer prompt caching feature, the response latency and cost for users could be dramatically reduced. Users may engage with longer multi turn conversations without starting a new thread and losing context. Cached prompts can be an optional feature that costs 90% less compared to uncached prompts per API call

[Boscop]

Yes 100%, @imtuyethan please prioritize this 🙂

Btw, related (partial dup): janhq/cortex.cpp#1570

[imtuyethan]

Awesome @kvn8888 We will try to prioritize it!

[imtuyethan]

@0xSage Please consider closing this as a dup of janhq/cortex.cpp#1570. Then we will follow up on janhq/cortex.cpp#1570.

[dan-homebrew]

I am merging this into #3786, which will be part of our bigger refactor of Remote APIs into Cortex backend