CodeGate garbles system configuration for Anthropic requests (such as cache control)

[kantord]

Describe the issue

When using Cline with CodeGate, CodeGate seems to break/garble some of the information that Cline sends in the system configuration to Anthropic.

A request made directly with Cline looks like this:

{
    "model": "claude-3-5-sonnet-20241022",
    "max_tokens": 8192,
    "temperature": 0,
    "system": [
        {
            "text": "...etc...",
            "type": "text",
            "cache_control": {
                "type": "ephemeral"
            }
        }
    ],

When proxying through CodeGate the request looks like this:

    "temperature": 0,
    "max_tokens": 8192,
    "system": [
        {
            "type": "text",
            "text": "[{'text': '...etc...', 'type': 'text', 'cache_control': {'type': 'ephemeral'}}]"
        }
    ],
    "stream": true
}

Notice that in the CodeGate version, all the system configuration is serialized into a single message string, which is not what Anthropic expects.

Steps to Reproduce

For testing without Codegate:

• simply set up Cline to target mitmproxy, and send a message through Cline in VsCode

For testing with Codegate:

• start codegate with ANTHROPIC_BASE_URL=http://127.0.0.1:8080 poetry run codegate serve
• simply set up Cline to target codegate, and send a message through Cline in VsCode

Operating System

Linux (Intel)

IDE and Version

VS CODE 1.9.5.3

Extension and Version

Cline v3.2.12

Provider

Anthropic

Model

claude-3-5-sonnet

Codegate version

CodeGate 0.1.7 (as returned by codegate version), 0ed1365

Logs

No response

Additional Context

No response

[blkt]

This issue is caused by these lines. It should be resolved once we wrote our internal representation.

[kantord]

This issue is caused by these lines. It should be resolved once we wrote our internal representation.

nice - does it mean that this issue should be closed as a duplicate? if so, please close it but mention where it is tracked now