You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
x-portkey-strict-open-ai-complaince: false the Chain of Thought message is included in the response and separated with \r\n\r\n
example:
{
"id": "portkey-2b003c25-7379-4620-a6f0-0b0ba3bc5d51",
"object": "chat_completion",
"created": 1736168989,
"model": "Unknown",
"provider": "vertex-ai",
"choices": [
{
"message": {
"role": "assistant",
"content": "The user is continuing the pattern of simple arithmetic questions, building upon the previous answer. They are asking for the product of 4 multiplied by itself. I need to perform this multiplication.\r\n\r\nFollowing that same line of thought, 4 * 4 = 16\n"
},
"index": 0,
"finish_reason": "STOP"
}
],
"usage": {
"prompt_tokens": 57,
"completion_tokens": 55,
"total_tokens": 112
}
}
x-portkey-strict-open-ai-complaince: true only the response is sent, not the CoT message
For streaming, we'll be sending both the messages together without any separator
edit:
Reason for not transforming streaming messages.
These two are responses sent by gemini when streaming for the same prompt:
[
{
text: 'The user is following a pattern of basic arithmetic questions. They previously asked for the'
}
]
[ { text: ' sum of 2+2, and now they are asking for the product of' } ]
[
{ text: ' 4*4. I need to perform this multiplication.' },
{ text: 'Following that line of thought, 4 * 4 = 16\n' }
]
[
{
text: 'The user is continuing with basic arithmetic questions, following the pattern of the previous question'
}
]
[ { text: '. I need to calculate the result of 4 multiplied by 4.' } ]
[ { text: 'Following that same line of thought:\n\n4 * 4 = 16\n' } ]
There is no index for the text part and so it is not possible to add the separator when both the text parts arrive in separate chunks
The text was updated successfully, but these errors were encountered:
What Would You Like to See with the Gateway?
x-portkey-strict-open-ai-complaince: false
the Chain of Thought message is included in the response and separated with\r\n\r\n
example:
x-portkey-strict-open-ai-complaince: true
only the response is sent, not the CoT messageFor streaming, we'll be sending both the messages together without any separator
edit:
Reason for not transforming streaming messages.
These two are responses sent by gemini when streaming for the same prompt:
There is no index for the text part and so it is not possible to add the separator when both the text parts arrive in separate chunks
The text was updated successfully, but these errors were encountered: