Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support choosing chatCompletionV2 or chatCompletionPro API for minimax provider #1593

Merged
merged 2 commits into from
Dec 15, 2024
Merged

feat: support choosing chatCompletionV2 or chatCompletionPro API for minimax provider #1593

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f4c8b52
Supports choosing chatCompletionV2 or chatCompletionPro API for minim…
hanxiantao Dec 14, 2024
f70b310
Merge branch 'main' into minimax-ai-proxy
hanxiantao Dec 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 19 additions & 13 deletions plugins/wasm-go/extensions/ai-proxy/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -174,9 +174,10 @@ Mistral 所对应的 `type` 为 `mistral`。它并无特有的配置字段。

MiniMax所对应的 `type` 为 `minimax`。它特有的配置字段如下：

| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
| ---------------- | -------- | ------------------------------------------------------------ | ------ | ------------------------------------------------------------ |
| `minimaxGroupId` | string | 当使用`abab6.5-chat`, `abab6.5s-chat`, `abab5.5s-chat`, `abab5.5-chat`四种模型时必填 | - | 当使用`abab6.5-chat`, `abab6.5s-chat`, `abab5.5s-chat`, `abab5.5-chat`四种模型时会使用ChatCompletion Pro，需要设置groupID |
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
| ---------------- | -------- | ------------------------------ | ------ |----------------------------------------------------------------|
| `minimaxApiType` | string | v2 和 pro 中选填一项 | v2 | v2 代表 ChatCompletion v2 API，pro 代表 ChatCompletion Pro API |
| `minimaxGroupId` | string | `minimaxApiType` 为 pro 时必填 | - | `minimaxApiType` 为 pro 时使用 ChatCompletion Pro API，需要设置 groupID |

#### Anthropic Claude

Expand Down Expand Up @@ -1000,17 +1001,16 @@ provider:
apiTokens:
- "YOUR_MINIMAX_API_TOKEN"
modelMapping:
"gpt-3": "abab6.5g-chat"
"gpt-4": "abab6.5-chat"
"*": "abab6.5g-chat"
minimaxGroupId: "YOUR_MINIMAX_GROUP_ID"
"gpt-3": "abab6.5s-chat"
"gpt-4": "abab6.5g-chat"
"*": "abab6.5t-chat"
```

**请求示例**

```json
{
"model": "gpt-4-turbo",
"model": "gpt-3",
"messages": [
{
"role": "user",
Expand All @@ -1025,27 +1025,33 @@ provider:

```json
{
"id": "02b2251f8c6c09d68c1743f07c72afd7",
"id": "03ac4fcfe1c6cc9c6a60f9d12046e2b4",
"choices": [
{
"finish_reason": "stop",
"index": 0,
"message": {
"content": "你好！我是MM智能助理，一款由MiniMax自研的大型语言模型。我可以帮助你解答问题，提供信息，进行对话等。有什么可以帮助你的吗？",
"role": "assistant"
"content": "你好，我是一个由MiniMax公司研发的大型语言模型，名为MM智能助理。我可以帮助回答问题、提供信息、进行对话和执行多种语言处理任务。如果你有任何问题或需要帮助，请随时告诉我！",
"role": "assistant",
"name": "MM智能助理",
"audio_content": ""
}
}
],
"created": 1717760544,
"created": 1734155471,
"model": "abab6.5s-chat",
"object": "chat.completion",
"usage": {
"total_tokens": 106
"total_tokens": 116,
"total_characters": 0,
"prompt_tokens": 70,
"completion_tokens": 46
},
"input_sensitive": false,
"output_sensitive": false,
"input_sensitive_type": 0,
"output_sensitive_type": 0,
"output_sensitive_int": 0,
"base_resp": {
"status_code": 0,
"status_msg": ""
Expand Down
119 changes: 44 additions & 75 deletions plugins/wasm-go/extensions/ai-proxy/provider/minimax.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,47 +11,37 @@ import (
"github.com/alibaba/higress/plugins/wasm-go/pkg/wrapper"
"github.com/higress-group/proxy-wasm-go-sdk/proxywasm"
"github.com/higress-group/proxy-wasm-go-sdk/proxywasm/types"
"github.com/tidwall/gjson"
"github.com/tidwall/sjson"
)

// minimaxProvider is the provider for minimax service.

const (
minimaxDomain = "api.minimax.chat"
// minimaxChatCompletionV2Path 接口请求响应格式与OpenAI相同
// 接口文档: https://platform.minimaxi.com/document/guides/chat-model/V2?id=65e0736ab2845de20908e2dd
minimaxApiTypeV2 = "v2" // minimaxApiTypeV2 represents chat completion V2 API.
minimaxApiTypePro = "pro" // minimaxApiTypePro represents chat completion Pro API.
minimaxDomain = "api.minimax.chat"
// minimaxChatCompletionV2Path represents the API path for chat completion V2 API which has a response format similar to OpenAI's.
minimaxChatCompletionV2Path = "/v1/text/chatcompletion_v2"
// minimaxChatCompletionProPath 接口请求响应格式与OpenAI不同
// 接口文档: https://platform.minimaxi.com/document/guides/chat-model/pro/api?id=6569c85948bc7b684b30377e
// minimaxChatCompletionProPath represents the API path for chat completion Pro API which has a different response format from OpenAI's.
minimaxChatCompletionProPath = "/v1/text/chatcompletion_pro"

senderTypeUser string = "USER" // 用户发送的内容
senderTypeBot string = "BOT" // 模型生成的内容
senderTypeUser string = "USER" // Content sent by the user.
senderTypeBot string = "BOT" // Content generated by the model.

// 默认机器人设置
// Default bot settings.
defaultBotName string = "MM智能助理"
defaultBotSettingContent string = "MM智能助理是一款由MiniMax自研的，没有调用其他产品的接口的大型语言模型。MiniMax是一家中国科技公司，一直致力于进行大模型相关的研究。"
defaultSenderName string = "小明"
)

// chatCompletionProModels 这些模型对应接口为ChatCompletion Pro
var chatCompletionProModels = map[string]struct{}{
"abab6.5-chat": {},
"abab6.5s-chat": {},
"abab5.5s-chat": {},
"abab5.5-chat": {},
}

type minimaxProviderInitializer struct {
}

func (m *minimaxProviderInitializer) ValidateConfig(config ProviderConfig) error {
// 如果存在模型对应接口为ChatCompletion Pro必须配置minimaxGroupId
if len(config.modelMapping) > 0 && config.minimaxGroupId == "" {
for _, minimaxModel := range config.modelMapping {
if _, exists := chatCompletionProModels[minimaxModel]; exists {
return errors.New(fmt.Sprintf("missing minimaxGroupId in provider config when %s model is provided", minimaxModel))
}
}
// If using the chat completion Pro API, a group ID must be set.
if minimaxApiTypePro == config.minimaxApiType && config.minimaxGroupId == "" {
return errors.New(fmt.Sprintf("missing minimaxGroupId in provider config when minimaxApiType is %s", minimaxApiTypePro))
}
if config.apiTokens == nil || len(config.apiTokens) == 0 {
return errors.New("no apiToken found in provider config")
Expand Down Expand Up @@ -94,23 +84,11 @@ func (m *minimaxProvider) OnRequestBody(ctx wrapper.HttpContext, apiName ApiName
if apiName != ApiNameChatCompletion {
return types.ActionContinue, errUnsupportedApiName
}
// 解析并映射模型,设置上下文
model, err := m.parseModel(body)
if err != nil {
return types.ActionContinue, err
}
ctx.SetContext(ctxKeyOriginalRequestModel, model)
mappedModel := getMappedModel(model, m.config.modelMapping, log)
if mappedModel == "" {
return types.ActionContinue, errors.New("model becomes empty after applying the configured mapping")
}
ctx.SetContext(ctxKeyFinalRequestModel, mappedModel)
_, ok := chatCompletionProModels[mappedModel]
if ok {
// 使用ChatCompletion Pro接口
if minimaxApiTypePro == m.config.minimaxApiType {
// Use chat completion Pro API.
return m.handleRequestBodyByChatCompletionPro(body, log)
} else {
// 使用ChatCompletion v2接口
// Use chat completion V2 API.
return m.config.handleRequestBody(m, m.contextCache, ctx, apiName, body, log)
}
}
Expand All @@ -119,14 +97,14 @@ func (m *minimaxProvider) TransformRequestBodyHeaders(ctx wrapper.HttpContext, a
return m.handleRequestBodyByChatCompletionV2(body, headers, log)
}

// handleRequestBodyByChatCompletionPro 使用ChatCompletion Pro接口处理请求体
// handleRequestBodyByChatCompletionPro processes the request body using the chat completion Pro API.
func (m *minimaxProvider) handleRequestBodyByChatCompletionPro(body []byte, log wrapper.Log) (types.Action, error) {
request := &chatCompletionRequest{}
if err := decodeChatCompletionRequest(body, request); err != nil {
return types.ActionContinue, err
}

// 映射模型重写requestPath
// Map the model and rewrite the request path.
request.Model = getMappedModel(request.Model, m.config.modelMapping, log)
_ = util.OverwriteRequestPath(fmt.Sprintf("%s?GroupId=%s", minimaxChatCompletionProPath, m.config.minimaxGroupId))

Expand All @@ -143,9 +121,9 @@ func (m *minimaxProvider) handleRequestBodyByChatCompletionPro(body []byte, log
log.Errorf("failed to load context file: %v", err)
util.ErrorHandler("ai-proxy.minimax.load_ctx_failed", fmt.Errorf("failed to load context file: %v", err))
}
// 由于 minimaxChatCompletionV2（格式和 OpenAI 一致）和 minimaxChatCompletionPro（格式和 OpenAI 不一致）中 insertHttpContextMessage 的逻辑不同，无法做到同一个 provider 统一
// 因此对于 minimaxChatCompletionPro 需要手动处理 context 消息
// minimaxChatCompletionV2 交给默认的 defaultInsertHttpContextMessage 方法插入 context 消息
// Since minimaxChatCompletionV2 (format consistent with OpenAI) and minimaxChatCompletionPro (different format from OpenAI) have different logic for insertHttpContextMessage, we cannot unify them within one provider.
// For minimaxChatCompletionPro, we need to manually handle context messages.
// minimaxChatCompletionV2 uses the default defaultInsertHttpContextMessage method to insert context messages.
minimaxRequest := m.buildMinimaxChatCompletionV2Request(request, content)
if err := replaceJsonRequestBody(minimaxRequest, log); err != nil {
util.ErrorHandler("ai-proxy.minimax.insert_ctx_failed", fmt.Errorf("failed to replace Request body: %v", err))
Expand All @@ -157,54 +135,45 @@ func (m *minimaxProvider) handleRequestBodyByChatCompletionPro(body []byte, log
return types.ActionContinue, err
}

// handleRequestBodyByChatCompletionV2 使用ChatCompletion v2接口处理请求体
// handleRequestBodyByChatCompletionV2 processes the request body using the chat completion V2 API.
func (m *minimaxProvider) handleRequestBodyByChatCompletionV2(body []byte, headers http.Header, log wrapper.Log) ([]byte, error) {
request := &chatCompletionRequest{}
if err := decodeChatCompletionRequest(body, request); err != nil {
return nil, err
}

// 映射模型重写requestPath
request.Model = getMappedModel(request.Model, m.config.modelMapping, log)
util.OverwriteRequestPathHeader(headers, minimaxChatCompletionV2Path)

return body, nil
rawModel := gjson.GetBytes(body, "model").String()
mappedModel := getMappedModel(rawModel, m.config.modelMapping, log)
return sjson.SetBytes(body, "model", mappedModel)
}

func (m *minimaxProvider) OnResponseHeaders(ctx wrapper.HttpContext, apiName ApiName, log wrapper.Log) (types.Action, error) {
// 使用minimax接口协议,跳过OnStreamingResponseBody()和OnResponseBody()
// Skip OnStreamingResponseBody() and OnResponseBody() when using original protocol.
if m.config.protocol == protocolOriginal {
ctx.DontReadResponseBody()
return types.ActionContinue, nil
}
// 模型对应接口为ChatCompletion v2,跳过OnStreamingResponseBody()和OnResponseBody()
model := ctx.GetStringContext(ctxKeyFinalRequestModel, "")
if model != "" {
_, ok := chatCompletionProModels[model]
if !ok {
ctx.DontReadResponseBody()
return types.ActionContinue, nil
}
// Skip OnStreamingResponseBody() and OnResponseBody() when the model corresponds to the chat completion V2 interface.
if minimaxApiTypePro != m.config.minimaxApiType {
ctx.DontReadResponseBody()
return types.ActionContinue, nil
}
_ = proxywasm.RemoveHttpResponseHeader("Content-Length")
return types.ActionContinue, nil
}

// OnStreamingResponseBody 只处理使用OpenAI协议 且 模型对应接口为ChatCompletion Pro的流式响应
// OnStreamingResponseBody handles streaming response chunks from the Minimax service only for requests using the OpenAI protocol and corresponding to the chat completion Pro API.
func (m *minimaxProvider) OnStreamingResponseBody(ctx wrapper.HttpContext, name ApiName, chunk []byte, isLastChunk bool, log wrapper.Log) ([]byte, error) {
if isLastChunk || len(chunk) == 0 {
return nil, nil
}
// sample event response:
// Sample event response:
// data: {"created":1689747645,"model":"abab6.5s-chat","reply":"","choices":[{"messages":[{"sender_type":"BOT","sender_name":"MM智能助理","text":"am from China."}]}],"output_sensitive":false}

// sample end event response:
// Sample end event response:
// data: {"created":1689747645,"model":"abab6.5s-chat","reply":"I am from China.","choices":[{"finish_reason":"stop","messages":[{"sender_type":"BOT","sender_name":"MM智能助理","text":"I am from China."}]}],"usage":{"total_tokens":187},"input_sensitive":false,"output_sensitive":false,"id":"0106b3bc9fd844a9f3de1aa06004e2ab","base_resp":{"status_code":0,"status_msg":""}}
responseBuilder := &strings.Builder{}
lines := strings.Split(string(chunk), "\n")
for _, data := range lines {
if len(data) < 6 {
// ignore blank line or wrong format
// Ignore blank line or improperly formatted lines.
continue
}
data = data[6:]
Expand All @@ -226,7 +195,7 @@ func (m *minimaxProvider) OnStreamingResponseBody(ctx wrapper.HttpContext, name
return []byte(modifiedResponseChunk), nil
}

// OnResponseBody 只处理使用OpenAI协议 且 模型对应接口为ChatCompletion Pro的流式响应
// OnResponseBody handles the final response body from the Minimax service only for requests using the OpenAI protocol and corresponding to the chat completion Pro API.
func (m *minimaxProvider) OnResponseBody(ctx wrapper.HttpContext, apiName ApiName, body []byte, log wrapper.Log) (types.Action, error) {
minimaxResp := &minimaxChatCompletionV2Resp{}
if err := json.Unmarshal(body, minimaxResp); err != nil {
Expand All @@ -239,39 +208,39 @@ func (m *minimaxProvider) OnResponseBody(ctx wrapper.HttpContext, apiName ApiNam
return types.ActionContinue, replaceJsonResponseBody(response, log)
}

// minimaxChatCompletionV2Request 表示ChatCompletion V2请求的结构体
// minimaxChatCompletionV2Request represents the structure of a chat completion V2 request.
type minimaxChatCompletionV2Request struct {
Model string `json:"model"`
Stream bool `json:"stream,omitempty"`
TokensToGenerate int64 `json:"tokens_to_generate,omitempty"`
Temperature float64 `json:"temperature,omitempty"`
TopP float64 `json:"top_p,omitempty"`
MaskSensitiveInfo bool `json:"mask_sensitive_info"` // 是否开启隐私信息打码,默认true
MaskSensitiveInfo bool `json:"mask_sensitive_info"` // Whether to mask sensitive information, defaults to true.
Messages []minimaxMessage `json:"messages"`
BotSettings []minimaxBotSetting `json:"bot_setting"`
ReplyConstraints minimaxReplyConstraints `json:"reply_constraints"`
}

// minimaxMessage 表示对话中的消息
// minimaxMessage represents a message in the conversation.
type minimaxMessage struct {
SenderType string `json:"sender_type"`
SenderName string `json:"sender_name"`
Text string `json:"text"`
}

// minimaxBotSetting 表示机器人的设置
// minimaxBotSetting represents the bot's settings.
type minimaxBotSetting struct {
BotName string `json:"bot_name"`
Content string `json:"content"`
}

// minimaxReplyConstraints 表示模型回复要求
// minimaxReplyConstraints represents requirements for model replies.
type minimaxReplyConstraints struct {
SenderType string `json:"sender_type"`
SenderName string `json:"sender_name"`
}

// minimaxChatCompletionV2Resp Minimax Chat Completion V2响应结构体
// minimaxChatCompletionV2Resp represents the structure of a Minimax Chat Completion V2 response.
type minimaxChatCompletionV2Resp struct {
Created int64 `json:"created"`
Model string `json:"model"`
Expand All @@ -286,20 +255,20 @@ type minimaxChatCompletionV2Resp struct {
BaseResp minimaxBaseResp `json:"base_resp"`
}

// minimaxBaseResp 包含错误状态码和详情
// minimaxBaseResp contains error status code and details.
type minimaxBaseResp struct {
StatusCode int64 `json:"status_code"`
StatusMsg string `json:"status_msg"`
}

// minimaxChoice 结果选项
// minimaxChoice represents a result option.
type minimaxChoice struct {
Messages []minimaxMessage `json:"messages"`
Index int64 `json:"index"`
FinishReason string `json:"finish_reason"`
}

// minimaxUsage 令牌使用情况
// minimaxUsage represents token usage statistics.
type minimaxUsage struct {
TotalTokens int64 `json:"total_tokens"`
}
Expand Down
6 changes: 5 additions & 1 deletion plugins/wasm-go/extensions/ai-proxy/provider/provider.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -206,8 +206,11 @@ type ProviderConfig struct {
// @Title zh-CN hunyuan api id for authorization
// @Description zh-CN 仅适用于Hun Yuan AI服务鉴权
hunyuanAuthId string `required:"false" yaml:"hunyuanAuthId" json:"hunyuanAuthId"`
// @Title zh-CN minimax API type
// @Description zh-CN 仅适用于 minimax 服务。minimax API 类型，v2 和 pro 中选填一项，默认值为 v2
minimaxApiType string `required:"false" yaml:"minimaxApiType" json:"minimaxApiType"`
// @Title zh-CN minimax group id
// @Description zh-CN 仅适用于minimax使用ChatCompletion Pro接口的模型
// @Description zh-CN 仅适用于 minimax 服务。minimax API 类型为 pro 时必填
minimaxGroupId string `required:"false" yaml:"minimaxGroupId" json:"minimaxGroupId"`
// @Title zh-CN 模型名称映射表
// @Description zh-CN 用于将请求中的模型名称映射为目标AI服务商支持的模型名称。支持通过“*”来配置全局映射
Expand Down Expand Up @@ -303,6 +306,7 @@ func (c *ProviderConfig) FromJson(json gjson.Result) {
c.claudeVersion = json.Get("claudeVersion").String()
c.hunyuanAuthId = json.Get("hunyuanAuthId").String()
c.hunyuanAuthKey = json.Get("hunyuanAuthKey").String()
c.minimaxApiType = json.Get("minimaxApiType").String()
c.minimaxGroupId = json.Get("minimaxGroupId").String()
c.cloudflareAccountId = json.Get("cloudflareAccountId").String()
if c.typ == providerTypeGemini {
Expand Down
Loading