Generate function calling with "strict" enabled for assistants

[owengo]

The latest versions of gpt-4o and gpt-4o-mini support "strict" structured output and "strict" function calling.
Technically OpenAI generates a "token mask" which prevents the llm to generate an invalid output / function call.

For example, when using assistant's actions in librechat, under the hood, function calls are generated for the subjacent assistant in the api. As of today these function calls are not strict, so it's possible ( and it happens ) that the llm hallucinates the function call.

For example the function generated to call a basic calculator is like this:
{ "name": "math_action_YXBpLm1hdG", "description": "Evaluate a mathematical expression", "parameters": { "type": "object", "properties": { "expr": { "type": "string", "description": "The mathematical expression to evaluate (e.g., 2+3)." } }, "required": [ "expr" ] }

Just typing correctly the name of the function - math_action_YXBpLm1hdG - is a challenge for the LLM because it's a lot of different tokens. When the llm misstypes the function names librechat's backend does not call the tool and returns an empty output.

If instead the function call was like this:
{ "name": "math_action_YXBpLm1hdG", "description": "Evaluate a mathematical expression", "strict": true, "parameters": { "type": "object", "properties": { "expr": { "type": "string", "description": "The mathematical expression to evaluate (e.g., 2+3)." } }, "required": [ "expr" ], "additionalProperties": false } }

( note the: "strict": true, "additionalProperties": false )

It would be impossible for the llm to mistype the function name because each non-matching token would have a 0 probabitity forced on it. And it would apply to the whole json of the function call. It's also very efficient with json enum types etc.

My proposal is to support a "x-strict": true attribute in the assistant openapi spec which would trigger the generated of a function call with "strict": true and "additionalProperties": false .

[owengo]

#4639