feat: model as first parameter in body for proxy routing efficiency #2400
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I would like that the generator could produce code with the "model" field as the first parameter only.
Proxies such as open-router (or custom proxies) use the "model" field as a routing parameter. We don't need to read the entire payload for routing, we can readily exit the read when we find the "model" field.
The important thing is that the "model" field should be before the "prompt" or "inputs" as those can be very large.