You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
This is closely linked to #500, which (correct me if i am wrong) should already be implemented, but not via the Batch API. Integrating the batch API will save evaluations by costs up to 50% lower costs 1, 2.
Describe the solution you'd like
Ideally have a switch that allows user to select batch_mode which is available only if user has selected a valid model that has batch API support (e.g. openai/gemini)
Describe alternatives you've considered
n/a
Additional context
n/a
The text was updated successfully, but these errors were encountered:
Hey @luarss will try to look at this a bit later, batch mode right now in deepeval is the most best supported since we're not the most familiar with it. If you have any suggestions or PRs, would also appreciate it greatly
Is your feature request related to a problem? Please describe.
This is closely linked to #500, which (correct me if i am wrong) should already be implemented, but not via the Batch API. Integrating the batch API will save evaluations by costs up to 50% lower costs 1, 2.
Describe the solution you'd like
Ideally have a switch that allows user to select
batch_mode
which is available only if user has selected a valid model that has batch API support (e.g. openai/gemini)Describe alternatives you've considered
n/a
Additional context
n/a
The text was updated successfully, but these errors were encountered: