Chatgpt4o and context window/number of request

[RistoRoosipuu]

Hey.

Has anyone tried to translate an entire movie/episode in a single request using gpt4o? I am wondering if it makes sense since it would be cheaper. Right now, per episode, its about 30-40 cents. Expensive but quite worth it.

[machinewrapped]

The largest batch size I've tried with 4o so far is 100 or maybe 120 lines, which it handled easily, so it can probably go higher.

Usually translations are limited by output tokens, which were historically limited to 4096 even on the 128,000 token gpt-4 models:
https://platform.openai.com/docs/models/gpt-4-turbo-and-gpt-4

They don't mention a specific output token limit for gpt-4o or gpt4-turbo though. I'm not sure if that means the full context can potentially be used for output or it's just an omission, but I would guess there is still a limit.

(this forum thread suggests it is the same: https://community.openai.com/t/gpt-4o-max-tokens-output-response-length/748822)

Gemini 1.5 Flash supposedly supports up to 8192 output tokens, so worth trying that too, but I have already seen it return an incomplete translation of a large batch once or twice.

Larger batches mean more opportunities for errors, and if it means more retries it can negate the savings pretty quickly. Output tokens are about 3x more expensive than input tokens, so one large batch that needs to be retried in full will probably cost more than three smaller, safer batches. It would be good to find what the "safe" limits are though!

[machinewrapped]

New record!