Unexpected end of stream error while waiting for transcription

[A1trdX]

I use the "whisper-large" model. Very often, audio transcriptions (especially those lasting 1 hour or more) take more than 1 minute. And for some reason, the server simply disconnects after exactly 1 minute (at this time, the website logs still show the "Pending" status) without any response, making it impossible to get the transcription.

Examples:

1. request_id=c760e8b2-b979-42e8-a637-cbd954fcae15
2. request_id=338eab89-d381-43bc-9d1d-79148b835b04

Only model and language request params used.

The program is written in Java using OkHttp 4.12.0.
Program log:

20-09-2024 17:31:16.083/+0300 [#1 main] [INFO]: Transcribing...
20-09-2024 17:31:17.258/+0300 [#1 main] [INFO]: Uploading audio file: 0% (0.01 / 128.79 MiB)
20-09-2024 17:31:18.476/+0300 [#1 main] [INFO]: Uploading audio file: 10% (12.89 / 128.79 MiB)
20-09-2024 17:31:19.035/+0300 [#1 main] [INFO]: Uploading audio file: 20% (25.77 / 128.79 MiB)
20-09-2024 17:31:19.589/+0300 [#1 main] [INFO]: Uploading audio file: 30% (38.66 / 128.79 MiB)
20-09-2024 17:31:20.034/+0300 [#1 main] [INFO]: Uploading audio file: 40% (51.54 / 128.79 MiB)
20-09-2024 17:31:20.473/+0300 [#1 main] [INFO]: Uploading audio file: 50% (64.42 / 128.79 MiB)
20-09-2024 17:31:20.897/+0300 [#1 main] [INFO]: Uploading audio file: 60% (77.30 / 128.79 MiB)
20-09-2024 17:31:21.379/+0300 [#1 main] [INFO]: Uploading audio file: 70% (90.19 / 128.79 MiB)
20-09-2024 17:31:21.828/+0300 [#1 main] [INFO]: Uploading audio file: 80% (103.07 / 128.79 MiB)
20-09-2024 17:31:22.354/+0300 [#1 main] [INFO]: Uploading audio file: 90% (115.95 / 128.79 MiB)
20-09-2024 17:31:22.955/+0300 [#1 main] [INFO]: Uploading audio file: 100% (128.79 / 128.79 MiB)
20-09-2024 17:31:22.956/+0300 [#1 main] [INFO]: Waiting for response...
20-09-2024 17:32:23.438/+0300 [#1 main] [ERROR]: Unhandled exception
java.io.IOException: unexpected end of stream on https://api.deepgram.com/...
	at okhttp3.internal.http1.Http1ExchangeCodec.readResponseHeaders(Http1ExchangeCodec.kt:210) ~[okhttp-4.12.0.jar:?]
	at okhttp3.internal.connection.Exchange.readResponseHeaders(Exchange.kt:110) ~[okhttp-4.12.0.jar:?]
	at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.kt:93) ~[okhttp-4.12.0.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) ~[okhttp-4.12.0.jar:?]
	at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:34) ~[okhttp-4.12.0.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) ~[okhttp-4.12.0.jar:?]
	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:95) ~[okhttp-4.12.0.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) ~[okhttp-4.12.0.jar:?]
	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83) ~[okhttp-4.12.0.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) ~[okhttp-4.12.0.jar:?]
	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76) ~[okhttp-4.12.0.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) ~[okhttp-4.12.0.jar:?]
	at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201) ~[okhttp-4.12.0.jar:?]
	at okhttp3.internal.connection.RealCall.execute(RealCall.kt:154) ~[okhttp-4.12.0.jar:?]
	at org.a1trdx.transcriber.transcription.provider.types.deepgram.network.DeepgramClient.send(DeepgramClient.java:84) ~[Transcriber-1.0.0-SNAPSHOT.jar:?]
	at org.a1trdx.transcriber.transcription.provider.types.deepgram.DeepgramTranscriptionProvider.transcribe(DeepgramTranscriptionProvider.java:31) ~[Transcriber-1.0.0-SNAPSHOT.jar:?]
	at org.a1trdx.transcriber.Boot.run(Boot.java:56) ~[Transcriber-1.0.0-SNAPSHOT.jar:?]
	at org.a1trdx.transcriber.Boot.main(Boot.java:21) [Transcriber-1.0.0-SNAPSHOT.jar:?]
Caused by: java.io.EOFException: \n not found: limit=0 content=…
	at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.kt:335) ~[okio-jvm-3.6.0.jar:?]
	at okhttp3.internal.http1.HeadersReader.readLine(HeadersReader.kt:29) ~[okhttp-4.12.0.jar:?]
	at okhttp3.internal.http1.Http1ExchangeCodec.readResponseHeaders(Http1ExchangeCodec.kt:180) ~[okhttp-4.12.0.jar:?]
	... 17 more

Code that sends the request:

public DeepgramResponse send(DeepgramRequest request) {
    try {
        var httpClient = createHttpClient();
        var mimeType = "application/octet-stream";

        @SuppressWarnings("DataFlowIssue")
        var urlBuilder = API_URL.newBuilder();
        addQueryParams(urlBuilder, request);
        var url = urlBuilder.build();

        var progressUpdateLimiter = new ProgressUpdateLimiter(0.10);
        var httpRequestBody = new ProgressRequestBody(
                RequestBody.create(request.getAudioFile().toFile(), MediaType.parse(mimeType)),
                new ProgressRequestBody.ProgressListener() {
                    @Override
                    public void onProgress(long bytesWritten, long contentLength) {
                        if (progressUpdateLimiter.setProgress(bytesWritten, contentLength)) {
                            LOGGER.info(
                                    "Uploading audio file: {}% ({} / {} MiB)",
                                    (long) ((double) bytesWritten / contentLength * 100),
                                    String.format("%.2f", bytesToMib(bytesWritten)),
                                    String.format("%.2f", bytesToMib(contentLength)));
                        }
                    }

                    @Override
                    public void onUploadFinish() {
                        LOGGER.info("Waiting for response...");
                    }

                    private double bytesToMib(long numBytes) {
                        return (double) numBytes / 1024 / 1024;
                    }
                });


        var httpRequest = new Request.Builder()
                .url(url)
                .post(httpRequestBody)
                .addHeader("Authorization", "Token " + apiKey)
                .addHeader("Content-Type", mimeType)
                .build();

        try (var httpResponse = httpClient.newCall(httpRequest).execute()) { // DeepgramClient.java:84
            if (!httpResponse.isSuccessful()) {
                var httpResponseBody = httpResponse.body();
                var r = httpResponseBody == null ? String.valueOf(httpResponse) : httpResponseBody.string();
                throw new IOException(r);
            }

            var httpResponseBody = Objects.requireNonNull(httpResponse.body());
            var httpResponseBodyAsString = httpResponseBody.string();

            var gson = new Gson();
            return gson.fromJson(httpResponseBodyAsString, DeepgramResponse.class);
        }
    } catch (Exception ex) {
        return SneakyThrow.accept(ex);
    }
}

HTTP client setup:

private static OkHttpClient createHttpClient() {
    return new OkHttpClient.Builder()
            .retryOnConnectionFailure(false)
            .connectTimeout(30, TimeUnit.SECONDS)
            .writeTimeout(10, TimeUnit.MINUTES)
            .readTimeout(10, TimeUnit.MINUTES)
            .callTimeout(10, TimeUnit.MINUTES)
            .pingInterval(30, TimeUnit.SECONDS)
            .build();
}

The error only occurs when the wait is longer than 1 minute, otherwise everything works fine.

[team-deepgram]

Thanks for asking your question about Deepgram! If you didn't already include it in your post, please be sure to add as much detail as possible so we can assist you efficiently, such as:

• The request_id if you have a question about your requests or transcription responses.
• The features you used or the full api.deepgram.com URL you sent your request to, including parameters.
• Any code snippets you can share.

[A1trdX]

I noticed that a very similar problem is present in the Deepgram API Playground. I uploaded a 2-hour audio (m4a), configured all the same parameters and also added "smart_format". After 2 minutes, I got the following error:

I went to the "Usage" tab and saw that it made 2 requests with an interval of one minute:

• f3430bb4-e61f-475f-99fc-099a0a773b3d
• 95e0ca1a-ef70-419b-81ae-8c365f7250ac

There are no errors in the requests (same as in the previous ones), the result is 200 OK.

[jpvajda]

@A1trdX we advise moving away from using Whisper if you can . Our Nova-2 model is much more accurate and efficient, so unless you need to stay on Whisper for some other reason, I'd suggest this approach.

You are likely hitting our time out limit for Whisper since it's not a model that is highly optimized. You can read more about that here:

[A1trdX]

The documentation says that the maximum processing time for "nova-2" is 10 minutes, and for "whisper-large" - 20 minutes. But the server disconnects after 1 minute (the transcription itself continues to work). Therefore, this is a bug. At the moment, I found a workaround - use callback and install additional intermediary software on the VPS. But this is a bit inconvenient and incurs additional costs.

I agree with you that "nova-2" is faster, but in some cases "whisper-large" still has an advantage in transcription quality. For example, it can correctly recognize coordinates and moves in chess and write it down as a notation. But the main problem with "nova-2" is that in some languages ​​it skips entire language constructs, which is unacceptable for my purposes.

However, this is a separate topic. I just want to emphasize that the connection breaks down much earlier than it should.

P.S. I unmarked the answer because using a different model does not solve the main problem - too short server timeout, different from the documentation.