Error rate and Decoding heuristics

[tom-huntington]

The original paper gives really good information on the accuracy of the different heuristics:

	TED-LIUM3	Meanwhile	Kincaid46	Rev16	Earnings-21	Earnings-22	CORAAL	Average
Greedy decoding only	3.55	5.95	9.96	12.1	10.5	13.9	21.3	11.0
+ Beam search	3.66	5.78	9.18	12.1	10.2	13.5	20.1	10.6
+ Temperature fallback	3.66	5.72	8.93	12.0	10.2	13.4	19.8	10.5
+ Previous-text conditioning	3.68	5.35	9.19	11.3	9.78	12.8	19.8	10.3
+ Voice activity detection	3.57	5.28	9.16	10.8	9.75	12.9	19.9	10.2
+ Initial timestamp constraint	3.56	5.33	8.87	10.6	9.68	13.0	19.9	10.1

In total, the different decoding heuristics correctly transcribe about 1 extra word in 100 when the error is sufficiently high.

Would have been interested to know the word error rate breakdown for function vs content words.

Greedy decoding is good enough for me, but 1 extra content word in 100 might make it worth implementing additional heretics in for a onnx/cpp deployment.

[tom-huntington]

We developed a strategy to perform buffered transcription of long audio by consecutively transcribing 30-second
segments of audio and shifting the window according to the timestamps predicted by the model.

I'm wondering about the error rates of transcribing the segments in parallel with some kind of appropriate windowing heuristic.

You would loose the previous-text conditioning, but that doesn't seem to affect the error rate that much.

[tom-huntington]

transcribing the segments in parallel

The current kv caching isn't going to work for this. You'd need to batch the kv cache rather than broadcast it.