Updating to Whisper.cpp v1.6.2+

[abbotadmin]

Whisper .NET seems to be quite far behind the main releases of Whisper.cpp, including some major improvements to how it works with GPUs, etc.

I am curious if this framework is abandoned or whether this work is currently in progress. @sandrohanea would you mind providing some guidance here? Are you currently still maintaining the framework?

[dfengpo]

I have same question, looking forward to updates

[sandrohanea]

Hello all,
It's not abandoned, but indeed, I didn't have that much time to respond to all the issues that people encountered lately (mostly loading different runtimes and sometimes not supported ones).

Regarding the upgrade, it is due indeed and will try to do it soon.

[abbotadmin]

That's great news. Of course you can only do what you have time for and I'm currently using what you've got here to great effect so thanks for the hard work so far.

Do you think the whispercpp upgrade itself is a lot of work? It's something I was considering looking into myself but alas I too am short on time at the moment.

[sandrohanea]

WIP branch is here: #201 in case someone wants to play with it and provide some early feedback :)

I managed to make it work with Windows x64, but for the other platforms / GPUs, there is still some work to be done.

[dfengpo]

Can you give me a rough idea of the implementation plan? I am willing to try to make some contributions

[Sing303]

Will it be possible to do #114 within this PR ?
Very important feature (if you have time)

[sandrohanea]

Hello @Sing303,
It seems that Clblast was removed from whisper.cpp: ggerganov/whisper.cpp@3b1ac03

However, Cuda will still be available.
To switch from Cuda (GPU) to CPU, Ill try to see if useGpu: can be used with the newer version, but initially, I just try to upgrade it and create some preview version.

[Sing303]

I think a CUDA backend with Gpu = false won't work if there is no Nvidia video card anyway. But we need to check

[abbotadmin]

@sandrohanea I thought I'd give the WIP 1.6.0 a go however unfortunately it looks like your LFS quota is maxed out and the clone doesn't complete sadly.

[furkanguzel161]

@sandrohanea hello will there be an update for openvino ? #146
in this issue you said there are no eta , i like to help but i have not enough knowledge to add support myself.

[dfengpo]

any updates?

[apipino]

Would it be possible to implement something like FastWhisper? (https://github.com/SYSTRAN/faster-whisper)

Faster Whisper transcription with CTranslate2:
Implementation / Precision / Beam size / Time / Max. GPU memory / Max. CPU memory
openai/whisper / fp16 / 5 / 4m30s / 11325MB / 9439MB
faster-whisper / fp16 / 5 / 54s / 4755MB / 3244MB

[sandrohanea]

Great news everyone!
Whisper.net 1.7.0 was released (skipped 1.6.0 after all).

In compensation for such a long pause between releases, this one is quite big, check release notes: https://github.com/sandrohanea/whisper.net/releases/tag/1.7.0

It contains open vino, multiple runtimes support, vulkan, etc.