Improve performance, inspire from whisper-jax

[thewh1teagle]

I just saw the project whisper-jax which says it works 70x faster than original using google-jax.
But they use Python and it's not really usable at this moment for using in projects like vibe
But 70x faster is really attractive.
Do we have a way to improve whisper.cpp to get similar performance? what so special in google-jax?

[bobqianic]

70x faster

Whisper-jax segments audio into 30-second chunks and processes them in batches, which enhances the Compute-to-Memory Ratio. You can find more about this ratio here: Compute-To-Memory Ratio. In most cases, especially when using any kind of accelerator, the main limitation in decoding is the memory bandwidth. We can definitely implement this feature. It just might take a little time since I'm not a full-time developer here.