whisper speech to text support

[trappedinspacetime]

I am running Ubuntu 22.04 Mate. I set up whispercpp and I use it with:

     notify-send -u normal  -t 10000 "$( rec /dev/shm/prerecorded.wav rate 16k  silence 1 0.1 3% 1 1.0 3% -c 1 > /dev/null 2>&1 && /home/whatnext/Desktop/2022-10/whisper.cpp/main -m /home/whatnext/Desktop/2022-10/whisper.cpp/models/ggml-small.bin -nt -l tr -f /dev/shm/prerecorded.wav |  awk '/turkish/ {for(i=1; i<=3; i++) {getline; print}}' )"

It performs very well in my language. I want to integrate it into ibus-typing-booster.
Michael Sheldon posted a deepspeech module for your ibus-typing-booster here but I don't know how to integrate it to ibus-typing-booster in my case.

[mike-fabian]

I will have a look.

[mike-fabian]

ibus-typing-booster already supports speech-to-text via googles service for quite a long time.

But that service is neither open source nor free, only one hour per month is free and the setup is very difficult.

I used it myself for a while trying to keep below the free limit but as it was hard to tell how much free time is left I stopped using it and have not even tested it for a while, so I am not even sure whether it still works.

I want to add other services for speech to text, so that the user can choose which one to use and make a good free one the default.

Because of this discussion: #324 (comment)

I discovered VOSK:

https://github.com/alphacep/vosk-api
https://alphacephei.com/vosk/

I tested it a bit on the command line.

It seemed to work quite well.

I tried only English though, I did not try how well the Turkish model works.

For English I tried:

https://alphacephei.com/vosk/models/vosk-model-small-en-us-0.15.zip (Lightweight wideband model for Android and RPi 40M)

and

https://alphacephei.com/vosk/models/vosk-model-en-us-0.22.zip (ccurate generic US English model, 1.8G)

The accurate model is huge, 1.8 GB, but it worked very well. I was not so impressed by the 40M lightweight model.

For Turkish there seems to be only:

https://alphacephei.com/vosk/models/vosk-model-small-tr-0.3.zip (Lightweight wideband model for Android and RPi, 35M)

I wonder how good that is.

I definitely want to add more speech to text engines and make the best free one the default.

Which one is better? VOSK or whisper.ccp?

In the long run I want to give the user a choice between different engines, but which one should I try to implement first?

[trappedinspacetime]

@mike-fabian Thank you very much for your support. I tried a few different STT in Turkish. I also tried Turkish version of VOSKas you mentioned above.

The other STT engine I tried:

     from asrecognition import ASREngine
     asr = ASREngine("tr", model_path="mpoyraz/wav2vec2-xls-r-300m-cv6-turkish")
     audio_paths = ["prerecorded.wav"]
     transcriptions = asr.transcribe(audio_paths)
     print(transcriptions)

If you install asrecognition, the above command downloads model from huggingface and tranbscribe the wav file.

whispercpp is a c fork of open-ai/whisper
Whisper STT is the best engine I tested. It is even better than google stt. Whisper has multilanguage support and real time translation from any language to English. It performs far better than Turkish trained model of mpoyraz/wav2vec2-xls-r-300m-cv6-turkish

I prefered whispercpp because c version performs faster and uses less memory. But open-ai/whisper offficially has better support.
English model of VOSK is not too bad but Whisper is awesome.
I hope you test it and see the difference. I must add that transcription time of Whisper is too slow for large audio transcription but it almost does a professional transcription even with small or tiny model.
Please read the following links.
https://github.com/openai/whisper/discussions/categories/show-and-tell
openai/whisper#208

[trappedinspacetime]

Another thing I noticed now, whispercpp added mic support. I just tested it after building it with

    sudo apt install libsdl2-dev
   
    make stream
    
    bash ./download-ggml-model.sh base
    
   ./stream -m ./models/ggml-base.bin -l en -t 8 --step 500 --length 5000

It perfectly transcribes my broken accent.