Add support for large-v3

[turicas]

In summary:

• Upgrade ctranslate2 and transformers
• Add large-v3 to the list of models (currently using turicas/faster-whisper-large-v3)
• Fix feature_size (n_mels) for large-v3
• Change tokenizer to be compatible with large-v3 (the older cannot be used, tokens like transcribe and translate were changed)
• Update README

Note: I'm not sure if the way I've implemented is the best one, feel free to give feedbacks. =) One of the things that can be optimized is the loading of the new tokenizer, which requires the transformers library when using large-v3.

[hedrergudene]

Shouldn't "large" be the v3 version?

[turicas]

Good point! Should large be a shortcut to the latest large model?

[captainyugi00]

Hello, I am currently getting this error after downloading the large-v3 model. Is there any way I can fix this?

File "C:\Users\User\OneDrive\Desktop\GitHub-Projects\Proj-B\faster_whisper\transcribe.py", line 149, in init
processor = AutoProcessor.from_pretrained("openai/whisper-large-v3")
File "C:\Users\User.conda\envs\Proj\lib\site-packages\transformers\models\auto\processing_auto.py", line 287, in from_pretrained
return processor_class.from_pretrained(
File "C:\Users\User.conda\envs\Proj\lib\site-packages\transformers\processing_utils.py", line 226, in from_pretrained
args = cls._get_arguments_from_pretrained(pretrained_model_name_or_path, **kwargs)
File "C:\Users\User.conda\envs\Proj\lib\site-packages\transformers\processing_utils.py", line 270, in _get_arguments_from_pretrained
args.append(attribute_class.from_pretrained(pretrained_model_name_or_path, **kwargs))
File "C:\Users\User.conda\envs\Proj\lib\site-packages\transformers\tokenization_utils_base.py", line 1854, in from_pretrained
return cls._from_pretrained(
File "C:\Users\User.conda\envs\Proj\lib\site-packages\transformers\tokenization_utils_base.py", line 2073, in _from_pretrained
raise ValueError(
ValueError: Non-consecutive added token '<|0.02|>' found. Should have index 50365 but has index 50366 in saved vocabulary.

[turicas]

Hello, I am currently getting this error after downloading the large-v3 model. Is there any way I can fix this?

File "C:\Users\User\OneDrive\Desktop\GitHub-Projects\Proj-B\faster_whisper\transcribe.py", line 149, in init processor = AutoProcessor.from_pretrained("openai/whisper-large-v3") File "C:\Users\User.conda\envs\Proj\lib\site-packages\transformers\models\auto\processing_auto.py", line 287, in from_pretrained return processor_class.from_pretrained( File "C:\Users\User.conda\envs\Proj\lib\site-packages\transformers\processing_utils.py", line 226, in from_pretrained args = cls._get_arguments_from_pretrained(pretrained_model_name_or_path, **kwargs) File "C:\Users\User.conda\envs\Proj\lib\site-packages\transformers\processing_utils.py", line 270, in _get_arguments_from_pretrained args.append(attribute_class.from_pretrained(pretrained_model_name_or_path, **kwargs)) File "C:\Users\User.conda\envs\Proj\lib\site-packages\transformers\tokenization_utils_base.py", line 1854, in from_pretrained return cls._from_pretrained( File "C:\Users\User.conda\envs\Proj\lib\site-packages\transformers\tokenization_utils_base.py", line 2073, in _from_pretrained raise ValueError( ValueError: Non-consecutive added token '<|0.02|>' found. Should have index 50365 but has index 50366 in saved vocabulary.

Try to upgrade the transformers library.

[captainyugi00]

Hello, I am currently getting this error after downloading the large-v3 model. Is there any way I can fix this?

File "C:\Users\User\OneDrive\Desktop\GitHub-Projects\Proj-B\faster_whisper\transcribe.py", line 149, in init processor = AutoProcessor.from_pretrained("openai/whisper-large-v3") File "C:\Users\User.conda\envs\Proj\lib\site-packages\transformers\models\auto\processing_auto.py", line 287, in from_pretrained return processor_class.from_pretrained( File "C:\Users\User.conda\envs\Proj\lib\site-packages\transformers\processing_utils.py", line 226, in from_pretrained args = cls._get_arguments_from_pretrained(pretrained_model_name_or_path, **kwargs) File "C:\Users\User.conda\envs\Proj\lib\site-packages\transformers\processing_utils.py", line 270, in _get_arguments_from_pretrained args.append(attribute_class.from_pretrained(pretrained_model_name_or_path, **kwargs)) File "C:\Users\User.conda\envs\Proj\lib\site-packages\transformers\tokenization_utils_base.py", line 1854, in from_pretrained return cls._from_pretrained( File "C:\Users\User.conda\envs\Proj\lib\site-packages\transformers\tokenization_utils_base.py", line 2073, in _from_pretrained raise ValueError( ValueError: Non-consecutive added token '<|0.02|>' found. Should have index 50365 but has index 50366 in saved vocabulary.

Try to upgrade the transformers library.

Thanks, that worked 👍

[salahzoubi]

Do you intend on adding batch_transcribe? Because it seems like the whisper v3 on HF pipes allows transcribing multiple audio files at once... would be an amazing feature!

[ILG2021]

Wish the maintainer merge soon. It delays so long.

[EIIisD]

please please please

[nguyendc-systran]

Thanks for your PR, it seems a duplicate of this #578
So, I'm closing it, feel free to reopen if needed.