Add tests for Whisper static pipeline #1250
Conversation
eshiryae
force-pushed
the
b_whisper_static_tests
branch
from
a1bab59 to
48cb438
Compare
|
With transformers 4.46.3 version encoder model have different dynamic shape for input_features, fix for StaticWhisperPipeline - PR #1293 |
eshiryae
force-pushed
the
b_whisper_static_tests
branch
from
e697a90 to
3bab16c
Compare
| @pytest.mark.parametrize("model_descr", get_whisper_models_list(tiny_only=True)) | ||
| @pytest.mark.parametrize("test_sample", | ||
| [ | ||
| # *get_samples_from_dataset(language="fr", length=2), # 1/2 failed |
There was a problem hiding this comment.
What is failed? Do we have ticket for this?
There was a problem hiding this comment.
For one test (with Spanish language), there's mismatch between expected and actual output (looks like language is not detected correctly)
expected: Habritan aguas poco profundas y lo cosas.
actual_out: Habt ihr da noch was poco perfundes und lohosen?
For one test (with French language), there's an error:
RuntimeError: Check '*roi_end <= *max_dim' failed at src\inference\src\dev\make_tensor.cpp:34
I will create tickets for found fails.
eshiryae
force-pushed
the
b_whisper_static_tests
branch
from
d9fe208 to
2fdac12
Compare
eshiryae
force-pushed
the
b_whisper_static_tests
branch
from
2fdac12 to
0e11d54
Compare
| "test_sample", get_samples_from_dataset(language="de", length=3) | ||
| ) | ||
| @pytest.mark.precommit | ||
| def test_static_whisper_language_de(model_descr, test_sample): |
There was a problem hiding this comment.
What does it actually check? How it's different from test_static_whisper_autodetect?
| "test_sample", get_samples_from_dataset(language="fr", length=3) | ||
| ) | ||
| @pytest.mark.precommit | ||
| def test_static_whisper_language_fr(model_descr, test_sample): |
There was a problem hiding this comment.
Same question, how it's different from test_static_whisper_autodetect
No description provided.