Provided sound format makes difference. So which one is the best one?

[FurkanGozukara]

I have transcribed my newest with with directly providing the MKV file and the MP3 sound file extracted by FFMPEG

There are differences between 2 transcribe. In some cases MKV video file input better and in some cases provided MP3 file is better

So which format is best? Which approach is best?

Actually I find MP3 slightly better

[ArtyomZemlyak]

I think most appropriate format is pcm_s16le WAV mono.
You can convert any video/audio file to this format using ffmpeg.

If you interesting, python code:

data = input_audio_or_video

if type(data) == io.BytesIO:
    data = data.getvalue()

cmd = ["ffmpeg"]

if time_start:
    cmd.extend(["-ss", get_hms(time_start)])

    if time_end and time_end > time_start:
        cmd.extend(["-t", get_hms(time_end - time_start)])
cmd.extend(
    [
        "-i",
        "-",
        "-acodec",
        "pcm_s16le",
        "-ar",
        "16000",
        "-ac",
        "1",
        "pipe:.wav",
    ]
)

proc = subprocess.Popen(
    cmd,
    stdout=subprocess.PIPE,
    stdin=subprocess.PIPE,
    bufsize=len(data),
    shell=False,
)
out = proc.communicate(input=data, timeout=None)[0]

proc.wait()

converted_audio = io.BytesIO(out)

[jongwook]

load_audio() does precisely that and reads the audio into memory instead of saving to a WAV file. So no need to use the disk space!

[FurkanGozukara]

@jongwook but it produces different results. When I give MP3 formatted audio or directly MKV file.

[FurkanGozukara]

Its can be like:

ffmpeg -i name_of_input_file.mp3 -acodec pcm_s16le -ar 16000 -ac 1 name_of_output_file.wav

This generated same results as directly providing video file itself

[jongwook]

Yeah that's probably because you're doing lossy compression when saving the MP3, and the model can produce different outputs because it'll contain a different waveform. With WAV you didn't see this because it's a lossless format. Individual examples might be better or worse between MP3 or MKV (which likely contained AAC), but since MP3 re-encoding inevitably degrades the audio, I expect worse accuracy on average.

[FurkanGozukara]

Yeah that's probably because you're doing lossy compression when saving the MP3, and the model can produce different outputs because it'll contain a different waveform. With WAV you didn't see this because it's a lossless format. Individual examples might be better or worse between MP3 or MKV (which likely contained AAC), but since MP3 re-encoding inevitably degrades the audio, I expect worse accuracy on average.

I see makes sense