Voice clone quality?

[Deadstarrr]

Has anyone been able to get a decent result out of the voice cloning? I'm wondering if I don't have the settings right. It either doesn't sound like me, or it's a different gender. My training voice clip was a little over a minute.

If someone has been able to generate a good quality voice clone, what were the steps and what settings are you using when you click generate?

[gkucsko]

we don't support custom voice cloning, or am i misunderstanding?

[p4vv37]

Are you planning to? Would it be possible, to train a custom voice? Does the model support, or can it support emotions?

[p4vv37]

Maybe I should create a separate thread for those questions?

[DDBE12]

Seems like they changed the code/description when I asked about the fact it did say custom voice cloning. That basically renders the entire project useless for me overnight now that they're canceling it.

[Deadstarrr]

It is a shame. I run a youtube channel and it would be awesome to be able to dub in lines that didn't record properly with my own voice or to be able to quickly fix a mistake I discovered while editing.

[DDBE12]

Anybody who wanted voice cloning, you should try voice.ai . We even have our own Discord support server. It's a proprietary Windows app (though devs are working on MacOS and mobile versions), but most of the functions are entirely free. Bonus features to subcribe for only concern the length of files you can convert in record mode, and a higher mp3 quality on the output.

Keep in mind, it's not TTS like Bark, it's STS: You talk into the app and it converts your voice into a voice of your choice in real time (thing is, you need a rather powerful GPU for live mode). In record mode, you can also do the same with pre-recorded audio files, as in, you can convert them as well, and you don't even need a GPU for record mode.

You can even make your own custom voice clones, provided you have enough good, clean source material (it takes a minimum of c. 2 hours for the voices to start sounding really good).

[abhiprojectz]

Don't worry guys, i have found a solution, The only thing that prevents from custom voice cloning is getting semantic tokens from a audio file.

Recap:

The project already supports getting those from text, as generate_text_semantics as far as i reversed the project, it seems hubert model from some supports from AudioLM project you can get Bark supported semantics tokens from there.

But, the catch is pretrained weights are only in english and portuguese and russian.

So, technically for others languages you need to train the hubert model. But then there's a lot of work and again the quality of cloned voice is not up to the mark.

The solutions

So, its simple i found a way to get high quality custom clone in all supported languages, just extract semantics embedding from whisper just like the fellow people's in SpearTTS approaches.

Moreover just to create a patch for any wav2vec model and extract speaker embeddings then convert them to Bark compatible format that's all.

If anyone is interested, Custom voice cloning in Indic languages, i have posted.