Just too slow unusable - API - RTX 3060 12 GB

[FurkanGozukara]

Anyway to speed up during API generation?

I mean 10 sec audio is literally taking 4 minutes

It is running on GPU RTX 3060 (12 GB vram), Core i7 10700 F @ 4.7 GHZ and 64 GB ram memory

on the other hand this one is taking 5 seconds literally : https://huggingface.co/spaces/rendchevi/nix-tts

[FurkanGozukara]

@neonbjb thank you so much for all answers

I made it work but it is just too slow

This is my api code

import os
from pickle import TRUE
import utils
import utils.audio
import voices
import api
import torchaudio
from utils.audio import load_audio, load_voice, load_voices

path = "D:/86 se courses youtube kanali/tortoise-tts/tortoise/voices/myvoices"
clips_paths = os.listdir(path)
text="So if you use the latest framework of Microsoft dot NET platform it is core and it supports Mac OS and Linux as well for console applications."

reference_clips = [utils.audio.load_audio(path+'/'+p, 22050) for p in clips_paths]

text2="However dot NET Core console supports C Sharp, Linux, Mac OS, Windows and console.";

CUSTOM_VOICE_NAME="jordan"
_preset="fast"
#_voice_samples, _conditioning_latents = load_voice(CUSTOM_VOICE_NAME)

_voice_samples, _conditioning_latents = load_voice(CUSTOM_VOICE_NAME)

tts = api.TextToSpeech()

#pcm_audio = tts.tts_with_preset(text, voice_samples=reference_clips, preset='fast')

#torchaudio.save('generated.wav', pcm_audio.squeeze(0).cpu(), 24000)


#gen = tts.tts_with_preset(text, voice_samples=_voice_samples, conditioning_latents=_conditioning_latents, 
#                          preset=_preset)

mylist = {text, text2}
counter=1
for _text in mylist:
    gen = tts.tts_with_preset(_text, voice_samples=_voice_samples, conditioning_latents=_conditioning_latents, 
                          preset=_preset,use_deterministic_seed=1)

    torchaudio.save(f'generated-{CUSTOM_VOICE_NAME}{counter}.wav', gen.squeeze(0).cpu(), 24000)
    counter=counter+1

[JetRoderick]

Cool, im going to dl that, it does suffer from the ai sibilance, hopefully the package allows for trade offs.

[JetRoderick]

I had a look at the model , it's so funny once it gets past about 20 words, she sounds like a drunk.

An easy fix for a noob like me was to just split the input text into small parts using the commas and full stops and when she's had to much to drink and output to wave. She works well like this and results are quite consistent, thanks for link I think it's a really useful tool.

[FurkanGozukara]

I had a look at the model , it's so funny once it gets past about 20 words, she sounds like a drunk.

An easy fix for a noob like me was to just split the input text into small parts using the commas and full stops and when she's had to much to drink and output to wave. She works well like this and results are quite consistent, thanks for link I think it's a really useful tool.

yep. if tortoise had same speed and consistency would be amazing

[JetRoderick]

totally tortoise sounds so much smoother and the dreaded sibilance issue that seems to plague all TTS isn't there with my own personal tests and equipment. A weird quirk of lofi digital is harsh tops compared to lofi analogue that has a nice gradual smoothing on the top of freq spectrum.

I still need to test some of the settings with tortoise, i think there could be settings somewhere between ultra fast and fast that could be very useful, if i can wrangle out more consistency form one chunk to the next, perhaps setting seed? . I also think maybe shifting the presets naming might re-adjust people's expectations.

[FurkanGozukara]

@JetRoderick do you know how can I improve sound of my older recordings any idea?

example : https://youtu.be/s-FZ2pW6UF4

another example : https://youtu.be/9OO438j8jCs

[JetRoderick]

more sound damping in the room and/or close mic with pop shield. I use a MD 421-II into zoom recorder. Clear and little room noise. I had an idea to feed the model with 2 set of same vocal but one set with shifted semi tones to produce a random musicality to the spoken word.

also on the other model the nix, i cant seem to get the live model output with out serious noise issues. i'm not using jupter notebook or whatever it's called as in the example.