User provided device_type of 'cuda', but CUDA is not available. Disabling

[Lohnii]

When trying:
python tortoise/do_tts.py --text "I'm going to speak this" --voice random --preset fast

I get this warning:

Generating autoregressive samples..
C:\Users\User\miniconda3\envs\tortoise\lib\site-packages\torch\amp\autocast_mode.py:204: UserWarning: User provided device_type of 'cuda', but CUDA is not available. Disabling
warnings.warn('User provided device_type of 'cuda', but CUDA is not available. Disabling')

Is there a way to fix this? It makes it take over 40 minutes to generate the samples

I have installed the cuda toolkit (ver 11) and restarted my computern and it's still happening.
My graphics card is an RTX 3060 Ti

This is my 6th try installing tortoise and it's the first time I've gotten this far, I would like to try fixing this without reinstalling

[Lohnii]

I was able to fix this by running

conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia

[Jidis83]

It's a train wreck here. The re-installs surprised me by working perfectly (with functioning CUDA), and after processing for ages, it throws a CUDA out of memory error. Tried the fixes for that to no avail, then installed a couple CUDA memory optimization packages (no good). I gave up and tried to get the CPU one back up and got two new errors, so I called it quits.

Sorry to hijack this with sob stories, and glad you finally got yours working. I'll see what I can get tonight then figure out who to ask if I still can't get anywhere.

PS- Funny, but I set this Lubuntu system up because it sounded like it was developed on Linux and I'd have less to do to make it work.

[islewis]

I was having the same issue, but had already tried to run conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia.

The fix had to do with getting the correct pytorch-cuda version- I'm running cuda 12.3, and so the version that ended up working for me was pytorch-cuda 12.1:
conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia

If anyone else is running into this same issue, try playing around with that version.

[robkau]

Thanks @islewis that fixed the issue for me on Windows miniconda environment. These are the steps I used:

Do the Conda installation steps from here and open the Anaconda Powershell Prompt from Start Menu.

Check cuda version is 12.3:

nvcc --version

Cuda compilation tools, release 12.3, V12.3.103

Reinstall pytorch with your example command:

conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia


The following packages will be DOWNGRADED:

  pytorch                                 2.1.1-py3.9_cpu_0 --> 2.1.1-py3.9_cuda12.1_cudnn8_0
  torchaudio                                 2.1.1-py39_cpu --> 2.1.1-py39_cu121
  torchvision                               0.16.1-py39_cpu --> 0.16.1-py39_cu121

Now CUDA works! From 3 minute generation to 5 second generation.

python .\tortoise\do_tts.py --output_path results --preset ultra_fast --voice geralt --text "time flies like an arrow."

miniconda3\envs\tortoise\lib\site-packages\torch\nn\utils\weight_norm.py:30: UserWarning: torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.
  warnings.warn("torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.")

Generating autoregressive samples..
100%|███████████████████████████████████████████████████████████████████████████████████| 1/1 [00:03<00:00,  3.55s/it]
Computing best candidates using CLVP
100%|███████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.38it/s]
Transforming autoregressive outputs into audio..
100%|█████████████████████████████████████████████████████████████████████████████████| 30/30 [00:01<00:00, 25.94it/s]
100%|█████████████████████████████████████████████████████████████████████████████████| 30/30 [00:01<00:00, 26.29it/s]
100%|█████████████████████████████████████████████████████████████████████████████████| 30/30 [00:00<00:00, 31.12it/s]

[Digvijay1999]

My CUDA version is 12.4 and latest CUDA that i can see here pytorch is 12.1 what to do in this case ?

[Digvijay1999]

conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia
it worked for me too even with CUDA 12.4