Should Tortoise be using the GPU?

[kacpon]

Hi, I'm new to python and I'm not sure if I understand all of this stuff so sorry if the question is on a noob-level. Just have no idea where else to ask it. I have managed to make Tortoise work, but it's painfully slow. I'm wondering if that's how it's supposed to be (a 5-word sentence needs around 3 hours to be generated), or if something's wrong with my setup. When the generation is running, my CPU runs at 100%. The GPU isn't being utilized at all. Is Tortoise supposed to run on CPU? My GPU is a GTX 1070.

[I-Have-No-Idea-What-IAmDoing]

It should use the gpu, it uses on my computer.

[Pandaily591]

You should install pytorch using the command from this page:
https://pytorch.org/get-started/locally/
Use conda if you are using a conda environment.

This installs pytorch with cuda enabled, it tells it how to use the GPU.

[Norgus]

I set up using Anaconda on windows, how can I tell if the GPU is being utilised? Task manager doesn't make it look like much is happening with the GPU.

[Lonas13]

I'm dealing with the same issue, extremely high CPU and RAM usage with little to no GPU usage

[Norgus]

I think for me it turned out to be using the GPU, but it's not as processing intensive as it is memory intensive, so it's better to check the VRAM utilisation than the GPU clock utilisation.

[DJonesCode]

I already have PyTorch installed and configured to use my GPU as its been working with other AI programs, but I'm seeing 0 GPU usage and almost no VRAM usage. Meanwhile CPU usage is at 100%.