Possibility of a DirectML fork?

[ArchAngelAries]

Just wondering if it could be possible to implement directml, using cpu only isn't very viable, especially if you want to train your own voice. With a directml backend option AMD users might be able to use this as well. Just an idea.

[fakerybakery]

I'm using it on Linux and AMD works amazingly. But ROCm in PyTorch doesn't support Windows. Have you considered setting up a Linux environment alongside your Windows OS?

[Chapoly1305]

@ArchAngelAries @fakerybakery
For PoC, I have created a fork with some modifications inspired by #550 and tested on my home pc with 6800XT. I have observed the usage of my GPU and I think it works, but I don't have a CUDA GPU so I can't provide a benchmark.

https://github.com/Chapoly1305/tortoise-tts-directml

To give it a try, clone this fork,
pip install torch-directml
, then
python tortoise/do_tts.py --voice random --preset fast --text "we need some information about how the state of the world is reflected in the agent’s percepts"

Walkthrough demo can be found at https://youtu.be/la2ZkNfDNQo
And audio files generated during the demo generated audios.zip

I am new to torch community and feedbacks are appreciated.

[tondeaf]

ERROR: Could not find a version that satisfies the requirement torch-directml (from versions: none)
ERROR: No matching distribution found for torch-directml

[Chapoly1305]

Thanks for the message, you may refer to https://github.com/Chapoly1305/tortoise-tts-directml.
Give Python 3.10 a try, newer version of python (e.g., 3.12) were tested incompatible due to DirectML has not yet been updated. The pip says DirectML supports between 3.7 to 3.10 inclusively.

[eglische]

I was trying this fork on my setup (i never used python before, so i may jingled up something).
however, after dealing with some depedencies and getting rust etc. to work, i could clone your env and set it active, installed the dependencies you mentioned in your fork and tried to run it.

the result is a error "m_device->CreateOperator(&opDesc, IID_PPV_ARGS(&op))"
there is little i can find on that, other than it's a memory allocation issue and no solution.

you might want to adjust your provided activate.bat, "set VIRTUAL_ENV=C" points your user profile.

here is the full readout:

Traceback (most recent call last): File "C:\tortoise-tts-directml-main\tortoise\do_tts.py", line 48, in <module> gen, dbg_state = tts.tts_with_preset(args.text, k=args.candidates, voice_samples=voice_samples, conditioning_latents=conditioning_latents, File "C:\tortoise-tts-directml-main\tortoise\api.py", line 373, in tts_with_preset return self.tts(text, **settings) File "C:\tortoise-tts-directml-main\tortoise\api.py", line 658, in tts best_latents = autoregressive(auto_conditioning.repeat(k, 1), text_tokens.repeat(k, 1), File "C:\Users\manager\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "C:\Users\manager\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, **kwargs) File "C:\Users\manager\AppData\Local\Programs\Python\Python310\lib\site-packages\tortoise\models\autoregressive.py", line 501, in forward text_logits, mel_logits = self.get_logits(conds, text_emb, self.text_head, mel_emb, self.mel_head, get_attns=return_attentions, return_latent=return_latent) File "C:\Users\manager\AppData\Local\Programs\Python\Python310\lib\site-packages\tortoise\models\autoregressive.py", line 428, in get_logits enc = self.final_norm(enc) File "C:\Users\manager\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "C:\Users\manager\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, **kwargs) File "C:\Users\manager\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\normalization.py", line 201, in forward return F.layer_norm( File "C:\Users\manager\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\functional.py", line 2546, in layer_norm return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled) RuntimeError: m_device->CreateOperator(&opDesc, IID_PPV_ARGS(&op))

[Chapoly1305]

Thank you for the feedback, I will look into the issue. You may temporary use --candidates 1 to workaround.