Fix missing install candidate for ONNX Runtime on Apple Silicon

[dr-duplo]

This patch alters the requirements to use another ONNX runtime package which provides pre-build wheels for Apple Silicon in case of running on arm64/M1/M2 Mac.

Fixes #1505

[hbredin]

PR is failing build tests. Can you please check?

[dr-duplo]

PR is failing build tests. Can you please check?

It seems like macos-latest is on macos-12 andx86_64 arch. This clashes with onnxruntime-silicon which only provides macosx_13_0_arm64wheels. That being said, onnxruntime-gpu only provides x86_64 and amd64 wheels for linux and windows. @hbredin I wonder how you can use onnxruntime-gpu on your M1.

Would it be ok to fallback to onnxruntime in case of macosx < 13 and !arm64 ?

[stygmate]

I have tried this patch, it work for me, but my macstudio m1 GPU is not used. (i correctly set torch to mps backend)
any thougts about this ?

[dr-duplo]

I have tried this patch, it work for me, but my macstudio m1 GPU is not used. (i correctly set torch to mps backend) any thougts about this ?

Which MacOS version are you running?

[stygmate]

@dr-duplo

Which MacOS version are you running?

13.6.1

edit: i said that gpu is not used but i'm unsure. if used, I'm sure it is not fully used. i have monitored gpu usage and it's usage seems no more than in minimal usage condition (display of mac os desktop).
The two first stages of the pipeline are fully using gpu but embeddings don't seem.

[dr-duplo]

@dr-duplo

Which MacOS version are you running?
@stygmate
13.6.1

At least this proves it installs on Ventura. Did you experience just speed degradation like in pytorch/pytorch#77799 or did you prove the M1 GPU was not used?

[stygmate]

@dr-duplo when running this

import torch
import timeit

b_cpu = torch.rand((10000, 10000), device='cpu')
b_mps = torch.rand((10000, 10000), device='mps')

print('cpu', timeit.timeit(lambda: b_cpu @ b_cpu, number=100))
print('mps', timeit.timeit(lambda: b_mps @ b_mps, number=100))

i get:

cpu 100.59644954199939
mps 12.907685874999515

For pyannote, when settings pipeline to cpu:

• all stages of the pipeline are slow
• the evaluated time for embeddings process (for 1h21 of audio) is ~1h

when settings pipeline to mps:

• the two first stages of the pipeline are really fast
• the evaluated time for embeddings process is ~1h (the same as for cpu processing)

so GPU don't seem to be used.

[dr-duplo]

@stygmate
I'm not a pyannote expert, but the code tells me that only the speaker embeddings computation is gpu accelerated, but the speaker clustering, which computes the final speaker embedding afterwards, is not. Maybe this is the source for the long processing time.

[stygmate]

@dr-duplo Yesterday, I discovered that in the Pyannote code the Onnxruntime execution provider (CoreMLExecutionProvider) corresponding to the Torch mps device was not set.

I added it:

Index: pyannote/audio/pipelines/speaker_verification.py
@@ -455,6 +455,8 @@
                     },
                 )
             ]
+        elif device.type == "mps":
+            providers = ["CoreMLExecutionProvider"]
         else:
             warnings.warn(
                 f"Unsupported device type: {device.type}, falling back to CPU"

But the MPS backend does not support FFT operators and it crashes 😩🔫

[hbredin]

FYI: #1537

[hbredin]

Closing as pyannote 3.1 will get rid of this ONNX mess.

[hbredin]

Latest version no longer relies on ONNX runtime.
Please update to pyannote.audio 3.1 and pyannote/speaker-diarization-3.1 (and open new issues if needed).