Rust metal features not working in Mac

[venkyyuvy]

Backend impacted

The Rust implementation

Operating system

Mac OS X

Hardware

Metal with MLX

Description

➜ rust git:(main) ✗ cargo run --features metal --bin moshi-backend -r -- --config moshi-backend/config-q8.json standalone
warning: profiles for the non root package will be ignored, specify profiles at the workspace root:
package: /Users/venkat/Documents/moshi/rust/moshi-backend/Cargo.toml
workspace: /Users/venkat/Documents/moshi/rust/Cargo.toml
Finished release profile [optimized + debuginfo] target(s) in 0.16s
Running target/release/moshi-backend --config moshi-backend/config-q8.json standalone
2025-01-18T05:39:15.791503Z INFO moshi_backend: build_info=BuildInfo { build_timestamp: "2025-01-18T05:35:51.560888000Z", build_date: "2025-01-18", git_branch: "main", git_timestamp: "2025-01-14T12:02:31.000000000+01:00", git_date: "2025-01-14", git_hash: "77f9215629f1ff7914f0a3bb82508824a6436413", git_describe: "77f9215", rustc_host_triple: "aarch64-apple-darwin", rustc_version: "1.81.0", cargo_target_triple: "aarch64-apple-darwin" }
2025-01-18T05:39:15.791558Z INFO moshi_backend: starting process with pid 15606
2025-01-18T05:39:15.791651Z INFO hf_hub: Token file not found "/Users/venkat/.cache/huggingface/token"
model.q8.gguf [00:07:31] [███████████████████████████████████████████████████████████████████████████████████████████████████████████████] 7.61 GiB/7.61 GiB 17.24 MiB/s (0s)
..c8d8-checkpoint125.safetensors [00:00:38] [█████████████████████████████████████████████████████████████████████████████████████████] 366.83 MiB/366.83 MiB 9.63 MiB/s (0s)
tokenizer_spm_32k_3.model [00:00:01] [██████████████████████████████████████████████████████████████████████████████████████████████] 539.82 KiB/539.82 KiB 305.76 KiB/s (0s)2025-01-18T05:47:31.123527Z INFO hf_hub: Token file not found "/Users/venkat/.cache/huggingface/token"
dist.tgz [00:00:00] [███████████████████████████████████████████████████████████████████████████████████████████████████████████████] 575.53 KiB/575.53 KiB 970.31 KiB/s (0s)2025-01-18T05:47:53.835292Z INFO moshi_backend::standalone: warming up the model dtype=F32 device=Metal(MetalDevice(DeviceId(1)))
2025-01-18T05:48:52.474201Z INFO moshi_backend::standalone: model is ready to roll!
2025-01-18T05:48:52.475776Z INFO moshi_backend::standalone: serving static dir /Users/venkat/.cache/huggingface/hub/models--kyutai--moshi-artifacts/snapshots/8481e95f73827e4e70ac7311c12b0be099276182/dist
2025-01-18T05:48:52.478792Z INFO moshi_backend::standalone: standalone worker listening on https://0.0.0.0:8998
2025-01-18T05:51:59.755022Z INFO moshi_backend::standalone: received connection addr=127.0.0.1:49360
2025-01-18T05:51:59.755147Z INFO moshi_backend::stream_both: accepted websocket connection
2025-01-18T05:51:59.757206Z INFO moshi_backend::stream_both: starting streaming
2025-01-18T05:51:59.758849Z INFO moshi_backend::stream_both: processing loop
2025-01-18T05:52:09.765055Z INFO moshi_backend::stream_both: socket closed
2025-01-18T05:52:09.765125Z ERROR moshi_backend::stream_both: loop1 ended r=Ok(Ok(()))
2025-01-18T05:52:09.765148Z INFO moshi_backend::stream_both: decoder closed
2025-01-18T05:52:23.031768Z INFO moshi_backend::standalone: received connection addr=127.0.0.1:49366
2025-01-18T05:52:23.031812Z INFO moshi_backend::stream_both: accepted websocket connection
2025-01-18T05:52:23.031886Z INFO moshi_backend::stream_both: starting streaming
2025-01-18T05:52:23.032403Z INFO moshi_backend::stream_both: processing loop
2025-01-18T05:52:26.267085Z INFO moshi_backend::standalone: received connection addr=127.0.0.1:49368
2025-01-18T05:52:26.267130Z INFO moshi_backend::stream_both: accepted websocket connection
2025-01-18T05:52:26.267154Z INFO moshi_backend::stream_both: starting streaming
2025-01-18T05:52:26.267572Z INFO moshi_backend::stream_both: processing loop
[1] 15606 segmentation fault cargo run --features metal --bin moshi-backend -r -- --config standalone

Extra information

After receiving audio through Chrome browser, I get the ERROR message in the logs.

ERROR moshi_backend::stream_both: loop1 ended r=Ok(Ok(()))

Environment

Fill in the following information on your system.

• Operating system version:

If the backend impacted is PyTorch:

• Python version:
• PyTorch version:
• CUDA version (run python -c 'import torch; print(torch.version.cuda)'):
• GPU model and memory:

If the backend is MLX:

• Mac model: