Apple shows MLX running an 816B-parameter model across four Mac Studios
Overview
Apple Machine Learning Research says MLX can run Muse Spark Open, an 816B-parameter mixture-of-experts model, across four Mac Studios with M5 Ultra chips at over 36 tokens per second.
The post describes splitting the model with pipeline and tensor parallelization over Thunderbolt RDMA and includes mlx.launch commands for setup.
The speed depends on the four-machine cluster. The RDMA driver that makes the link low-latency shipped in macOS 26.2 over Thunderbolt 5, and the new communication collectives, called jaccl, require MLX v0.30.1 or later.
The post also reports time-to-first-token and decode speedups over the M3 Ultra for Qwen 3.8 and Mistral Medium 3.5.
Written by AI from the articles below · updated Oct 15, 1:54 PM ET
Check the sources:
Article timeline
The articles in this story. Times are ET.
- Apple Machine Learning ResearchOfficialPickApple shows how to run an 816B-parameter model across Mac Studios with MLX and RDMA
AIApple's machine learning research team says MLX can run Muse Spark Open, an 816B-parameter mixture-of-experts model, across four Mac Studios with M5 Ultra at over 36 tokens per second. The post compares pipeline and tensor parallelization over Thunderbolt RDMA and gives the mlx.launch commands. It also reports TTFT and decode speedups over the M3 Ultra for Qwen 3.8 and Mistral Medium 3.5.
Heat trend
Not enough continuous observations to show a trend yet.