PyTorch 2.15 adds FP8 and faster LLM decoding on Apple Silicon MPS
AIHugging Face's PyTorch team says PyTorch 2.15 adds FP8 tensor support on MPS and speeds up mixture-of-experts decoding 5 to 6.5 times. On an M3 Ultra, Qwen3.6-35B-A3B rises from 9.4 to 61.3 tokens/s.
AIWhy it matters.
The post gives measured speedups and memory figures for MPS in PyTorch 2.15, useful for judging whether local Mac inference now suits a given model.
Source details
Hugging Face Blog · huggingface.coPublished · added here
Original title: MPS in PyTorch 2.15