# Zyphra schedules attention softmax and matrix multiplies to overlap in assembly

> Original title: We write the instruction schedule in assembly so two groups of threads take turns: one computes the softmax (attention weights) while the...

- Source: Zyphra
- Published: 2026-11-06T02:36:47.000Z
- Bad Influence: https://badinfluence.news/items/tzcs18q4dcpqzaa0nvpovdymh
- Original: https://x.com/ZyphraAI/status/2118533022794531147

## Summary

Zyphra writes its attention instruction schedule in assembly so two thread groups alternate, one computing softmax while the other runs matrix multiplications. With lean P, the matrix unit is busy 86% of steady-state cycles, up from 60% using exponentials.

## Post

We write the instruction schedule in assembly so two groups of threads take turns: one computes the softmax (attention weights) while the other performs matrix multiplications.

With lean P, the matrix unit is busy for 86% of steady-state cycles, up from 60% using exponentials.

[Video](https://badinfluence.news/api/img-proxy?u=https%3A%2F%2Fpbs.twimg.com%2Famplify_video_thumb%2F2118532710801285120%2Fimg%2FruKp9FOuxJb5STGD.jpg&mode=thumb&exp=1794182400&sig=34aa5286f366373d)