# Zyphra explains why top-p attention selection may be subquadratic

> Original title: But why should top-p selection work, and is this actually subquadratic?

- Source: Zyphra
- Published: 2026-11-02T22:04:56.000Z
- Bad Influence: https://badinfluence.news/items/ylxh73fb4ei1eih02wvfhiy84
- Original: https://x.com/ZyphraAI/status/2117377446278484128

## Summary

Zyphra says the top-p set in dense attention models scales like a sublinear power law, suggesting attention is already intrinsically sparse. It says its ZHA method uncovers this structure, and the post asks whether top-p selection is actually subquadratic.

## Post

But why should top-p selection work, and is this actually subquadratic?

In dense attention models, it turns out that the top-p set naturally scales like a sublinear power law.

This suggests that attention is already intrinsically sparse. ZHA simply uncovers this structure.

![](https://badinfluence.news/api/img-proxy?u=https%3A%2F%2Fpbs.twimg.com%2Fmedia%2FHWJsYkvaoAASxRR.jpg&mode=thumb&exp=1793923200&sig=9b9e14d37f186b28)