Zyphra explains why top-p attention selection may be subquadratic

But why should top-p selection work, and is this actually subquadratic?

In dense attention models, it turns out that the top-p set naturally scales like a sublinear power law.

This suggests that attention is already intrinsically sparse. ZHA simply uncovers this structure.