Zyphra tests 28 sparse-attention router geometries; deeper trees retrieve better

Zyphra tested 28 router geometries for sparse attention and reports that, at fixed compression, deeper trees often retrieve better at the same sparsity.

Its 6-level router scores 93.3% on RULER-13 at 256K context, against 96.0% for dense attention, while attending to 2.8% of decode keys.

In a separate post, Zyphra describes top-p tree routing, in which each query starts at the top of the tree and follows promising branches down to the original keys during generation. Zyphra says top-p adapts to query complexity, exploring few branches for precise lookups and more for complex queries, whereas top-K routing must discard evidence as context grows.