# Zyphra suggests ultra-long contexts may need subquadratic attention

> Original title: This suggests a deeper idea: ultra-long contexts may require subquadratic attention.

- Source: Zyphra
- Published: 2026-11-02T22:04:57.000Z
- Bad Influence: https://badinfluence.news/items/ylemj1fa7jbnwil3snf2rgrym
- Original: https://x.com/ZyphraAI/status/2117377448358850788

## Summary

Zyphra says ultra-long contexts may require subquadratic attention, arguing that attending to everything buries the signal in noise while overly sparse attention loses it. The company suggests hierarchical attention may be essential for attention to work at all, not just for efficiency.

## Post

This suggests a deeper idea: ultra-long contexts may require subquadratic attention.

Attend to everything, and the signal gets lost in noise. Too sparse, and we lose the signal.

Hierarchical attention may be essential not just for efficiency, but for attention to work at all.