# Zyphra introduces Hierarchical Attention, a subquadratic attention method

> Original title: Today we’re introducing Zyphra Hierarchical Attention (ZHA), a path to true subquadratic attention.

- Source: Zyphra
- Published: 2026-11-02T22:04:52.000Z
- Bad Influence: https://badinfluence.news/items/bco5rvg5mce4rdfipcc5lse27
- Original: https://x.com/ZyphraAI/status/2117377426242367669

## Summary

Zyphra introduces Zyphra Hierarchical Attention (ZHA), which searches a learned tree of compressed summaries to approach subquadratic attention. The company projects speedups of 35× over dense attention and 5× over MSA at 10M tokens, rising to about 400× and 55× at 1B tokens.

## Post

Today we’re introducing Zyphra Hierarchical Attention (ZHA), a path to true subquadratic attention.

ZHA searches a learned tree of compressed summaries, with projected speedups of 35× over dense attention and 5× over MSA at 10M tokens, increasing to ~400× and ~55× at 1B. 🧵

![](https://badinfluence.news/api/img-proxy?u=https%3A%2F%2Fpbs.twimg.com%2Fmedia%2FHWJriaEaQAADkH8.jpg&mode=thumb&exp=1793923200&sig=6e111ed5530f2019)