Open LLMs shift from full attention to hybrid architectures, Raschka says

Sebastian Raschka says open LLM architectures moved from full attention models toward hybrids in 2025 and 2026.

He reports a 50/50 split in the second half of 2026 between hybrid models and models with no full attention layers. Raschka says this trend makes long contexts cheaper, because it requires less attention computation and less memory to retain context.