Liquid AI's STAR uses evolutionary search to synthesize tailored model architectures
Original titleAutomated Foundation Model Design
AISummary
Liquid AI reports STAR, an evolutionary algorithm that synthesizes tailored neural network architectures from a numerical genome representation.
The authors say it produced hundreds of designs that outperform Transformer and hybrid architectures in quality, with smaller caches and parameter counts, and can optimize for latency on target hardware.
The full method is described in the arXiv technical report 2411.17800.
AIWhy it matters
The post explains how evolutionary search over a new architecture design space produced designs beating Transformers and hybrids, giving a concrete method for quality versus latency and memory trade-offs.
Source: Liquid AI Blog · liquid.aiPublished · added here