Skip to content
View original post on X: Ai2Official
Published
26/100

Ai2 shows Gated DeltaNet generalizes to sequences 10× longer than training

AIAi2 says a flexible Gated DeltaNet handled sequences about 10 times longer than its training data on StateBench's hardest level. The company also reports a new proof that Gated DeltaNet with non-negative eigenvalues can solve some periodic tasks under fixed-precision arithmetic with rounding.
Ai2Verified on XOfficial
@allen_ai
View on X

Part of a thread · earlier post

On StateBench’s hardest level, a flexible Gated DeltaNet handled sequences ~10× longer than those seen in training.

Our new proof shows that even Gated DeltaNet with non-negative eigenvalues can solve some periodic tasks under fixed-precision arithmetic with rounding.

Source details

Ai2 · x.comPublished · added here

Original title: On StateBench’s hardest level, a flexible Gated DeltaNet handled sequences ~10× longer than those seen in training.

Topics

ReasoningData & trainingPapers & research