i14 launches Sugar Glider, LLM architectures with fixed-size KV cache

AIi14 introduces Sugar Glider, a new family of LLM architectures built for long contexts, with waitlist access at i14.ai. i14 says the architecture keeps a fixed-size KV cache as context grows and reports up to 32x more requests per server at one million tokens than its baseline model. Per the i14 post, memory size can be chosen at deployment without retraining.

elvis

@omarsar0

View on X

I'm bullish on new architectures for long-running agents.

But long context needs to be practical to serve.

Sugar Glider is a new family of LLM architectures from @i14labs built for long contexts.

It keeps a fixed-size KV cache as context grows. At one million tokens, i14 reports up to 32x more requests per server than its baseline model.

What does this mean in practice?

Your agents could keep much longer histories without constantly trimming or summarizing them.

And you can choose that memory size at deployment without retraining.

Tom Mcleod@todomcl

Frontier performance at a fraction of the compute. Today at @i14labs we’re launching Sugar Glider, our new family of LLM architectures! Join the waitlist for early access: http://i14.ai Sugar Glider delivers a new Pareto frontier: extreme memory and compute efficiency whilst maintaining frontier performance, even for large models (tested up to ~1T params). O(1) per token inference → ✅ Fixed size KV cache → ✅ Frontier long-context retrieval quality → ✅ 1 billion token long context retrieval → ☑️** (**100% -> single needle @ 1B-token context, 90.5% multikey 1 @ 1B-token context.) How did we do it? Turns out a few decades in the trenches of human memory and language model research is a pretty handy starting point to solve the “intelligence vs efficiency vs memory” conundrum. ... and just for something extra, we’ve been converting some of the most powerful and popular open-weight models to O(1) inference and fixed memory, with each conversion taking just hours :) Tune in via our waitlist for early model access http://i14.ai More results dropping soon!

View quoted post on X

Source details

elvis · x.comPublished Nov 10, 8:11 AM ET · added here 8:33 AM ET

Original title: I'm bullish on new architectures for long-running agents.

i14 launches Sugar Glider, LLM architectures with fixed-size KV cache

Topics