Skip to content
View original post on X: RadixArkOfficial
Published
36/100

RadixArk praises Sasha's Lean-verified Rust tokenizer for agentic RL

AIRadixArk praises Sasha's work formally verifying properties of a Rust tokenizer in Lean, which extends Miles' token-in-token-out idea beyond testing that training sees the exact tokens the model generated. Sasha's prototype, posted by Sasha Rush (@srush_nlp), addresses training-inference mismatch in RL by verifying the tokenizer pipeline rather than relying on messages built in Jinja.
RadixArkVerified on XOfficial
@radixark
View on X

Love seeing ideas from Miles' Token-In-Token-Out work inspire new research!

Instead of just testing that training sees the exact tokens the model generated, Sasha takes it a step further, formally verifying key properties of a Rust tokenizer in Lean. That's the kind of rigor agentic RL needs.

Fascinating work, @srush_nlp!

Sasha Rush@srush_nlp
Lean Verified Tokenizers (https://srush.github.io/verified-tokenizer/) Training-inference mismatch kills RL runs, but yet we build messages in Jinja? Prototypes a Lean-verified Rust agent pipeline.
View quoted post on X
Source details

RadixArk · x.comPublished

Original title: Love seeing ideas from Miles' Token-In-Token-Out work inspire new research!

Topics

Safety & alignmentPapers & researchEvals & benchmarks