Shanghai AI Lab open-sources Mobius, a Transformer alternative claiming 4x faster reasoning
Original title🔥Shanghai AI Lab open-sourced Mobius, whose transformation from Transformer bears a striking parallel to the RNN → Transformer shift:
AISummary
Shanghai AI Lab open-sourced Mobius, an architecture its authors compare to the RNN-to-Transformer shift in both token and knowledge dimensions.
Against Transformers, the post claims about 4x faster reasoning, the same MMLU score with 40% less data, and 2x better compositional generalization.
Mobius is supported by XTuner, LMDeploy, vLLM, and SGLang, and its experimental setup and training pipeline will be released later.
Source: Intern Large Models · x.comPublished · added here