Cognition adds a mid-training stage before reinforcement learning

AICognition says most gains in its latest training run came from higher-quality, more diverse reinforcement learning tasks rather than changes to its RL recipe from SWE-2. The research team audited task instructions, environment setup, and verifiers, and added several new task types.

AICognition says it added a mid-training stage before reinforcement learning for the first time. The stage trains on several trillion tokens of code, scientific articles, and blogs, with the scarcest, highest-quality data saved for last.