Skip to content
Trending storyDeveloping

Google Research proposes a 'sleep' phase to help LLMs learn continually without forgetting

2 articles2 sourcessince Oct 27Last article

Overview

AISummary of 2 articles

Google Research introduces a sleep framework for large language models that alternates active interaction with offline memory consolidation, so models can keep new knowledge without catastrophic forgetting.

The company's blog post, announced on X on October 27, 2026, says the method consolidates short-term context into stable long-term parameters, and that it aims to reduce static knowledge cutoffs.

The post reports results on unseen-language translation, long-context retrieval up to 10M tokens, factual knowledge incorporation, and the AIME 24/25 and HMMT 25 math benchmarks. It says the sleep-enhanced base model outperformed SFT, GRPO, and OPSD baselines on the math benchmarks.

Written by AI from the articles below · updated Oct 27, 2:37 PM ET

Check the sources:

Latest developmentOct 27, 1:40 PM ET
Google Research adds a "sleep" phase to continual learning for LLMs

Article timeline

The articles in this story. Times are ET.

Oct 27
  1. Google ResearchOfficial
    Google Research adds a "sleep" phase to continual learning for LLMs

    AIGoogle Research says it has developed a "sleep" phase for continual learning that lets LLMs keep integrating new facts and improve reasoning. The method aims to reduce catastrophic forgetting and static knowledge cutoffs. The company describes the work in a blog post.

    Overview of our sleep framework: Input streams from the active phase are stored in short-term memory before entering the sleep cycle, which bifurcates into Consolidation (transferring fragile short-term tokens into expanded long-term parametric memory using low-rank structures and MoE) and Dreaming (autonomous self-improvement via synthetic routing with random experts and RL refinement).
  2. Google ResearchOfficialPick
    Google Research proposes a sleep framework to help LLMs retain knowledge

    AIGoogle Research describes a sleep framework that alternates active interaction with offline memory consolidation so LLMs can keep new knowledge without catastrophic forgetting. The post reports results on unseen-language translation, long-context retrieval up to 10M tokens, factual knowledge incorporation, and AIME 24/25 and HMMT 25 math benchmarks, where the sleep-enhanced base model outperformed SFT, GRPO, and OPSD baselines.

Heat Trend

Current comparable heat
6
Peak comparable heat
10Oct 27, 2:00 PM ET
Change in past 24 hours
–

The trend compares only continuously observed participants, so its scope may be smaller than the current heat total. Hover or tap for hourly heat; use arrow keys with a keyboard.