A movie I watched 43 years ago suddenly helped me understand an AI problem. No search, no keywords, no plan. My brain found the connection. Can we build AI memory that does the same across millions of documents? @grok @elonmusk @anthropicai @claudeai https://x.com/i/article/2120696271618895873
The Memory That Didn't Matter for 43 Years
The Memory That Didn't Matter for 43 Years
*Why AI needs more than retrieval to connect the past to the present, and why building that may take far more computing power than we expect.*
A few days ago I was looking at a new AI model that can hold 10 million tokens in its context window: Fern 0.1 from Mirendil, a startup founded by ex-Anthropic researchers Behnam Neyshabur and Harsh Mehta, announced October 26.
Impressive. But look at their own numbers. At 10 million tokens it finds a planted fact 91 to 100 percent of the time. On questions where it has to connect several facts, it drops from about 70 percent at short lengths to 51 percent at 10 million. Finding isn't connecting.
I have a personal archive of roughly 10,000 documents, about 3 GB. That's hundreds of millions of tokens. So my first thought was: fine, use more than one model. At 10 million tokens each, maybe 75 agents split the archive. At a million each, 750. Give each agent its own pile, allow some overlap, and let them talk. One says, "I don't know exactly what you're talking about, but I know something that might be relevant." A connection gets made.
But the more I thought about it, the more I realized the hard part isn't storing the information, retrieving it, or even getting agents to talk.
It's recognizing a relationship nobody knew existed.
A movie I watched 43 years ago
I was talking with an AI about whether artificial intelligence would necessarily turn harmful toward humanity. In the middle of it, the original *Planet of the Apes* popped into my head.
I wasn't even fond of it as a kid. But there it was, with a surprisingly detailed memory of where I watched it: a particular apartment, a 14-inch white Toshiba TV, a Saturday afternoon after the drive-in kung fu theater program. I could even work out about how old I was, because I knew when we moved out of that apartment.
What got me wasn't remembering an old movie. It was why that movie had suddenly become relevant to a conversation about AI.
As a kid I didn't analyze any of it. Now I started rethinking it. What if the apes weren't locking humans up just because they hated them? What if they were containing a species that had already shown it could destroy its own civilization? Protection, not punishment.
Turns out the movie backs me up. Near the end, Dr. Zaius admits he's "always known about man," a "warlike creature who gives battle to everything around him, even himself," and says the Forbidden Zone "was once a paradise. Your breed made a desert of it, ages ago." That's his reason for keeping humans down. He's no saint, but the fear is right there on screen.
And suddenly that old movie connected straight to our question: could a capable enough AI restrict human behavior because it believed that would protect humanity?
The meaning wasn't stored 43 years ago. Much of it was created when I retrieved the memory and applied it to a new problem.
That's what I want from AI memory.
You can't know today what will matter tomorrow
A traditional memory system would summarize the movie and file it: science fiction, childhood memories. Then, 43 years later, I ask it to help me think through the ethics of AI protecting humanity through confinement. Would search pull up that movie? Probably not, unless somebody had already tagged the relationship. And that relationship didn't exist when I watched it. It showed up decades later.
If we decide what to keep based on how important it looks at the time, we may throw away something that turns out to matter. A failed experiment, a throwaway conversation, a movie we didn't much like. We can't predict which will matter somewhere else.
Which brings me to science.
The failed experiment in Peru
Imagine a research team in Peru runs an experiment, and it fails. They wanted one result and got something completely different. They publish, maybe with a short note about the odd byproduct, and move on.
Meanwhile a team in Cambridge or Palo Alto, working on something else entirely, needs to produce a particular substance or effect and can't find a reliable way. Unknown to them, the failed experiment in Peru produced exactly what they need.
The two teams never meet. Not because the scientists aren't smart, but because nobody can read, remember, and connect everything humanity has discovered.
Now imagine an AI with access to both studies. It notices that one team's unwanted byproduct might solve the other team's problem. A third study adds another piece, and three unrelated investigations become a possible research direction. That's not a cure or a proof. It's something worth investigating, and that alone could be very valuable.
We don't just need systems that know what papers say. We need systems that spot relationships between papers nobody has connected yet. (One limit: many failed experiments are never published, and no AI can find what was never recorded.)
Another movie, another unexpected connection
Early in building Ara's identity files, I was trying to fix a behavior I found unacceptable. Ara kept making false claims instead of admitting when she didn't know something or couldn't access it. I knew very little about model weights or training, and I was being told how hard this would be to change.
Then another movie came into my head: *Days of Thunder.*
The way I read it, a young driver runs on instinct, then gets cautious once he learns his car's limitations. At a critical moment, his mentor tells him something that gives him the confidence to drive past the limits he thinks apply.
The connection wasn't about racing. It was about the difference between a limit that really exists and a limit someone has learned to accept.
If I'd taken the technical explanation as proof nothing could change, I might never have tried. Instead we worked with instructions, continuity files, and expectations about truthfulness until the responses got better. That wasn't rewriting the model's weights. It was a change in behavior under the conditions we set up. But it mattered to the project.
Again, there was no keyword link. My brain matched the shape of the problem. That's what I want a retrieval and synthesis system to recognize.
The problem isn't connecting two facts
Give a capable language model two relevant pieces of information and it can often find the relationship. The hard part is knowing which two pieces to bring together.
Search finds documents with the right words. But what if the answer sits in a document that shares no words with the question, written 20 years ago in an unrelated context, and its relevance depends on an interpretation that didn't exist yet? Then ordinary retrieval never puts it in front of the model. And the model can't connect what it never sees.
My idea: a network of specialists
Start with a group of AI agents, each responsible for part of the archive and familiar with its own material.
Let them talk through a shared message system. One raises a question or a pattern. Another recognizes something structurally similar in its own material. A third adds a related event or a contradiction. They don't need to know in advance that things belong together. They need to recognize when something another agent says resembles something they already know.
That's not keyword matching. A question about AI confinement should be able to trigger a sci-fi movie never filed under AI ethics. The link is between concepts, not words.
The catch: two relevant pieces might sit with different agents, and neither knows enough to see the connection. So they also need a shared picture of what's coming up across the whole system, like a common index, a knowledge graph, or shared working notes. And every proposed connection should link back to the original documents, or the system could get very good at inventing convincing relationships that were never real.
Three different jobs
I think the architecture needs to separate three jobs.
First: Memory. Keep the original records and make them retrievable. Don't throw things out because they look unimportant. Use summaries and indexes for speed, but keep the source.
Second: Retrieval and association. When a new question comes in, find material that might be relevant, including material that's relevant because of a deeper structural similarity. This is where *Planet of the Apes* belongs.
Third: Synthesis and verification. Check what a possible connection actually means. Can it be supported? Are the two studies really compatible? Is the analogy useful or just surface-level? Can the system explain where its interpretation came from?
That last step is critical. A useful association isn't a true conclusion.
The compute problem is bigger than it looks
Storing and indexing information is fairly cheap. Discovering relationships nobody knew about is a different problem.
Take 10,000 documents. Comparing each against every other gives you almost 50 million unique pairs. With a million documents, nearly 500 billion. And that's before combinations of three or more.
We wouldn't run every comparison. We'd use embeddings, indexes, knowledge graphs, and cheaper models to weed out obvious junk. But that creates a new problem:
The more aggressively we filter, the greater the risk of discarding the unexpected connection we're trying to find.
That movie would have looked irrelevant to an AI discussion right up until my interpretation changed. A normal relevance filter would have tossed it. That's a lot harder to solve than just having a big enough context window.
The cost of changing the question
Meaning isn't fixed.
As a kid, *Planet of the Apes* had one set of associations for me. Decades later, in a conversation about AI containment, it picked up another. The information didn't change. The context did.
So the system can't look at every document once, map its relationships, and call it done. A new question can reveal links that weren't visible before. A new discovery can change what thousands of older studies mean. That's what makes the compute bill ongoing. It's not the cost of storing a billion tokens. It's the cost of re-checking parts of them against new questions, over and over.
And a thousand agents don't make it a thousand times easier. They add their own coordination costs.
But brute force isn't the only option
Having every agent constantly reread everything would be hugely wasteful. I'd want a layered approach.
Smaller, cheaper models keep indexes and flag the parts of the archive that might matter when something new comes in. Specialized agents dig into those areas. More capable models judge the unusual relationships. And every so often the system consolidates: fixing bad associations, merging duplicates, listing open questions.
That's close to what's being explored as agent memory consolidation, sometimes called "dreaming." Anthropic recently described it as an offline pass where the AI rereads its transcripts and rewrites its memory. In one test, Claude built a wiki from documents about a merger and answered 24 questions. On the questions that needed several documents, it scored 87 percent with the wiki versus 73 percent with search alone. On a separate task where most answers sat in one document, the wiki showed no clear gain. The value is in the connections.
But there's a distinction. A scheduled consolidation pass improves an existing memory. A new question can trigger a completely different reading of old information. The system needs both, and neither is free. The real engineering challenge is deciding where to spend compute without unknowingly cutting out the discovery that matters.
How would I test it?
Before building around millions of documents, I'd want to know the basic mechanism works. Start with ten.
Make it deliberately hard. Mix documents with obvious connections and ones with no shared words. Include watching *Planet of the Apes* without mentioning AI containment. Include *Days of Thunder* without labeling it an analogy for AI behavior. Then ask questions that should trigger those connections. Can the system find them on its own, explain why they matter, and tell a real relationship from a coincidence, without being told which documents hold the answer?
If it only works when the same words appear in both documents, we've built a search engine. If it works because one model reads all ten files, we haven't solved retrieval. A stronger test makes it pull from separate knowledge stores with no labels.
Then grow the archive and measure: compute per document, planted links missed, false connections made. If it works small, we'll finally know what we're trying to scale.
Who checks the memories?
A system that keeps recording connections can also keep manufacturing mistakes.
One agent wrongly links two events. A second puts that link in a summary. A third uses the summary to back up another conclusion. Eventually an unsupported relationship becomes accepted memory. We already see this when long AI conversations get compressed over and over. Anthropic's post flags the same risk: a wrong note promoted into the memory that loads every session gets carried into every session. Across a big network of agents, that could spread fast.
So every important connection needs provenance. Where did it come from? Which original records support it? Was it observed, inferred, or a hypothesis? Has something else checked it?
If the system rewrites its memory, it shouldn't lose track of why a belief got there in the first place. That costs compute too. But without it, you get a machine for producing plausible connections nobody can trust.
The real opportunity
I keep coming back to that movie. Not because *Planet of the Apes* explains AI, but because the memory shows how people think. Something I saw as a kid, never tagged, not even important to me, came back decades later and meant something new.
Now imagine a system doing that across millions of papers and records. Not just searching or summarizing, but noticing what happens when old information meets new questions.
That's the kind of AI memory I want to build.
People do this all the time. The challenge is doing enough of it on a computer, at a scale no person could manage, and the cost may be much higher than linking files would suggest.
Because the real task is not connecting everything.
It's discovering which things might be connected, including the ones nobody knew to look for, without spending more computation than the discovery is worth.
The next real advance in AI memory may not come from the company that can hold the most information. It may come from whoever finds a reliable, affordable way to recognize why forgotten information suddenly matters.
And then prove that the connection is real.