Latent Semantic Memory: 16 memory tokens match or beat full-context inference on FactKG

A paper proposes Latent Semantic Memory, which encodes documents once into embeddings and writes 16 query-relevant memory tokens into a frozen LLM's input space for each query.

Per Elvis Saravia's summary of the paper, on FactKG these 16 tokens score 84.33% against 80.43% for full-context inference, while using 108x fewer tokens for the source documents.

Saravia also reports that the method beats Mem0 and MemoRAG on five of eight benchmarks, and that in TextWorld, using the agent's own history as the collection raises agent success from 18.6% to 24.2%. The source is a social media post summarizing the paper, not the paper itself.