# Continual-ARC benchmark finds agent harnesses fail to learn, while training a 9B model succeeds

> Original title: Highly recommended read.

- Source: Elvis Saravia
- Published: 2026-11-11T22:04:12.000Z
- Bad Influence: https://badinfluence.news/items/w4znajw7a4a6bqlk1hwamsed5
- Original: https://x.com/omarsar0/status/2120638750740844672

## Summary

Daniel Lenton of Unify says OpenClaw, Hermes Agent and Prime Agent do not meaningfully learn from experience. On the new Continual-ARC benchmark, none of the three memorized anything of value or generalized to new tasks, while training a small 9B model's weights did both. Elvis Saravia argues the harness is increasingly part of training pipelines for continual learning, though it can stay minimal and efficient.

## Post

Highly recommended read. 

This is a topic I have been researching and writing about recently. 

The way we use agent memory in harnesses today works, but mostly for simple, one-off, and general tasks. 

For more novel, complex, and domain-specific tasks, you will often find that current harness and memory systems don't generalize so well.

So the big question is: how can we encode continual learning capabilities in models and harnesses?

This is where the idea of co-evolving harnesses and models begins to make sense.

This work, along with the benchmark (Continual-ARC), provides an interesting analysis of this. 

The harness is not just for steering the model and providing tools and context; it is increasingly becoming a key part of training pipelines to unlock continual learning in models. The harness continues to be important, but it can remain minimal and efficient.

## Quoted post @DanielLenton1

> OpenClaw, Hermes Agent and Prime Agent don't meaningfully learn from experience.
> 
> On Continual-ARC, our new benchmark at @letsunifyai, none of them were able to memorize anything of value and none were able to generalize to new tasks. Training a small 9B model's weights did both.

https://x.com/DanielLenton1/status/2120535187587268777