Google and CMU propose Harmony Harness to keep security gains in self-improving agents

Google and Carnegie Mellon researchers describe Harmony Harness, a self-evolving agent harness that tracks security as a separate goal.

The harness evolves several versions balanced between passing tests and blocking exploits, and keeps general fixes separate from benchmark-specific ones. Rohan Paul, who reported the paper on X, says it is the researchers' claim that security gains persist far better when security is evolved separately.

On unseen crash types in SecRepoBench, 66.7% of Harmony Harness fixes both worked and blocked the exploit, against 36.7% for the starting harness. The researchers say the harness kept 55% of its security gain on those tasks, while the DGM baseline kept 0%.