Skip to content
Trending storyDeveloping

Self-improving agents should accept harness edits only after fresh-task wins

1 article1 sourcesince Oct 15Last article 2h ago ·

Overview

AISummary of 1 article

Rohan Paul says self-improving agents should accept a harness edit only after it beats the current version on fresh, independent tasks.

He maps harness self-improvement methods onto evolutionary algorithms, including keep-the-best search, archives, Pareto selection and bandit-guided budgets.

In one compiled case, Paul reports that GPT-5.4-nano with an evolved harness and skills scored 77.58% on SpreadsheetBench. GPT-5.5 with skill tuning alone scored 76.20%. Paul notes that costs were not matched between the two runs.

Written by AI from the articles below · updated Oct 15, 1:24 PM ET

Check the sources:

Latest developmentOct 15, 1:06 PM ET
Self-improving agents should accept harness edits only after fresh-task wins

Article timeline

The articles in this story. Times are ET.

Oct 15
  1. Rohan PaulX
    Self-improving agents should accept harness edits only after fresh-task wins

    AIRohan Paul says self-improving agents should accept a harness edit only after it beats the current version on fresh, independent tasks. He maps harness self-improvement methods onto evolutionary algorithms, from keep-the-best search to archives, Pareto selection, and bandit-guided budgets. In one compiled case, GPT-5.4-nano with an evolved harness and skills scored 77.58% on SpreadsheetBench, while GPT-5.5 with skill tuning alone scored 76.20%, though costs were not matched.

    Image from @rohanpaul_ai's post

Heat trend

Not enough continuous observations to show a trend yet.