Skip to content
Trending storyDeveloping

NeoCognition's ApprenticeBench finds frontier AI agents beat human testers at 2.5x cost

1 article1 sourcesince Nov 7Last article 43m ago ·

Overview

AISummary of 1 article

NeoCognition shares results from ApprenticeBench, a benchmark that tests whether AI agents can learn a job like new employees, including company conventions, tools and tacit knowledge.

Yu Su of NeoCognition says frontier models beat human testers on task success rate but cost 2.5 times as much per task.

The finding was reported by LMSYS Org on X, which relayed Su's results. The report gives no further figures on task counts, models tested or the size of the success-rate gap.

Written by AI from the articles below · updated Nov 7, 6:15 PM ET

Check the sources:

Latest developmentNov 7, 6:08 PM ET
NeoCognition's ApprenticeBench finds frontier models beat human testers at 2.5× cost

Article timeline

The articles in this story. Times are ET.

Nov 7
  1. LMSYS OrgOfficial
    NeoCognition's ApprenticeBench finds frontier models beat human testers at 2.5× cost

    AINeoCognition's ApprenticeBench tests whether AI agents can learn a job like new employees, including company conventions, tools, and tacit knowledge. Yu Su of NeoCognition says frontier models beat human testers on task success rate but cost 2.5× as much per task.

    Image from @lmsysorg's post

Heat Trend

Not enough continuous observations to show a trend yet.