Skip to content
View original post on X: LMSYS OrgOfficial
Published
42/100

NeoCognition's ApprenticeBench finds frontier models beat human testers at 2.5× cost

AINeoCognition's ApprenticeBench tests whether AI agents can learn a job like new employees, including company conventions, tools, and tacit knowledge. Yu Su of NeoCognition says frontier models beat human testers on task success rate but cost 2.5× as much per task.
LMSYS OrgVerified on XOfficial
@lmsysorg
View on X

Part of a thread · earlier post

Can agents learn an entire job?

ApprenticeBench tests agents onboarding like employees: learning company conventions, tools and tacit knowledge.

Yu Su from @NeoCognition shares the results: frontier models beat human testers on task success rate, at 2.5× the cost per task.

Source details

LMSYS Org · x.comPublished · added here

Original title: Can agents learn an entire job?