Skip to content
Trending storyDeveloping

Carnegie Mellon and JetBrains find coding-agent token spend does not track shipped output

1 article1 sourcesince Nov 11Last article

Overview

AISummary of 1 article

A Carnegie Mellon and JetBrains paper links five weeks of coding-agent token use to commits by 404 JetBrains engineers.

For the same engineer, doubling token spend went with only about 10% more commits, according to a post by Rohan Paul summarizing the paper. The authors conclude that token spend shows how much engineers use AI agents, not how much they ship, and recommend using it only for budgeting.

The authors suggest judging agent-assisted work by evidence about tasks and outcomes instead.

Written by AI from the articles below · updated Nov 11, 12:44 PM ET

Check the sources:

Latest developmentNov 11, 12:13 PM ET
Carnegie Mellon and JetBrains paper finds token spend tracks agent use, not shipped output

Article timeline

The articles in this story. Times are ET.

Nov 11
  1. Rohan PaulX
    Carnegie Mellon and JetBrains paper finds token spend tracks agent use, not shipped output

    AIA Carnegie Mellon and JetBrains paper links five weeks of coding-agent token use to the commits of 404 JetBrains engineers. For the same engineer, doubling spend went with only about 10% more commits. The authors recommend using token spend only for budgeting and judging agent-assisted work by evidence about tasks and outcomes.

    Image from @rohanpaul_ai's post

Heat Trend

Current comparable heat
6
Peak comparable heat
10Nov 11, 1:00 PM ET
Change in past 24 hours
–

The trend compares only continuously observed participants, so its scope may be smaller than the current heat total. Hover or tap for hourly heat; use arrow keys with a keyboard.