CMU adds guest lecture arguing AI agent benchmarks miss real human work

Zora Wang's guest lecture "AI Agents for Human Work" in CMU's 11-768 course argues that high leaderboard scores do not show that AI agents can do real human work, according to a post by X user meng shao (@shao__meng).

The lecture cites the Remote Labor Index, which the post says shows real remote work remains difficult for agents, and a finding that mainstream agent benchmarks cover only about 5% of employment. The slides are linked from the post.