Stanford HAI finds medical AI agents finish under half of real clinical tasks
Overview
Stanford HAI researchers report that the best medical AI agents finished less than half of the tasks in two new benchmarks of real clinical and administrative work.
Stanford HAI says the results contrast with these systems' ability to pass medical licensing exams. Faculty affiliates Nigam and Jon Chen explain the findings in a linked article. The reported source does not give the benchmark names, task counts, or the specific agents tested.
Written by AI from the articles below · updated Nov 12, 12:11 PM ET
Check the sources:
Article timeline
The articles in this story. Times are ET.
Stanford HAI@StanfordHAIOfficialStanford HAI reports medical AI agents finish under half of real clinical tasksAIStanford HAI says two new benchmarks of real clinical and administrative work show the best medical AI agents finished less than half of the tasks. The post contrasts this with the systems' ability to pass licensing exams. Faculty affiliates Nigam and Jon Chen explain the findings in a linked article.
Heat Trend
- Current comparable heat
- 5
- Peak comparable heat
- 10Nov 12, 1:00 PM ET
- Change in past 24 hours
- -49%
The trend compares only continuously observed participants, so its scope may be smaller than the current heat total. Hover or tap for hourly heat; use arrow keys with a keyboard.