# Rohan Paul says easier curation of test datasets from production usage would help

> Original title: Making it easier to curate test datasets from production usage sounds pretty handy.

- Source: Rohan Paul
- Published: 2026-10-27T17:13:36.000Z
- Bad Influence: https://badinfluence.news/items/n9z7zpzxpk9a7zpa3omv0svvb
- Original: https://x.com/rohanpaul_ai/status/2115129800876863857

## Summary

Rohan Paul says it would be useful to curate test datasets from production usage, since people ask for things nobody wrote down in advance. He wants more of those real requests added to the tests. The post links to Judgment Labs' general-access launch, which says agent traces hold data on user requests, agent failures, and external interactions.

## Post

Making it easier to curate test datasets from production usage sounds pretty handy.

People ask for all sorts of things nobody thought to write down in advance. Would be nice to get more of those into the tests

## Quoted post @alexshander03

> Today, @JudgmentLabs opens for general access.
> 
> We believe long running agent systems will generate more exhaust data than the world can currently process.
> 
> Hidden within agent traces are rich records of how to make these systems better: what users ask for, when agents stumble, and how agents interact with the external world.
> 
> Judgment is the most performant monitoring and improvement layer for long-horizon agents.
> 
> We’re excited to see what you build!

https://x.com/alexshander03/status/2115111288066171211