Together with NVIDIA, we built 2,500 hard enterprise tasks modeled on AutomationBench
We tested for shortcuts and answer leakage and ensured that graders reward valid solutions without accepting incomplete work
Prime Intellect and NVIDIA build 2,500 hard enterprise tasks
AISummary
Prime Intellect, working with NVIDIA, built 2,500 hard enterprise tasks modeled on AutomationBench. The post says the tasks were tested for shortcuts and answer leakage, and that graders reward valid solutions without accepting incomplete work.
Post on XView on X
Source: Prime Intellect · x.comPublished · added here