New Amazon paper finds that benchmarks used to grade AI models on tax, legal and business rules often mark correct answers wrong, because the tests’ own answer keys or grading code contain mistakes.
Benchmarks like RuleArena and LegalBench test whether models apply written rules to tax, legal, and business cases. If the reference answer or the scorer is wrong, the score measures the benchmark's bugs, not the model.
They audited 9 public benchmarks by tracing references back to the source rules, rerunning reference programs, and testing scorers with controlled answers. Then they rescored the same saved model responses after each fix.
LegalBench's tax scorer deleted decimal points, so $259,487.13 was read as 25,948,713. Fixing it raised accepted answers from 8 or 9 to between 87 and 91 out of 96 for each of 3 models.
Before trusting a low score on a rule-following benchmark, read the scorer code and check a few failed answers by hand.