# Cheap verifier Jev nearly matches GPT-6 Astra on RAG claim checks

> Original title: This paper finds that a cheap verifier can match a frontier LLM's accuracy at checking RAG claims, while letting more unsupported claims ...

- Source: Rohan Paul
- Published: 2026-10-27T10:24:24.000Z
- Bad Influence: https://badinfluence.news/items/ktaaos4auoyjztqff6e44lkot
- Original: https://x.com/rohanpaul_ai/status/2115026823377748246

## Summary

A paper tests Jev, a model that returns only a label and probability, against GPT-6 Astra on 495 human-labelled RAG claims. Jev lands within 0.6 points of Astra's accuracy at 187 times lower cost, but approves 23.2% of unsupported claims versus Astra's 14.1%. The authors say teams should measure how often a checker approves unsupported claims, since headline accuracy hides that gap.

## Post

This paper finds that a cheap verifier can match a frontier LLM's accuracy at checking RAG claims, while letting more unsupported claims slip through.

RAG pipelines keep asking whether evidence supports a claim, a simple yes or no. Most teams get that answer from a text-generating LLM at generation prices.

They tested Jev, which returns only a label and a probability, against GPT-6 Astra on 495 human-labelled claims. They also built a cascade where Jev answers confident claims and passes the rest to Astra.

Jev landed within 0.6 points of Astra's accuracy at 187 times lower cost, but it approved 23.2% of unsupported claims against Astra's 14.1%. The cascade also cut Astra's bill, yet sending the same share of claims to Astra at random saved about as much.

Before picking a claim checker for a RAG product, measure how often it approves unsupported claims, since headline accuracy hides that.

![](https://badinfluence.news/api/img-proxy?u=https%3A%2F%2Fpbs.twimg.com%2Fmedia%2FHVoVoPpb0AA2Jiu.png&mode=thumb&exp=1793404800&sig=409d506b4d4346d2)