Skip to content
Trending storyDeveloping

LlamaIndex releases ExtractBenchPro, a harder document extraction benchmark

1 article1 sourcesince Oct 15Last article 3h ago ·

Overview

AISummary of 1 article

LlamaIndex releases ExtractBenchPro, a document extraction benchmark built from complex real-world documents such as trial balances, well logs and loss runs.

The benchmark is available on a website and technical blog, and on GitHub and Hugging Face. Its headline metric is the share of documents with a perfect 100% value F1 score. Jerry Liu, LlamaIndex's founder, says the best frontier model, Codex with Astra, reaches about 58% value F1 at nearly $1 per page, while LlamaIndex's agentic parsing modes score 20–40% at prices he calls feasible for full production use.

Written by AI from the articles below · updated Oct 15, 12:45 PM ET

Check the sources:

Latest developmentOct 15, 12:16 PM ET
LlamaIndex launches ExtractBenchPro, a harder document extraction benchmark

Article timeline

The articles in this story. Times are ET.

Oct 15
  1. Jerry LiuX
    LlamaIndex launches ExtractBenchPro, a harder document extraction benchmark

    AILlamaIndex releases ExtractBenchPro, a document extraction benchmark built from complex real-world use cases such as trial balances, well logs, and loss runs. Its headline metric is the share of documents with a perfect 100% value F1 score, and the best frontier model, Codex with Astra, reaches about 58% Value F1 at nearly $1 per page. LlamaIndex's agentic parsing modes score 20–40% at prices it says are feasible for full production use.

    Video from @jerryjliu0's post

Heat trend

Current comparable heat
9
Peak comparable heat
10Oct 15, 1:00 PM ET
Change in past 24 hours
–

The trend compares only continuously observed participants, so its scope may be smaller than the current heat total. Hover or tap for hourly heat; use arrow keys with a keyboard.