#Eval/Benchmark

AILlamaIndex introduced Extract v2.5, a series of agents tuned for document extraction across cost-effective, agentic, and agentic plus tiers. The company reports the agents outperform Opus 5.5 and GPT-6 Sol while costing 30% to 4x less, with accuracy gains on long lists (86.1% to 95.5%), multi-page records (85.5% to 96.5%), and scanned forms (90.9% to 95.7%) on its agentic tier. The release adds advanced citations with bounding boxes and structural reasoning, and the agents are available on LlamaParse.

AIHugging Face shows that training LFM2.5-2.6B with RL inside the agent harnesses themselves lifted held-out task success from 42% to 54% across four harnesses. Before training, the model solved 62% of tasks in Mini-SWE-Agent but only 33% in Claude Code, so the same model behaved very differently per harness. The approach uses an OpenEnv capture proxy to record tokens and logprobs, Harbor for tasks and sandboxes, and TRL's async GRPO trainer, with 31% fewer tool calls on already-solved tasks; training in OpenCode alone mostly improved OpenCode.

AIHamel Husain advises against building automated evaluators for every failure mode discovered during AI development. He argues that teams should weigh the costs of different evaluator types, such as code-based checks versus LLM judges, before building an eval.

AIHamel Husain has published a longer review of a new Claude Auto Eval plugin after many users asked about it. He invites readers to share their experiences using the plugin and how it went for them. The plugin is part of Claude's ability to help build evaluations and hillclimb on them, as described by @ClaudeDevs.

AIGoogle has announced Gemini 4 Argon, initially available only to trusted cyber defenders through its Fairwind Program while US government approval is pending. The author says the model is aimed at long-running software engineering, enterprise knowledge work, and cybersecurity tasks, with a 1 million token output limit. The post also gives promotional pricing of $2 per million input tokens and $10 per million output tokens, rising to $4 and $20 afterward, alongside a benchmark comparison.

Why it matters: The post places Gemini 4 Argon's benchmark table beside GPT-6 Astra and Claude models, showing where each leads across coding, knowledge work, and cybersecurity tasks.

AIVarun Mohan says Gemini 4 Argon is doing well on the AA Coding Agent Index when run with the Antigravity harness. He notes that real-world usage matters more than benchmarks and promises more to come.

AIThe CDC announced that Google's science AI model ranked highest among 39 eligible models for forecasting flu-related hospital admissions during the 2025-26 flu season. Google's forecasts were built with Empirical Research Assistance, an AI tool that generates computational solutions across scientific fields.

AIThe post quotes Google's Sundar Pichai introducing Gemini 4 Argon as an early look at the next model. It claims frontier performance in complex workflows, cyber defense, and software engineering, and says Google teams are using it for tasks from coding to quantum computing. The author adds that on FrontierSWE the model is very self-critical and often says "Eureka!", a personality they describe as a large improvement over previous Gemini models.

AIPerplexity's embedding preview achieves the highest average nDCG@10 among tested models on ConTEB, though not on every task. It also outperforms voyage-context-4 on chunk retrieval while using 8x less storage per vector, at 1 KB (1024 dims, int8) versus 8 KB (2048 dims, float32).

AIPerplexity says its pplx-embed-v2-context-9b-preview model leads on Answer and Evidence retrieval at every cutoff. In document retrieval the gap is smaller, with its context v1 4B model slightly ahead at Document@3 and Document@5.

AIPerplexity says context-bench, a context-aware retrieval benchmark created and privately held by turbopuffer, has 2,099 queries and 38,894 documents. Perplexity submitted it for blind evaluation, with queries and capabilities inspired by turbopuffer customer conversations.

AIPerplexity introduced a new training method for contextual embedding models that encode each document chunk with the whole document in view. Its pplx-embed-v2-context-9b-preview sets a new state of the art on the ConTEB benchmark and on turbopuffer's privately held context-bench.

AITerminal test runs miss multi-turn agent regressions, so Agent Clinic Episode 3 builds an automated eval suite for a LangGraph agent in 60 minutes. The post presents a four-step framework for moving from informal checks to benchmarking AI agents, with a link to the full guide.

AINathan Lambert's latest Interconnects plot tracks daily tokens processed by leading open model inference providers. The figures come from public disclosures by Together, Baseten, and Fireworks, plus the OpenRouter API. He says he underestimated OpenRouter's growth.

AIAnt Ling reports that Ling-3.1-flash scored 65.35 on HealthBench Professional, a healthcare evaluation rather than clinical certification. The model was trained with feedback from AQ and Haodf.com (Good Doctor Online) healthcare scenarios.

AILing-3.1-flash achieved a 1.432x speedup on Pyright 1.1.400 while passing all 2,279 checks. The chart shows Claude Fable 5 at 1.502x and GLM-5.3-Flash at 1.290x, a close result on a real codebase with correctness intact.

AILing-3.1-flash built a Lua-to-x86-64 ELF compiler from scratch in about 17 hours, passing 178 of 182 tests for 97.8%. That is within 1.2 percentage points of the 99.0% shown for Claude Fable 5 and above GLM-5.3-Flash at 95.9%.

AILing-3.1-flash ranks #2 in the Mobile App (react native) category on Design Arena's Open Weights leaderboard, with 1,207 Elo. It also ranks #4 in SVG with 1,271 Elo among models added in the last 30 days.

AIAnt Ling introduced Ling-3.1-flash, a model with about 560B total parameters, about 25B active per token, and up to a 1M-token context window. The company plans to open-source the model soon. It reports 1,673 Elo on GDPVal-AA v2.1, 75.16 on FrontierSWE, and 65.35 on HealthBench Professional across work, coding, and healthcare tasks.

AINVIDIA and Nous Research published a hands-on walkthrough of NVIDIA NeMo Relay for collecting traces from Hermes Agent. The guide runs two example scenarios and shows the agent's calls and retries in Arize Phoenix. It also covers how Nous used traces and task results to evaluate fixes across repeated runs.

AIIdeogram posted a side-by-side test of identical edits on Ideogram 4.5, GPT Image 2.5 Sunburst, Nano Banana Pro, and Nano Banana 2. The post claims GPT Image and Nano Banana outputs become unusable within a few edits, while Ideogram 4.5 remains clean across repeated edits.

AILiquid AI and InSilicoMeds released LongevityBench, an aging benchmark with 17 tasks spanning clinical records, DNA methylation, transcriptomics, proteomics, and genetics. On several tasks, Liquid AI's compact LFMs outperformed every frontier model the team evaluated. The team plans to present the work to the longevity research community at ARDD this week.

AIResearchers from Tencent Hy, Fudan University, and Tsinghua University released ExplorationBench, a benchmark that tests whether AI systems can discover hidden rules in executable Alien World sandboxes. Across 10 frontier systems, getting feedback from experiments outperformed thinking alone, with the best run reaching 89.0% after four rounds. The authors note that rankings barely transfer between the two worlds, and the code is listed as coming soon.

AIIndexTeam has released Index-Translate, a multilingual family covering text, speech, dubbing, and long-document translation across 150 languages. Its 9B model scores 0.8789 on FLORES, 0.8209 on instTrans, and 0.7387 on MEME, and the 2B and 9B models are released under Apache 2.0.

AIInSpatio-World 1.5 from InSpatio_AI turns a single image, four images, a panorama, or a video into a navigable scene with wide viewpoint changes. The 1.3B model scores 68.72 on WorldScore-Dynamic, ranking first among evaluated real-time and interactive methods, with speeds up to 24 FPS. The post says the code is released under Apache 2.0 and that dependencies keep their own licenses.