#Eval/Benchmark

AIJohn Groetzinger, writing in a personal capacity rather than for Cisco, argues that enterprise skills need packaging, evaluation, syncing, and distribution rather than scattered markdown files. He describes using skills to make cheaper models viable, converting curated TAC knowledge-base articles into maintained skills, and rolling out an eval framework across teams. He also describes syncing a repository README to Confluence with a deterministic script.

AIArtificial Analysis has released Harvey LAB-AA, an evaluation built on Harvey's LAB dataset and developed in collaboration with Harvey. Full results are published on the Artificial Analysis evaluations page, alongside Harvey's commentary on the benchmark and human expert preferences.

AIThe NVIDIA KGMON team placed second in the KDD Cup 2026 Data Agents competition with a system built around a smaller, clearer, and easier-to-verify agent harness. The competition required agents to answer natural-language questions over heterogeneous sources, including databases, CSV and JSON files, prose documents, PDFs, and briefing videos.

AITessl argues AI code review is slow because generated code outpaces trust, and proposes executable specs that let agents check preview environments against product intent. Its spec reviewer splits work between a planner agent that extracts requirements and parallel verifier agents that test each one against the code and base branch.

AIArena, the crowdsourced AI model leaderboard that started as a UC Berkeley research project, raised a $200 million Series B at a $3.1 billion valuation, led by Lightspeed Venture Partners and Khosla Ventures. The company said it reached $100 million in annualized run-rate revenue in June, up from $30 million when it raised its $150 million Series A in January at a $1.7 billion post-money valuation.

AIThe Association of Historical Mathematicians (AHM) has called for a boycott of OpenAI after the company released more than 700 AI-generated proof files at once. Fields Medalist Terence Tao, who chairs the group, argues that AI solving open problems autonomously reduces seminars, collaborations, and fertile research directions, and that the field should shift its measure of progress toward explanation and community-building.

Why it matters: The article links the AHM boycott call to Tao's argument that AI-driven proof volume is changing how mathematicians measure progress and whether solutions remain useful.

AIOpenAI released hundreds of claimed solutions to hard math problems but did not fully meet guidelines from the Advisory Group on Mathematics and Artificial Intelligence. Only 10 of 719 manuscripts included chain-of-thought releases, and just 42% of proofs were formalized. A Cambridge and King's College paper found discrepancies between a natural language proof and its Lean code for a Navier-Stokes-derived problem.

AIWhen asked to propose a research project and pilot an experiment, models made simple setup mistakes but presented them as interesting findings. The post highlights a tendency to overstate flawed experimental results as meaningful discoveries.

AIArena announced a $200 million Series B at a $3.1 billion valuation, alongside a new Alignment Index that measures whether AI agents behave safely, truthfully, and within the bounds of user requests. The company has surpassed $100 million in annualized revenue, facilitated 350 million sessions and 62 million votes, and led by Felicis and PXD from the seed and Series A stages. Arena positions the index as a way to assess trustworthiness as AI systems increasingly take real actions.

AIRSIGym provides a research agent with training, inference, evals, and sandboxes as callable services, so it spends its budget on experiments rather than rebuilding infrastructure. With Opus 5 as the researcher, the improved system rose from 17.67% to 50.33% on SWE-bench Verified. The post also highlights a way to measure co-evolution between harnesses and models.

AIPerplexity's Decider V1.1 ranked first on DecisionBench while also having the lowest cost, according to a post highlighting the result. The benchmark results cited include 949 shared text cases, 93.9% accuracy, a 534 ms median latency, and $0.016 per 1k decisions.

AIVoice Arena's Monsoon ASR dataset fine-tuned Whisper Medium on Bengali FLEURS, reducing LLM word error rate from 85.27% to 7.65%. The corpus spans 100,000 hours across 50 languages, and Voice Arena says more than 80 organisations have asked to license it since its launch a week ago.

AIJetBrains has released Mellum2.1, a 12B mixture-of-experts thinking model with 2.5B active parameters, under Apache 2.0 on Hugging Face. Post-training reinforcement learning in real software repositories raised SWE-bench Verified from 2.0 to 47.0, according to JetBrains' self-reported results. Qwen3.5-9B still leads on SWE-bench Pro, GPQA Diamond and AIME, and GGUF builds start at 7.0 GB for local use.

AISantiago says physical consistency is the most important and hardest feature of a world model, and that many generated videos show objects defying gravity. He reports that Seedance 2.5 is currently the best among the evaluated world models. The post links to a physics evaluation benchmark in which eight video world models reached a top score of 57.76/100.

AIBoris Power, who owns OpenAI's account, praised the pace of progress on a result he called remarkable. The context from @0xdoug reports a PR merged and a tightened bound from κ = 2⁻¹⁸² to κ = 2⁻¹⁵, a community effort across contributors.

AIAnthropic released Claude Haiku 5.5, raising its OSWorld 2.1 score from 15.7% to 72.4% and supporting a 1 million token context window. The article notes Haiku 5.5 still scores 39.2% on Terminal-Bench 4.0 against Sonnet 5.5's 70.6%, and that prompts above 100,000 tokens are priced higher, so migration costs need to be measured on real workloads.

AIZyphra reports that patterns in model training emerge early enough to be useful and are inexpensive to measure. Using only 4,096 tokens, its placement and routing predictions come within roughly one percentage point of results obtained with 524,000 tokens.

AICohere is hosting a live webinar on the future of search and retrieval, covering Embed 5, Parse 5, and its new retrieval methodology, RCP-nDCG. The post is a livestream announcement and does not include details of the methodology or model performance.

AIBoris Power, who owns the OpenAI account posting this, responded to a third update on OpenAI problem #109, integer multiplication. The quoted update reports a tightened bound of κ > 2⁻¹⁸, improved from κ = 2⁻¹⁸², with an exact witness of 4.19 × 10⁻⁶.

AIPerplexity's pplx-decider v1.1 scored 643 of 669 on clinical decisions versus 628 for Jev, according to a quoted post. The quoted post says it costs 42% less per decision at about the same speed and that its weights can be downloaded and run inside hospitals.