Updated
Updated
Showing low-relevance items too. Hide low-relevance items
Aravind Srinivas@AravSrinivasAI score44AIPerplexity open-sourced pplx-decider-27b, a state-of-the-art multimodal Decision Model, and offers it through a new Decisions API. Input tokens cost $0.04 per million and output tokens are free, with further price cuts promised in the coming days.
Mike Knoop@mikeknoopAI score44AIMike Knoop notes that a verification of the roughly 27B-parameter model is notable because it is about as large as fits within the official ARC Prize Kaggle competition runtime. ARC Prize reports Qwen3.8-27B from Alibaba's Qwen team scored 42.4% on ARC-AGI v2 at $0.45 per task and 87.5% on v1 at $0.22 per task.
ARC Prize@arcprizeAI score22AIARC Prize has published full ARC-AGI results for Alibaba's Qwen3.8-27B, with the testing run hosted by Baseten. The post links the public leaderboard, the open-source benchmarking repository for reproducing the results, and the testing policy.
TypeSafe AI@typesafeaiAI score14AIJev, a reranker from TypeSafe AI, improved Wrangle's main-app search by 30.64% in matches per 200 candidates while costing 75% of the previous reranking cost. The gains were measured on searches recruiters actually run, rather than standard 10-candidate benchmarks, and the full benchmark is slated for release soon.
TypeSafe AI@typesafeaiAI score34AIIn a business risk monitoring test by @edwardirby, the Jev judge matched the report quality of an ordinary LLM judge while missing no investigations, versus the LLM missing 5 of 11. The LLM's threat scores also flip-flopped from 0.35 to 0.68 to 0.50 on the same threat, while Jev was 250x cheaper and 3-6x faster.
François Chollet@fcholletAI score62AIFrançois Chollet argues the key difference between base LLMs and modern LRMs is a shift from transductive answer prediction to inductive prediction of the program or reasoning chain behind an answer. He says this enables test-time induction and substantial fluid intelligence in LRMs, which he claims base LLMs largely lack. He cites ARC 1 results: base LLMs remain around 10-15%, while LRMs of the same size or smaller saturated the benchmark in 2025.
Sophia Yang@sophiamyangAI score14AIFireworks AI's Ember-1 produced a near-identical composition to Kimi K3 on the same JavaScript painting prompt, according to Sophia Yang. Ember-1 used 9x fewer thinking tokens and ran 4x faster at half the cost.
Perplexity Developers@perplexitydevsAI score41AIPerplexity introduced its Decisions API, powered by pplx-decider-v1-27b, a multimodal model that outputs a probability distribution over a fixed set of answers rather than text. The company says the API costs $0.04 per million input tokens and scores 85.71% across benchmarks.
Guillermo Rauch@rauchgAI score38AIGuillermo Rauch argues that the future is verification engineering, spanning proofs, end-to-end tests, benchmarks, and linters. He expects some of these tests to be deterministic and others agentic, and he says the approach looks great. The quoted post introduces e2e, an open-source agentic testing framework that mixes deterministic and agentic APIs and runs locally or in CI.
Odyssey@odysseymlAI score26AIOdyssey introduces PROWL-2, a recursive loop that delivers up to 91% relative gains over the StarCraft world-model baseline. The post also reports stronger robot coordination in simulation.
Google Gemma@googlegemmaAI score54AIGoogle Gemma relays a StudentBench study reporting that AI tutors matched expert human tutors on immediate GRE learning gains. The author reports 2,383 students and a cost of 7 cents per AI tutor hour versus $75 for an expert human hour. The post also says the top AI tutor beat expert human tutors on average in 5 of 7 academic topics, and that the data and paper are publicly available.
Google Gemma@googlegemmaAI score35AINew research finds AI tutoring delivers immediate GRE learning gains comparable to expert human tutors, with Gemma 4 31B performing best. The post presents the findings as a way for educators to scale their impact and make guidance more accessible.
Goodfire@GoodfireAIAI score38AIGoodfire reports that its method catches more unsafe biological requests than frontier model safeguards while dramatically reducing false positives. The claim is illustrated by a plot in the first tweet of the thread.
Goodfire@GoodfireAIAI score29AIGoodfire built monitors that let AI agents perform more biology work while flagging dual-use risks. The monitors reportedly outperform frontier model safeguards, with fewer refusals on dual-use tasks, and run fast and in real time while staying robust to adversarial attacks.
Julien Chaumond@julien_cAI score38AICloudflare's Workers AI team has released Clef, its first models, two fast decision models that it says top benchmarks for quality and latency. They are available hosted on Workers AI or as open weights on Hugging Face.
Prime Intellect@PrimeIntellectAI score34AIPrime Intellect reports that after about 100 GRPO steps on Hosted Training, Qwen3.6's reward on held-out problems rose from 0.127 to 0.361, a 2.8x gain. Qwen3.5, trained the same way, reached 0.356, suggesting the method works across model families. Both post-trained models finished well ahead of other open models and narrowed the gap to Claude Opus 4.8, with Qwen3.6 activating only 3B parameters per token.
Yellowbrick Investing@joinyellowbrickAI score18AIYellowbrick 2.0 is live, tracking 35,000+ stock pitches from 4,000+ authors and adding 300+ new pitches weekly. The rebuilt platform adds author leaderboards, custom feeds and alerts, API access, and paid research partner discounts. Premium subscribers get a 30% discount on Koyfin, which the company says covers the cost of Yellowbrick Premium.
Mustafa Suleyman@mustafasuleymanAI score40AIMicrosoft AI launched MAI-Transcribe-2-Streaming, which Artificial Analysis ranks #1 of 38 models for final transcript accuracy at 2.5% WER, returned 0.13s after end of speech. Artificial Analysis lists its streaming price at $0.54 per hour of audio, at the higher end among leading streaming models. Microsoft's post claims the model is 55% faster and 60% cheaper than ElevenLabs and invites developers to build agents on its platform.
Jerry Liu@jerryjliu0AI score42AILlamaIndex introduced Extract v2.5, a series of agents tuned for document extraction across cost-effective, agentic, and agentic plus tiers. The company reports the agents outperform Opus 5.5 and GPT-6 Sol while costing 30% to 4x less, with accuracy gains on long lists (86.1% to 95.5%), multi-page records (85.5% to 96.5%), and scanned forms (90.9% to 95.7%) on its agentic tier. The release adds advanced citations with bounding boxes and structural reasoning, and the agents are available on LlamaParse.
Lewis Tunstall @ COLM 🌉@_lewtunAI score44AIHugging Face shows that training LFM2.5-2.6B with RL inside the agent harnesses themselves lifted held-out task success from 42% to 54% across four harnesses. Before training, the model solved 62% of tasks in Mini-SWE-Agent but only 33% in Claude Code, so the same model behaved very differently per harness. The approach uses an OpenEnv capture proxy to record tokens and logprobs, Harbor for tasks and sandboxes, and TRL's async GRPO trainer, with 31% fewer tool calls on already-solved tasks; training in OpenCode alone mostly improved OpenCode.
AICloudflare released Clef and Clef-flash, two decision models hosted on Workers AI and open-sourced on Hugging Face under Apache 2.0, and launched a reinforcement learning fine-tuning service. In Cloudflare's tests, Clef classified a domain in 2.2s versus 4.7s for gpt-oss-120b, and the models are Jev-API compatible. The company is offering fine-tuning first through a forward-deployed engineering team, with a self-serve platform planned later.
Hamel Husain@HamelHusainAI score25AIHamel Husain advises against building automated evaluators for every failure mode discovered during AI development. He argues that teams should weigh the costs of different evaluator types, such as code-based checks versus LLM judges, before building an eval.
Wan@Alibaba_WanAI score62AIAlibaba's Wan 3.0 ranks #1 overall on the new Artificial Analysis AA-Video-T2V v2.0 text-to-video leaderboard, priced at $12 per minute of video. The benchmark judges models at 1080p using over 68,000 human preference votes across 1,000 prompts, and the author states Wan 3.0 leads 10 of 20 category boards.
AIGoogle announced Gemini 4 Argon, a model it says is built to sustain deep reasoning across complex, long-horizon workflows, roughly seven months after its last flagship release in February. The article says cybersecurity testing will be completed after October 1, with no benchmark scores, pricing, or availability details provided.
AILangChain built a model router inside its open source coding agent Open SWE that picks one of three models for each thread. In an A/B test against always using GPT-6 Astra, the median cost per thread fell 64% with no measurable change in merged PR rate. The router runs on the thread's first message, using a base prompt, per-tier criteria, and a classifier model, and the post lists next steps including subagent routing and mid-thread re-routing.
Hamel Husain@HamelHusainAI score38AIHamel Husain has published a longer review of a new Claude Auto Eval plugin after many users asked about it. He invites readers to share their experiences using the plugin and how it went for them. The plugin is part of Claude's ability to help build evaluations and hillclimb on them, as described by @ClaudeDevs.
indigo@indigoxPickAI score81AIGoogle has announced Gemini 4 Argon, initially available only to trusted cyber defenders through its Fairwind Program while US government approval is pending. The author says the model is aimed at long-running software engineering, enterprise knowledge work, and cybersecurity tasks, with a 1 million token output limit. The post also gives promotional pricing of $2 per million input tokens and $10 per million output tokens, rising to $4 and $20 afterward, alongside a benchmark comparison.
Why it matters: The post places Gemini 4 Argon's benchmark table beside GPT-6 Astra and Claude models, showing where each leads across coding, knowledge work, and cybersecurity tasks.
Varun Mohan@_mohansoloAI score25AIVarun Mohan says Gemini 4 Argon is doing well on the AA Coding Agent Index when run with the Antigravity harness. He notes that real-world usage matters more than benchmarks and promises more to come.
AIUnder equal time budgets and the same frontier LLM backbone, a single session of a minimal-harness coding agent with read, write, and bash primitives matched open-source state-of-the-art autonomous machine learning engineering harnesses. Apple researchers found the added orchestration and retrieval machinery redundant in large-scale ablation studies, pointing to the backbone model as the main driver of performance. They conclude that hand-crafted harnesses around strong models yield poor returns on current MLE benchmarks.
Google Research@GoogleResearchAI score40AIThe CDC announced that Google's science AI model ranked highest among 39 eligible models for forecasting flu-related hospital admissions during the 2025-26 flu season. Google's forecasts were built with Empirical Research Assistance, an AI tool that generates computational solutions across scientific fields.
Yi Tay@YiTayMLAI score46AIGoogle DeepMind introduced Gemini 4 Argon, a new frontier model built for coding, enterprise knowledge work, and cybersecurity defense, rolling out to trusted testers through its Fairwind Program. Yi Tay says Gemini 4 outperforms astra and fable on many tasks, though the post gives no benchmark figures.
Deedy@deedydasAI score22AIDeedy argues that benchmark scores for frontier AI models are increasingly meaningless because labs tune for them before launch. He suggests trusting price instead: a high price indicates a genuinely good model, while a low one suggests the model is benchmaxxed.
wh@nrehiew_AI score67AIThe post quotes Google's Sundar Pichai introducing Gemini 4 Argon as an early look at the next model. It claims frontier performance in complex workflows, cyber defense, and software engineering, and says Google teams are using it for tasks from coding to quantum computing. The author adds that on FrontierSWE the model is very self-critical and often says "Eureka!", a personality they describe as a large improvement over previous Gemini models.
Yuchen Jin@Yuchenj_UWAI score12AIYuchen Jin suggests Google is back in the AI race after a model reportedly outperforms Astra and Opus 5.5 across the board on benchmarks. He adds a caveat that the results may reflect benchmark optimization rather than real capability gains, and says he would welcome Google rejoining the race.
AIGoogle DeepMind announced Gemini 4 Argon, rolling out first to trusted cyber defenders through its Fairwind Program. Argon will launch at an introductory price of $2 per million input tokens and $10 per million output tokens, with output limits raised to 1M tokens. The post cites a 77.9% score on DeepSWE v1.1 and 91.7% on LVBench, and says broad availability will follow safeguard testing.
Why it matters: The post pairs Argon's benchmark claims with the phased release, pricing, and safeguard details, helping readers weigh its frontier-level capabilities against its access limits.
AIGoogle announced Gemini 4 Argon, a new frontier model rolling out first to trusted cyber defenders through its Fairwind Program. The model's output limit rises to 1M tokens from 64K, and its introductory API price is $2 per million input tokens and $10 per million output tokens. Google says broader availability to developers, enterprises, and consumers will follow after more testing of guardrails.
Why it matters: The post pairs benchmark claims with a phased access plan, pricing, and safety measures, which helps readers judge how quickly Argon may reach developers.
Perplexity@perplexity_aiAI score20AIPerplexity's embedding preview achieves the highest average nDCG@10 among tested models on ConTEB, though not on every task. It also outperforms voyage-context-4 on chunk retrieval while using 8x less storage per vector, at 1 KB (1024 dims, int8) versus 8 KB (2048 dims, float32).
Perplexity@perplexity_aiAI score24AIPerplexity says its pplx-embed-v2-context-9b-preview model leads on Answer and Evidence retrieval at every cutoff. In document retrieval the gap is smaller, with its context v1 4B model slightly ahead at Document@3 and Document@5.
Perplexity@perplexity_aiAI score23AIPerplexity says context-bench, a context-aware retrieval benchmark created and privately held by turbopuffer, has 2,099 queries and 38,894 documents. Perplexity submitted it for blind evaluation, with queries and capabilities inspired by turbopuffer customer conversations.
Perplexity@perplexity_aiAI score34AIPerplexity introduced a new training method for contextual embedding models that encode each document chunk with the whole document in view. Its pplx-embed-v2-context-9b-preview sets a new state of the art on the ConTEB benchmark and on turbopuffer's privately held context-bench.
