#Data/Training

AICohere reports that after the first training step, its model could already translate over 90% of the training documents, which created a data problem. Kocmi describes how the most difficult samples were used to strengthen North Small Translate's capabilities. This post is part 3 of a six-part thread.

AICohere says its supervised fine-tuning extended machine translation with data focused on post-editing, error detection, and terminology. The team also ran multiple reinforcement learning and DPO stages aimed at errors the model showed.

AISebastian Raschka released round six of his Reasoning from Scratch series, introducing Reinforcement Learning with Verifiable Rewards (RLVR) and Group Relative Policy Optimization (GRPO) with an implementation. The video covers accuracy and format rewards, DeepSeek-R1 training, and GRPO versus PPO, then walks through a training loop and evaluates checkpoints on MATH-500.

AILlamaIndex introduced Extract v2.5, a set of document extraction agents that can reconstruct records split across pages and assemble them with thousands of other cells into structured tabular output. The post says the agents handle real-world documents like insurance claims, regulatory filings, and legal schedules, where a record may start on one page and finish on the next. The accompanying background post claims record-spanning-page accuracy rose from 85.5% to 96.5%, and that the agentic tier outperforms Opus 5.5 and GPT-6 Sol at 30% to 4x lower cost.

AICMU researchers released SMDD-Bench, a benchmark of 502 small-molecule drug design tasks that use RDKit, ADMET-AI, and Boltz-2 as feedback loops. The authors argue that long-horizon planning, exploration, and learning from imperfect feedback remain open problems beyond math and coding, and the benchmark is available in Prime Intellect's Environments Hub for training with prime-rl.

AIReplit chat now generates interactive charts when users ask Replit Agent to visualize data. Users can also choose GPT-6.1 Sol from OpenAI or Claude Sonnet 5.5 from Anthropic when building with Agent, or stay in auto mode. Jev is available through Replit AI Integrations for classifying content, routing requests, and scoring leads without managing API keys.

AIPrime Intellect says long-context agent serving depends on retaining history, scheduling new work, and moving cached state efficiently. It optimized three paths separately: prefill topology and scheduling, compressed KV with a fused attention kernel, and a transfer-friendly cache layout.

AIThe vLLM team integrated Helion, a PyTorch-native kernel DSL, into vLLM's linear backend, using per-shape autotuning to select among Standard GEMM, Split-K, and Swap-AB variants. On NVIDIA Hopper GPUs, the Helion backend outperformed the default CUTLASS and DeepGEMM backends across the evaluated models, with more than 10% throughput gains for some workloads. The work focuses on FP8 and INT8 quantized GEMM.

AIMeta researchers used Muse Spark to prove a clear rule for when a cycle-based relaxation of a hard optimization problem matches the original exactly and when it leaves a gap. The work concerns completed length-three alpha cycles, and the full paper is linked in the post.

AIResearchers disproved a proposed rule about symmetry-describing mathematical structures by finding a single counterexample. Muse Spark generated the search code that located it, and the team verified the result and completed the proof.

AIEpoch AI estimates that compute built from projected 2025 to 2027 high-bandwidth memory shipments could support tens to hundreds of millions of frontier AI agents, or billions of cheaper ones. Running nonstop, the top-tier agents would match the working hours of 140 million to 700 million full-time employees, and the central DeepSeek V4 Pro estimate of about 1.9 billion agents would match 8 billion workers.

Why it matters: The estimate converts memory shipments into agent capacity and revenue ranges, showing how hardware supply could translate into labor and sales if demand keeps up.

AIAdaption Labs has made Invent, its tool for generating AI training datasets from a plain-language description, available through an API. Developers can reportedly produce AI-ready training datasets in minutes with a few lines of code, according to the quoted post. Documentation is available at docs.adaptionlabs.ai.

AILlamaIndex says its LlamaParse agentic parsing tier turned the BEA's Q2 GDP-by-state infographic, which lacks a text layer, into a 52-row table. The post presents this as the tool faithfully recording data from an image-only document by default.

AINVIDIA reports that fine-tuning Nemotron 3.5 ASR reduced word error rate on Najdi and Hijazi Saudi Arabic from 55% to 30%. The post says a new tutorial shows how to adapt the model for other dialects and languages.

AINathan Lambert and Tom Zick have unveiled Trillium Labs, a new non-profit focused on the open science of frontier AI. The lab plans to build open post-training recipes and expand into open infrastructure to study topics such as RSI, reward hacking, and multi-agent systems. It is hiring, fundraising, and seeking compute, with support from Halcyon Futures and Schmidt Sciences.

AIHarrison Chase says LangChain is improving memory in managed Deepagents, noting that memory is difficult to get working well in company settings. The linked background post describes user memory in Managed Deep Agents 0.8, which lets an agent remember the people it works with.

AIGoogle AI announced that its Project Suncatcher prototype satellite, built with Planet, has launched into orbit on SpaceX's Transporter-18 rideshare mission. The initial mission will gather data on how Google TPUs handle the physical stress and extremes of spaceflight. The post says low Earth orbit systems could generate up to 8x more solar power than on Earth, and that future work may link multiple satellite constellations for scaled machine learning.

AIAi2 released AstaBrief 8B, an open-weights model that turns a research question and retrieved literature excerpts into a cited report, along with its training data. The model runs as Fast mode in Asta, averaging 51.1 seconds per report versus 178.5 seconds for Thinking mode, about 3.5x faster. The post also describes filtering synthetic training data by citation density and building DPO pairs judged by two models that agreed.

Why it matters: The post explains how supervised fine-tuning, preference data, and citation-density filtering were used to build a cited-report model, which is useful for teams training their own models.

AIGoogle Research introduced a next-generation Federated Learning system that uses Trusted Execution Environments to deliver verifiable differential privacy. The design moves computation server-side, which the post says cuts training times.

AIGoogle announces a next-generation federated learning system that uses Trusted Execution Environments to provide verifiable, auditable data anonymization. The system publishes access policies to a public transparency log and is deployed in Gboard, which has launched English and Japanese next-word prediction models with stronger privacy guarantees and improved accuracy. Training time has also sped up significantly because computation moved to the server and is parallelized across many machines.

Why it matters: The post shows how Trusted Execution Environments make federated learning's privacy claims externally verifiable, rather than relying on trust in the server operator.

AIHugging Face and collaborators published a guide to multi-harness RL that trains models through a capture proxy without changing the agent harness. The proxy records the token ids and logprobs vLLM samples, and the source reports LFM2.5-2.6B rising from 42% to 54% after training across four harnesses. Fine-tuning on 3,189 successful rollouts from Qwen3.8-27B plateaued at 47.5%, below both RL runs, and the capture proxy, trainer, tasks, SFT data, training code, and seven trained models are released openly.

Why it matters: The source gives a concrete method for training models across several agent harnesses, with measured gains and a note that imitation learning underperformed RL.

AIAi2 released AstaBrief 8B, a model that turns a research question and retrieved literature excerpts into a cited report, along with its training data. In Asta's Generate a report feature, Fast mode averages 51.1 seconds per report versus 178.5 seconds for Thinking mode, about 3.5x faster. The model is built on Qwen3-8B with supervised fine-tuning and DPO, and institutions can run its open weights on their own infrastructure.

Why it matters: The post explains the data filtering and one-pass generation choices behind a fast open-weights report model, showing what worked and what did not.

AICool eval. Simply ask an LLM “Land or Water?” and give it a latitude and longitude coordinate as text. Ask 16,200 times, plot as image. The models know. From compressing the internet.

AIServiceNow CoreAI introduced AutoSynthData, which uses a target model's failures and a stronger teacher's successes to generate and validate new agent training tasks. In EnterpriseOps Gym experiments, the Hybrid domain produced 2,000 samples and raised Gemma-4-26B-A4B-it mean Pass@1 by 7.2 percentage points, while the ITSM domain produced 1,994 samples and raised it from 18.77% to 27.18%.

Why it matters: The post shows how failure analysis, teacher demonstrations, and verifier checks combine into a repeatable pipeline for generating targeted agent training data.

AIMicron reported fiscal Q4 revenue of $54.23 billion, up 4.8X year on year, with net income of $37.7 billion. Its Cloud Memory unit, dominated by HBM, posted $16.28 billion in sales at an 83 percent gross margin.

AICoreWeave's new RL rollouts service uses ModelExpress and Router in NVIDIA Dynamo to speed up model weight reloads during RL post-training with minimal downtime. Working with NVIDIA and You.com, CoreWeave achieved 15× faster model reloads than its baseline while post-training Nemotron 3.5 Lightning. The speedup addresses GPUs sitting idle while inference workers wait to load updated weights between training iterations.

AIResearchers Maureen de Seyssel, Jie Chi, and Zakaria Aldeneh found that strengthening language discrimination during pretraining reduces the performance gap between multilingual and monolingual HuBERT speech models. In a controlled English/French setting, phone-ABX error fell from 11.6% to 10.4%, close to the monolingual 10.8%, while lexical sWUGGY scores rose from 52.1% to 56.7%. The gains were largest when language discrimination was introduced in the first training iteration.

AIEpoch AI estimates that high-bandwidth memory shipped in 2025–27 could eventually support about 30–170 million concurrent frontier-model agents once fully deployed and allocated. Using DeepSeek V4 Pro serving benchmarks, the estimate rises to about 1.9 billion concurrent agents. The authors compare the implied API-equivalent spending of $2.6–5.3 trillion per year with projected developer revenue of roughly $1 trillion by end-2027, suggesting demand may lag supply.

Why it matters: The analysis converts HBM shipment data into concurrent agent capacity and compares it with projected API revenue, showing where compute buildout may outpace demand.

AIApple Machine Learning Research reports that discrete diffusion steps match the training distribution only when simultaneously written token positions are conditionally independent given already-fixed tokens. The authors show that per-position distributions cannot determine such dependence, and on the synthetic ScanAndAdd task, confidence-ranked groups of two or more positions were dependent and produced a generated distribution 29 times the sampling-noise floor in total variation.

AIGoogle's prototype satellite for Project Suncatcher, built with Planet, launched into orbit on the Transporter-18 rideshare mission with SpaceX. The team confirmed contact and says the satellite is operating as expected. Over the coming weeks, it will gather in-orbit data on how Google's TPUs handle spaceflight stress, radiation, and thermal extremes, and a peer-reviewed paper detailing the research is available in Joule.

AIFireworks reports that numerical mismatch between training and rollout engines can destabilize reinforcement learning, with a GLM 5.2 experiment showing collapsing reward without alignment and stable reward with it over 25 steps. The post notes that MoE models add further alignment challenges, as Qwen3.5-MoE differences in expert output combination caused disagreement even when one implementation used higher precision. Fireworks says it co-develops its trainer and rollout engine to keep frontier RL training aligned across numerics, kernels, and MoEs.

AIRollouts account for most of RL's compute cost, and splitting them from training across separate engines can introduce numerical mismatches. In MoE models, such mismatches can even route tokens to different experts. Fireworks says it co-builds both engines so training stays fast and consistent.

AIDatabricks has released ai_decide(), a function that runs decision models natively across enterprise data, enabling fast typed decisions instead of slow text generation. The approach targets large datasets already governed within Databricks, according to the post and linked blog.

AIGoogle DeepMind's podcast discusses how to verify whether content is human-made, machine-generated, or natural, focusing on provenance and imperceptible watermarking. Episode sections cover SynthID for images and video, plus SynthID Bio for biological designs.

AIGoodfire Research developed sequence-aware monitors using protein language model embeddings to flag concerning biological sequences in dual-use AI agent tasks. On a custom benchmark, the monitors outperformed frontier model safeguards with fewer refusals on benign requests, and they held up better against paraphrasing and fragmentation attacks. The paraphrase results rely on in-silico estimates and do not establish whether the redesigned proteins keep biological activity, and the monitors run in milliseconds per sequence.

Why it matters: The post gives a concrete benchmark setup and fragmentation results, showing how sequence embeddings can separate dual-use biology requests that task-based safeguards handle poorly.

AIPrime Intellect reports that after about 100 GRPO steps on Hosted Training, Qwen3.6's reward on held-out problems rose from 0.127 to 0.361, a 2.8x gain. Qwen3.5, trained the same way, reached 0.356, suggesting the method works across model families. Both post-trained models finished well ahead of other open models and narrowed the gap to Claude Opus 4.8, with Qwen3.6 activating only 3B parameters per token.

AIExtropic designed the tasks and reward, while Prime Intellect supplied the RL infrastructure, including verifiers for environment construction, Hosted Training for the RL loop, and Prime Sandboxes for executing model code. Prime Inference serves the LLM judge and frontier baselines in the same workflow.

AIZyphra says its results clarify how state-of-the-art NoPE models encode position and which inductive biases support generalization. It adds that global NoPE could enable models to extrapolate to contexts longer than those seen in training, potentially indefinitely.

AIIn Zyphra's experiments, models with smaller context windows predicted text better while using less training compute. Zyphra believes the stronger recency bias induced by smaller windows outweighs the reduced compute benefit.

AIHybrid NoPE models combine sliding window attention or recurrent layers, which focus on nearby words, with global attention layers that use no positional encoding (NoPE). The post notes that NoPE layers receive no positional information yet can still learn long-range dependencies, and raises the question of how this works.