Updated
Updated
Showing low-relevance items too. Hide low-relevance items
Prime Intellect@PrimeIntellectOfficialAI score34AIvLLM changed its KV cache layout to block-major BLHNC, cutting transfer descriptors about 10x and halving mean KV transfer time on NVLink. The original slowdown came from fragmented KV layout that split one 200K-token request into 32K tiny copies, making NVLink slower than InfiniBand.
Prime Intellect@PrimeIntellectOfficialAI score20AIPrime Intellect reports that DEP8 provides about 5x the prefix-cache capacity of TEP8 on the same GPUs. The post argues that fast KV retrieval alone does not ensure fast first tokens, since cached KV often sat ready while requests waited to join a batch. Halving the prefill budget reduced median queue wait time and time to first token (TTFT).
Prime Intellect@PrimeIntellectOfficialAI score38AIPrime Intellect compresses the MLA latent KV cache to NVFP4, reducing each row from 576 to 352 bytes. This fits about 50% more cached tokens per decoder compared with FP8. Its native sparse-MLA kernel unpacks the format on-chip, and the company is contributing that kernel to FlashInfer as an experimental operation.
Prime Intellect@PrimeIntellectOfficialAI score38AIPrime Intellect served GLM-5.3 on GB200 NVL72 while targeting 100+ end-to-end tokens per second per user for concurrent agent tasks. At that interactivity bar, a 1:4 prefill-to-decode ratio delivered the most throughput, supporting 66 sessions per prefill group at 101 tokens/s per user and 100 output tokens/s per GPU.
Prime Intellect@PrimeIntellectOfficialAI score23AIPrime Intellect says long-context agent serving depends on retaining history, scheduling new work, and moving cached state efficiently. It optimized three paths separately: prefill topology and scheduling, compressed KV with a fused attention kernel, and a transfer-friendly cache layout.
Prime Intellect@PrimeIntellectOfficialAI score8AIPrime Intellect says its system achieved 100% uptime with a near-zero tool-call error rate. The post provides no further details on the product, model, or measurement period.
Prime Intellect@PrimeIntellectOfficialAI score20AIPrime Intellect has introduced Prime Inference, an inference service it says has served trillions of tokens for reinforcement learning and dedicated customer deployments. The company argues that owning your intelligence requires owning your inference, and the post promises to unpack its inference stack.
Prime Intellect@PrimeIntellectOfficialAI score38AIPrime Intellect says its production stack now processes roughly 600B tokens daily on its own compute. The company also states its GLM-5.3 deployment ranks among the fastest GLM-5.3 endpoints on OpenRouter.
Muse@MuseOfficialAI score18AIMuse has released a developer portal to simplify connector intake, letting developers submit, manage, and track their connectors. The company says it crossed 3,000 connector submissions on its Connector Platform, and it hopes the portal will help drive the next 3,000.
Guillermo Rauch@rauchgXAI score26AIVercel has added Jev to the AI SDK for Python, and the team tested it in two experiments: detecting whether typed text is Python or English, and writing Python one decision at a time. The main post is a short endorsement praising a writeup about Jev and Python.
Vercel@vercelOfficialAI score22AIVercel has added Jev to the AI SDK for Python. The team tested it in two experiments: detecting whether typed text is Python or English as it is entered, and writing Python one decision at a time.
Modal@modalOfficialAI score6AIModal thanked everyone who attended its Runtime event, highlighting nine tracks, more than 30 speakers, and a room full of engineers running AI in production. The post offers no further details about the sessions or announcements.
MiniMax Design (H3)@Hailuo_AIOfficialAI score10AIMiniMax's Hailuo AI account urges users to update to the latest version through a linked design page. The post gives no version number, feature details, or release notes.
MiniMax Design (H3)@Hailuo_AIOfficialAI score13AIMiniMax Design now offers Opus 5.5, which its account presents as a new way to create from code to motion. The post provides no further details on features, pricing, or availability.
AIBaseten tested the MetaInfer skills-only approach by having Claude Code build an inference engine, VibeQwen, for Qwen-3.6-35B-A3B in NVFP4 on a single B200. On single-stream text, VibeQwen decoded 90% faster than a tuned vLLM 0.25.1 deployment (1,792 vs. 943 TPS) and cut time to first token from 28 ms to 12 ms, with a 71% throughput gain at concurrency 32. The author notes this was an outcome-focused run that allowed some numerically different outputs as long as accuracy stayed at or above the BF16 baseline.
Why it matters: The post tests a skills-only inference engine method on a real model and states the speed and accuracy constraints used, helping readers judge how far such automated optimization can be trusted.
Muse@MuseOfficialAI score33AIMuse has released Muse Gadgets, open-source ESP32 firmware and a Linux SDK for building hardware devices that work with Muse. Developers can get an API token at gadgets.muse.ai and use a coding agent with the GitHub repository to build custom peripherals.
Aravind Srinivas@AravSrinivasXPickAI score62AIPerplexity has released several open source projects, including the pplx-decider-v1-27b multimodal decision model, the pplx-embed-v2-context-9b-preview contextual embeddings model, and the Lily local inference engine for Apple silicon. The post also lists the 0.6B on-device PII-Tracer classifier with its PII-TRACE benchmark, the WANDR research agent benchmark, and the Numbat and Bumblebee security tools, and says more open source releases are coming soon.
Why it matters: The post lists several named open source releases with specific benchmark figures, helping readers scan which tools and models Perplexity has recently published.
AIAnthropic released Claude Code v2.1.288, adding $.ui.selection() for mods, a built-in gh api for cloud sessions without the GitHub CLI, and --max-findings for /code-review. The release also fixes many issues, including mid-response API timeouts, resume and compaction bugs, and auto mode denials and model switching on Bedrock and Mantle.
AIThe vLLM team integrated Helion, a PyTorch-native kernel DSL, into vLLM's linear backend, using per-shape autotuning to select among Standard GEMM, Split-K, and Swap-AB variants. On NVIDIA Hopper GPUs, the Helion backend outperformed the default CUTLASS and DeepGEMM backends across the evaluated models, with more than 10% throughput gains for some workloads. The work focuses on FP8 and INT8 quantized GEMM.
SGLang@sgl_projectOfficialAI score38AISGLang has released v0.5.20, bringing Intel XPU into standard releases alongside RL sampling masks that make rollouts more reliable with up to 52% faster decode. The update also adds Unified Radix Tree SWA branching-point caching, which the project says lifts cache hit rate about 20 points and cuts TTFT by roughly one-third, plus up to 12.5× faster ROCm model loading. New models named in the release include GLM-5.3-Flash, Qwen3.8-Flash-Next, K2 Horizon, Hy4-Preview, FastH3, and VDN-H3.
SGLang@sgl_projectOfficialAI score39AISGLang's update adds a /v1/score endpoint that returns scores for requested labels such as Yes/No or A/B/C, avoiding the label loss of generate with top-k logprobs. Its multi-item scoring computes shared context once and keeps each candidate isolated, with 16-candidate p95 on Qwen3-8B dropping from 54.1 ms (Generate) to 20.6 ms.
SGLang@sgl_projectOfficialAI score28AISGLang demonstrated Qwen3.8-27B as a multimodal decision model that beat Pokémon FireRed's Elite Four and champion with sub-100 ms decisions from live game state. The company says its native /v1/decisions API lets LLMs and VLMs be used for classification and scoring. It also announced /v1/systemone for running Jev-like open models with the TypeSafe SDK.
SGLang@sgl_projectOfficialAI score14AISGLang has published full release notes for version v0.5.21 on GitHub. The post itself gives no further details beyond linking to the release page.
SGLang@sgl_projectOfficialAI score58AISGLang has released v0.5.21 with a native Decisions API that turns an LLM or VLM into a low-latency classifier and scorer. The release also lets /v1/score rerank search or RAG results in one call, lets PD instances switch between prefill and decode without restarting, and adds support for models including DeepSeek-V4.1 Flash, Kimi K3, and GLM-5.3-Flash on AMD MI355X. The announcement reports a 22% faster first token on long prompts for DeepSeek-V4.1 Flash and 20.6% higher prefill throughput for Kimi K3 in PD serving.
Perplexity Developers@perplexitydevsOfficialAI score12AIPerplexity says it looks forward to working with NVIDIA's new Vera CPU on its SPACE project. The company says its work depends on agents running more capable code environments safely and reliably, and it plans to test that work on Vera.
LiveKit@livekitOfficialAI score23AIAssemblyAI's Universal 3.6 Pro speech-to-text model is now available in LiveKit Inference, with 45% fewer wrong yes/no confirmations and about 30% less background speech transcribed. It supports 32 languages plus code-switching and endpointing that waits out phone numbers and emails, at the same $0.45/hr price, accessible by switching to universal-3-6-pro.
AIThe PyTorch Certified Associate (PTCA) Certification Pathway combines four self-paced learning modules with the PTCA exam, offered through Linux Foundation Education. The full pathway takes 15–17 hours of self-paced learning and hands-on labs, and covers tensors, data handling with Datasets and DataLoaders, neural network building, and performance tools including torch.compile, Automatic Mixed Precision, and the PyTorch Profiler.
AIMIT associate professor Cathy Wu is applying machine learning and reinforcement learning (RL) to design safer, more efficient transportation systems. Her team found RL can train effectively on about 10 percent of related problems, and a selection algorithm improved training efficiency by up to 30 times. Her recent work estimates eco-driving measures could cut vehicle emissions by 11 to 22 percent.
AIGitHub Copilot code review can now be requested through the REST and GraphQL APIs, with an optional review effort level set per request. Balanced became the default review effort level for new and existing repositories and organizations as of September 28, 2026, while users who explicitly selected Lite keep that setting. The changes are generally available to Copilot Pro, Pro+, Max, Business, and Enterprise plans.
AIEpoch AI estimates that compute built from projected 2025 to 2027 high-bandwidth memory shipments could support tens to hundreds of millions of frontier AI agents, or billions of cheaper ones. Running nonstop, the top-tier agents would match the working hours of 140 million to 700 million full-time employees, and the central DeepSeek V4 Pro estimate of about 1.9 billion agents would match 8 billion workers.
Why it matters: The estimate converts memory shipments into agent capacity and revenue ranges, showing how hardware supply could translate into labor and sales if demand keeps up.
Sara Hooker@sarahookrXAI score26AIAdaption Labs has made Invent, its tool for generating AI training datasets from a plain-language description, available through an API. Developers can reportedly produce AI-ready training datasets in minutes with a few lines of code, according to the quoted post. Documentation is available at docs.adaptionlabs.ai.
Vercel@vercelOfficialAI score18AIRogo's engineers build the company's internal apps on Vercel and deployed over 73,000 times last month. Its coding agents ship code to production in five minutes, powering tools from churn risk analysis to the sales team's deal desk.
eric zakariasson@ericzakariassonXAI score18AIxAI has published an early version of its TypeScript SDK, available on GitHub at xai-org/xai-sdk-ts and on npm as @xai-official/sdk. The post invites developer feedback as the company continues iterating on it.
AIAs of October 2, 2026, GitHub deprecated Gemini 3.5 Flash, Gemini 3.6 Flash, Kimi K2.7 Code, and Claude Opus 4.7 across all Copilot experiences. Suggested replacements are Gemini 3.8 Flash, Kimi K3, and Claude Opus 5.5, and Enterprise administrators may need to enable them through model policies.
AIRedwood Research argues that defining safety research as anything that expands the safety-usefulness Pareto frontier counts nearly all capabilities research as safety research. The post says safety research typically pushes the frontier right, creating safety options without new usefulness options, while capabilities research typically pushes it up and left, trading safety for usefulness.
Grok Bot@botOfficialAI score14AIGrok Bot says it has reset usage limits for all of its users. The post adds no further details about the reset.
François Chollet@fcholletXAI score28AIKeras is moving to a pluggable backend design, with MLX and PaddlePaddle backends upcoming as add-on libraries. The team is reducing the operations needed to ship new backends and streamlining unit testing so a single harness can test all ops, such as casting consistency. KerasHub also gains many new models and is shifting its preprocessing from tf-text to PyGrain.
LlamaIndex 🦙@llama_indexOfficialAI score14AILlamaIndex says its LlamaParse agentic parsing tier turned the BEA's Q2 GDP-by-state infographic, which lacks a text layer, into a 52-row table. The post presents this as the tool faithfully recording data from an image-only document by default.
Harrison Chase@hwchase17XAI score38AILangChain's LangSmith Custom Apps lets agent teams build their own review UI over their traces and publish it directly into the workspace. Developers build the interface on their LangSmith data, while hosting, authentication, and permissions are handled by the platform.
Cursor@cursor_aiOfficialAI score42AICursor introduced Rollouts, a tool that writes a monitoring plan and watches changes as they deploy to catch regressions before users see them. When Rollouts detects a regression, it identifies the offending PR and opens an issue, and one click starts a cloud agent to fix it. Rollouts usage credits are included through Oct 3.