#Deployment/Engineering

AIInferact, working with DeepSeek, NVIDIA, and SemiAnalysis alongside the vLLM community, says joint work across models, custom kernels, and engine serving cuts time to first token (TTFT) by nearly 70% at ~100K throughput. vLLM is the open-source inference engine, and Inferact optimizes it for enterprise production deployments.

AIMicrosoft announced the Surface Laptop Ultra, its first device using the Nvidia RTX Spark SoC, starting at $2,599 with up to 128GB of LPDDR5x unified memory. It ships October 16 and is available for preorder now. The article says the unified memory approach lets the laptop handle both gaming and local AI development and deployment.

AIGoogle's AI Edge Team open-sourced ML Drift under Apache 2.0, a GPU compute engine for on-device AI inference across OpenGL ES, OpenCL, Metal, and WebGPU. It serves as the core GPU acceleration engine within LiteRT and succeeds the legacy TFLite GPU delegate, which will no longer receive new features. The post cites benchmarks showing up to 40% lower frame latency in YouTube Shorts and up to 30% faster on-device performance in Adobe Lightroom and Photoshop.

Why it matters: The post explains how ML Drift unifies GPU shaders across platforms and replaces the TFLite GPU delegate, which matters for developers deploying on-device models.

AIGoogle Developers Blog introduces AQuA, an ambient quality agent that runs in a customer's Google Cloud project and samples production sessions to find recurring agent failures. In a 32-session travel-concierge sweep, it verified six issues and traced two of them to specific prompt lines, and a replay after the fixes raised full-session passes from 5/32 to 13/32. The post notes that verification and diagnosis are model-based, and that the tool proposes edits without applying them.

Why it matters: The post walks through a concrete production workflow, from sweep and verification to a code-anchored fix and replay, that shows how to diagnose silent agent failures.

AIMeta's Head of Infrastructure, Santosh Janardhan, discusses the company's approach to building infrastructure for AI in a conversation with Tom Shaw. The discussion covers why Meta views itself as more than a software company, why AI differs from other technologies, and why data centers are essential to AI development. It also addresses power for Meta's AI infrastructure, gigawatt-scale energy needs, chip selection, and the benefits of building its own data centers.

AIOpenAI announced on X that GPT-6 and Intelligent UI are now rolling out to all ChatGPT users, after GPT-6 Astra, Sol and Luna were previously limited to ChatGPT Work and Codex. Intelligent UI lets GPT-6 combine text, images and interactive elements such as charts, clickable buttons and forms, with a mahjong learning example shown.

AIA Highwire survey of 1,034 corporate employees found 78% of managers used AI to help draft, edit, summarize, or inform performance reviews in the past year. Fifty-four percent of employees said feedback became more specific and actionable, while 34% found it more generic and 32% said it was less useful. Nearly one in four employees rehearse difficult workplace conversations with an AI tool, according to the survey.

AISchneider Electric will acquire industrial design and data management firm PTC Inc. for about $22.6 billion, aiming to connect product design with operations through a digital thread. Interact Analysis says the deal could narrow Schneider's portfolio gap with Siemens, but integration with AVEVA and Cognite and the 42.3% premium weighed on Schneider's shares after the announcement.

AIVercel says Browserbase Search and Fetch tools are now available on AI Gateway, letting any model with tool calling search the web and read pages. Browserbase presents the tools as a way to reliably search and extract page contents through an existing Vercel plan.

AIEthan Mollick had early access to Intelligent UI and found it a welcome change from long blocks of text. He suggests interfaces will increasingly be built on demand for each user's problem. The quoted OpenAI post says GPT-6 and Intelligent UI are rolling out in ChatGPT for everyone, delivering fast, interactive answers with visual explanations and task tools.

AIPerplexity's open-weights multimodal decision model, Decider v1.1, is now available on OpenRouter. It accepts text, JSON, or images and returns typed answers with probabilities, priced at $0.02 per million input tokens with free output. Perplexity says it scores highest on Hugging Face's new Decision Index 0.3 benchmark and costs half as much as v1.

AIEnvato has launched Burst Mode, which starts from a rough idea or reference image and generates up to six image directions side by side. Users can steer those directions with moodboards and a creativity control ranging from focused to wild. Envato says it can produce up to six directions at once, up to 10× faster, for one AI credit.

AITogether with DeepSeek's model and kernels and NVIDIA's collaboration, vLLM reports that time to first token (TTFT) drops nearly 70% at roughly 100K throughput. The post credits Inferact and the vLLM community for the work and thanks SemiAnalysis for AgentX.

AIvLLM has integrated DeepSeek's MegaAttention, which uses NVFP4 KV cache and is 45% smaller, alongside Mega-mHC, Mega-Gate, and DeepSelect kernels. Its own fusions include a CuTe-DSL fused WO-A that lowers ITL by up to about 6–7%, and sparse MQA logits that run 14–23× faster per layer at 512K context.

AIvLLM's bounded sliding-window attention (SWA) replay reruns only each request's last 128 tokens after a prefix hit, instead of replaying 40 × 128 tokens. With CUDA graphs, prefill compute drops 30–40% for layers 21–39.

AIThree weeks after launch, vLLM's serving of DeepSeek-V4.1-Flash runs 1.9× faster at low concurrency and delivers 5.3× the throughput at 150 TPS per user on SemiAnalysis AgentX. The post offers interactive figures explaining how these gains were achieved.

AIAnthropic's Claude Haiku 5.5 is now generally available in GitHub Copilot, a lightweight model built for fast, high-volume work such as subagents, quick edits, and terminal tasks. GitHub's early testing found it matched Claude Sonnet 5 on many coding tasks while using significantly fewer tokens and steps. It can be used in the GitHub Copilot app, CLI, or @code.

AIAnthropic has released Claude Haiku 5.5, its cheapest and fastest small model, priced at $0.10 per million input tokens and $0.50 per million output tokens for prompts up to 100K tokens. It keeps a 1M token context window, up to 128K output tokens, and is generally available on the Claude API, Amazon Bedrock, Google Cloud, Microsoft Foundry and Claude Platform on AWS. Anthropic reports 72.4% on OSWorld 2.1 (offline subset) versus 15.7% for Haiku 4.5, and the article notes that non-default temperature, top_p or top_k values return a 400 error.

AIOpenAI's GPT-6 Luna Decisions model is now available on OpenRouter, letting apps choose the right model, tool, or action from text, JSON, or images. It returns typed answers with probabilities, costs $0.10 per million input tokens, has a 1M-token context window, and output is free.

AIA Google Research field experiment with 133 patent lawyers found AI tool access raised drafting scores by 0.34 to 0.38 standard deviations over three months. When the tool was removed for a redlining task, only senior lawyers kept an advantage of 0.45 SD, while junior lawyers showed no discernible improvement. The authors argue that tools which boost current output must not stop junior professionals from building the judgment that senior experts rely on.

Why it matters: The field experiment separates AI's short-term productivity gains from skill retained after the tool is removed, which matters for training junior professionals.

AIAnthropic's lightweight Claude Haiku 5.5 is now generally available in GitHub Copilot for fast, high-volume tasks such as subagents, quick edits, and terminal work. In early testing, it matched Claude Sonnet 5 on many coding tasks while using significantly fewer tokens and steps. The model is billed at provider list pricing under usage-based billing and is available to Copilot Pro, Pro+, Max, Business, and Enterprise users.

AIReplit is building a powerful desktop app with a focus on security and reliability, citing supply-chain attacks and catastrophic agent mistakes as risks of desktop AI apps. The company is partnering with Microsoft and will be an early adopter of Nvidia's OpenShell. A quoted Replit post says the desktop preview runs builds locally on Windows, with each build in its own sandbox powered by Microsoft Execution Containers and OpenShell, and offers a waitlist.

AIAWS has open sourced Strands Box, a local sandbox for AI agents. The author points readers to a blog post by Marc Brooker that covers design tradeoffs, how OS-level containment and Dogwood enable richer policies, and AWS's investment in agent safety.

AIVercel says OpenAI's Decisions API is live on AI Gateway, answering yes/no, choice, and score questions for any decision model. Developers can access it through the gateway using the model identifier openai/gpt-6-luna-decisions.

AILocal sandboxing for GitHub Copilot is now generally available. It lets Copilot run commands in an isolated environment with controlled access to files, networks, system capabilities, and credentials. Enterprise teams can also centrally manage policies, and the feature is available in GitHub Copilot CLI, the GitHub Copilot app, and @code.