#On-device

AILiquid AI released LFM2.5-2.6B, a 2.6B-parameter agentic model that runs on-device on phones and CPUs, along with a base variant on Hugging Face. The company reports it leads on every instruction-following benchmark and nearly every tool-use benchmark it tested, and decodes 220 tokens/s on an M5 Max. The source says larger models may still suit complex agentic or coding-heavy tasks.

Why it matters: The source reports benchmark results against several same-tier models and notes where larger models still lead, which helps judge fit for edge agent workloads.

AILiquid AI doubled the LFM2 tokenizer's vocabulary from 65K to 128K without retraining from scratch, extending the original BPE merges and initializing new embeddings as the mean of their sub-tokens. The expanded tokenizer needs 4.0× fewer tokens for Thai, 2.6× fewer for Vietnamese, and 2.4× fewer for Hindi, which the source says yields roughly 2.2–3.7× faster on-device decoding for these languages with no reported quality loss on previously supported languages. LFM2.5-8B-A1B and the expanded tokenizer are available on Hugging Face with open weights.

AILiquid AI released LFM2.5-Encoder-230M and LFM2.5-Encoder-350M, bidirectional encoders built on the LFM2 hybrid architecture and available on Hugging Face. They support an 8,192-token context and are designed for fine-tuning on classification and token-level tasks. On CPU, LFM2.5-Encoder-230M is the fastest model tested from 1K tokens up, running about 3.7x faster than ModernBERT-base at 8,192 tokens.

AIThe University of Pittsburgh's RAMMP team is integrating Meta's DINOv3 and SAM models into on-device assistive robotics to detect door buttons, cups, and curbs for navigation assistance. The models run on compact, battery-powered hardware, with optimizations such as reduced memory footprint and lower precision, enabling real-time perception without network connectivity. RAMMP's perception system pairs SAM-based auto-labeling with an RF-DETR detector fine-tuned on DINOv2 embeddings, and the team is now testing voice and touch input for object selection.

AITeam Motif fine-tuned Stable Audio 3.0 on Arabic maqam and built an Ableton plugin for microtonal style transfer. The project won the Stable Audio 3.0 Challenge at Music Hackspace and runs locally on device.

AISebastian Raschka's tutorial shows how to build a fully local coding agent by pairing an open-weight LLM served through an inference runtime with an open-source harness that can read files, edit code, and run commands. He recommends Qwen-Code for Qwen3.6, citing Nvidia's Polar paper, which found Qwen models performed best in Qwen-Code. The Qwen3.6 35B-A3B model is about 22 GB to download and needs roughly 30–40 GB of RAM.

AIPaddleOCR 3.7 adds an ONNX Runtime inference backend, switchable with a single parameter and no code changes. The release supports CPU via OpenVINO and GPU via CUDA and TensorRT, and introduces PP-OCRv6 Tiny, Small, and Medium models that run up to 3.9× faster on CPU.

AIAwni Hannun praised a WWDC video by @angeloskath explaining how to set up agentic AI to run locally with MLX in an approachable way. He said the demos work well now, which was impractical less than a year ago before M5 and recent gains in open-weights models.

AIApple's MLX team released three videos at WWDC covering running agents locally, distributed inference and training, and MLX Swift. The post shares YouTube links for each talk, presented by Angelos Katharopoulos, Tatiana Likhomanenko, and David Koski.

AIGeorgi Gerganov says Computex this year shows a strong signal for local AI, with major players like NVIDIA and Microsoft embracing and discussing local AI workloads. He adds that dedicated consumer hardware and models are on the way.

AIAwni Hannun highlights a workflow to write PyTorch models, export them, and run them with MLX on Apple silicon. The linked PyTorch post says ExecuTorch now includes an MLX delegate that runs PyTorch models on Apple silicon GPUs, supporting LLMs, speech-to-text, and MoE models.

AINVIDIA AI Developer thanked OpenClaw and Ollama, crediting Ollama as the local model-serving engine. The post itself gives no further details about the setup, features, or performance.

AINVIDIA's developer account published a step-by-step tutorial for building a fully local, sandboxed, always-on AI agent. The guide uses OpenClaw with NVIDIA NemoClaw and runs on NVIDIA DGX Spark.

AIOllama and oMLX showed impressive demos running Qwen 3.5 and Gemma 4 locally on Apple silicon using MLX. Awni Hannun said the capabilities of local LLMs and their surrounding ecosystem have advanced considerably over the past couple of years.

AIA blog post and release show how to run YOLO26 locally using MLX. The linked background post describes YOLO26-MLX as a native Apple Silicon port without PyTorch or an external GPU, with up to 2.6x faster inference and up to 1.7x faster training.

AITri Dao says a fast Muon optimizer is coming to consumer cards, since its symmetric matmul kernels work once Blackwell consumer GPU mainloop support is in place. Background from @jcz42 reports Gram Newton-Schulz symmetric kernels now support RTX 5090, with 2x faster Newton-Schulz and 1.7x faster optimizer time on 15 layers of Gemma-4 E2B.

AIBlack Forest Labs released FLUX.2 Small Decoder, a distilled VAE decoder that works as a drop-in replacement for the standard FLUX.2 decoder on Hugging Face. It decodes about 1.4x faster and uses about 1.4x less VRAM at decode time, with ~28M decoder parameters versus ~50M in the full decoder and minimal quality loss. It is available under the Apache 2.0 license and is compatible with FLUX.2-klein-4B, FLUX.2-klein-9B, FLUX.2-klein-9b-kv, and FLUX.2-dev.

AIMistral AI released Voxtral Mini 4B Realtime 2602, a multilingual realtime speech-transcription model with 13 supported languages under the Apache 2.0 license. The model has a configurable transcription delay from 240ms to 2.4s, and it matches leading offline open-source models at a 480ms delay. The source says it is optimized for on-device deployment and is currently supported only in vLLM.

Why it matters: The source specifies the 480ms delay operating point, 4B size, Apache 2.0 license, and vLLM serving path, which matter for teams weighing realtime transcription deployment.