NVIDIA developers are pairing frontier AI models, including GPT-6 Astra and Claude Fable 5, with Omniverse libraries to turn simulation ideas into working applications. Examples include a humanoid warehouse simulator, an autonomous-driving testing workflow, and sensor-matching digital twins. The projects are guided through natural-language instructions and reviewed by developers.
Developers are pairing frontier AI models with NVIDIA Omniverse libraries to turn simulation ideas into working applications, from humanoid warehouse simulators to autonomous-driving test environments. In the examples, developers direct AI agents through natural-language instructions and review results, while Omniverse provides GPU-accelerated physics, rendering and sensor simulation. One experiment reported a simulated Unitree G1 humanoid clearing a hurdle in 64 of 100 trials.
NVIDIA's Omniverse libraries, guided by SimReady Foundation specifications and agentic NVIDIA skills, provide a structured workflow for converting CAD assets to OpenUSD for robotics simulation. The workflow covers configuring and validating materials, collision geometry, joints, and other physics properties before testing robot behavior.
NVIDIA's technical blog describes how its team taught robots to assemble GB300 tester trays, a task that currently requires skilled manual labor in factories. The post also discusses lessons about robot learning, mechanical intelligence, and engineering. The source excerpt provides limited detail beyond this.
Microsoft Research Asia – Singapore, opened July 24, 2025 as Microsoft's first Southeast Asian research lab, reports progress after its first year. The lab's work spans next-generation AI models and agentic systems, domain-specific AI for real-world impact, AI-native research practices, and ecosystem and talent development. Its healthcare collaborations on multimodal and agentic AI for clinical decision-making are being deployed through partnerships across Singapore's healthcare ecosystem.
Microsoft Research reports that running physical AI inference on onboard GPUs can limit robot performance and battery life, while offloading inference to edge or cloud GPUs improved results in mobile manipulation tests. In its evaluation, smaller onboard GPUs slowed mapping and planning by up to 383% compared with an A100, and large onboard GPUs such as Jetson Thor drained robot batteries by up to 160%.
AIWhy it matters: The study measures how offloading robot inference to edge or cloud GPUs changes task success, battery life, and model size, offering evidence for infrastructure design.
Black Forest Labs released FLUX 3 Action, an open-weights 7B world action model that outputs robot joint commands from camera frames, robot state, and a text instruction. On the RoboLab-120 benchmark it reports 42.92% task success, ahead of Cosmos3-Nano-Policy at 36.8% and π0.5 at 28.0%. The model is fine-tuned on DROID, is distributed under the FLUX Kommunity License v.1.0, and runs in about 32 GB of GPU memory in bfloat16.
AIWhy it matters: The model card gives a benchmark comparison, parameter counts, and an action contract, so readers can judge how it compares with existing robot policies.
Black Forest Labs has published FLUX 3 Action SO-101 on Hugging Face as an open-weights 7B world action model. It takes two camera frames, the robot state, and a text instruction, then returns the next 42 actions with predicted video frames, with 32 executed at 30 Hz before replanning. The card also provides a rank-32 LoRA fine-tuning recipe for user datasets and states that the application must enforce joint velocity, force, and workspace limits.
Black Forest Labs has released flux-3-action-base, an open-weights 7B world action model that takes camera frames, robot state, and a text instruction to output the next action chunk. The release is an adaptation component rather than a complete robot policy, and new embodiments require their own action heads. The source says the weights are paired with shared video VAE and Qwen3-VL-4B-Instruct text encoders and is governed by the FLUX Kommunity License v.1.0.
AIWhy it matters: The source separates the adaptation base from full robot policies and states the shared encoders and new-embodiment requirements, which clarifies what developers must still build for their robots.
NVIDIA released FoundationPose, a transformer-based model for 6-DoF object pose estimation and tracking that works on novel objects at test time without fine-tuning, given a CAD model. It takes RGB and depth images, a 2D bounding box, a CAD model, and camera intrinsics as inputs, and is licensed under the NVIDIA Open Model License for commercial use. The model is trained on synthetic data from Objaverse and Google Scanned Objects, with evaluation on LINEMOD and YCB-Video.
NVIDIA released EgoHand-1.0, a 883.5M-parameter DINOv3-based transformer that predicts SOMA hand pose, MHR shape coefficients, and camera translation from a single 256×256 hand crop. The model is evaluated on the HOT3D egocentric benchmark and is intended for research and demonstration rather than production use. Its outputs can supply hand trajectories for training robotic manipulation policies, and it runs on NVIDIA Ampere GPUs under Linux with PyTorch.
Anthropic is building the Model Hardware Standard (MHS), a common way for AI models to connect to lab and manufacturing equipment and operate it with safety limits built into each device. MHS started as a collaboration between Anthropic and HHMI Janelia Research Campus and is launching as a research preview with partners across science, robotics, and manufacturing.
AIWhy it matters: The source describes a standard for connecting AI models to lab and manufacturing hardware, which matters for anyone building automated experimentation workflows.
The University of Pittsburgh's RAMMP team is integrating Meta's DINOv3 and SAM models into on-device assistive robotics to detect door buttons, cups, and curbs for navigation assistance. The models run on compact, battery-powered hardware, with optimizations such as reduced memory footprint and lower precision, enabling real-time perception without network connectivity. RAMMP's perception system pairs SAM-based auto-labeling with an RF-DETR detector fine-tuned on DINOv2 embeddings, and the team is now testing voice and touch input for object selection.