Deploy the artifact, not the framework.
How we run ProtoMotions policies on robots: the whole pipeline exported as one graph, a small C++ loop with TensorRT, the same binary in MuJoCo and on the robot. 1.3 ms per step. https://x.com/i/article/2120178903918444544
Deploy the Artifact, Not the Framework: Introducing the ProtoMotions Deployment Stack
One exported graph, one C++ control loop, and the same binary in MuJoCo and on the robot. This is the runtime behind every real-robot result in Metatron.
ProtoMotions is our open-source framework for training physics-based humanoid controllers, and Metatron is the whole-body policy for the Unitree G1 that we trained with it. Training ends with a checkpoint. Getting that checkpoint to drive a humanoid at 50 Hz is a different job, and one where people often waste a lot of time and can introduce a lot of bugs: observation code rewritten in a second language, a sensor history rebuilt by hand, a normalization constant gone stale.
This post introduces the deployment stack we built so that none of that happens. Every real-robot result in Metatron ran through it: walking, getting up from the floor, teleoperation, sitting on a sofa, all from one frozen policy on a G1. It lives in the ProtoMotions repository under deployment/, Apache-2.0.
What we wanted from it
Four requirements, written down before the code.
- The policy's meaning lives in one place, so the code on the robot cannot drift from the code that trained it.
- Nothing in Python sits in the control loop.
- Switching behaviors adds no branches to that loop. A new behavior is data, not code.
- An LLM should be able to integrate a new policy in a few minutes.
What is the ProtoMotions deployment stack?
Two parts: an exporter that turns a trained policy into a self-describing artifact, and a native runtime that runs it against a simulator or a robot.
The exporter. A trained ProtoMotions policy is more than a network. It is forward kinematics, observation assembly, normalization, a sensor history, the model, and action post-processing with per-joint PD gains. The exporter traces all of it into one ONNX graph and writes a contract beside it: every input and output with its shape and dtype, plus joint order and action map. It can also write a LEAPP bundle, the format NVIDIA Isaac Deploy loads.
The native runtime. A small C++ library compiles the graph to an NVIDIA TensorRT plan, captures it in a CUDA graph and runs it in a fixed-rate loop. Raw sensors and goals go in, joint targets and gains come out. The runtime fills exactly the fields the contract declares, by name. A missing or reshaped field fails at startup, not on the robot. Nothing in C++ knows what an observation means.
The result: the policy on the robot is the graph we evaluated in simulation. The deployment code is optimized for speed and reliability. Introducing a new policy family is a ProtoMotions change with minimal edits needed to the deployment code.
One loop, three calls
Each control cycle is three calls: read one state from the robot backend, step the active controller, validate the command and send it. The backend is the only code that knows the robot SDK. The controller is the only code that knows the policy. A safety supervisor sits between them and the actuators. Everything on that path is preallocated, and the whole policy call, transfers included, is captured in one CUDA graph.
On the Metatron setup, a laptop with an NVIDIA RTX 5090 connected to the G1 over Ethernet, one step of a roughly 100-million-parameter policy takes 1.3 ms from sensor acquisition to joint-command transmission, well inside the 2 ms step of the robot's 500 Hz joint controller. The runtime's own share is about 0.1 ms. While TensorRT and CUDA graphs provide dramatic speedup, these are optional adapters. The core library does not depends on them, nor on any specific robot SDK.
Modular where it matters, fixed where it doesn't
Three plug-in points around a core that never changes.
- Robot backend. Fills a state, accepts a validated command, implements a hard emergency stop. We ship MuJoCo and Unitree SDK2 for the G1.
- Controller. Owns a policy family's contract and goal builder, or wraps any exported artifact. We ship Metatron v1, a GPC prior that samples inside TensorRT, and a generic artifact controller. Controllers switch by name at cycle boundaries.
- Operator interface. Turns a device into one shared vocabulary: Xbox, the Unitree remote, a Pico VR headset for teleoperation. Buttons bind to controllers or to safety events. The safety layer overrides any of these controllers, ensuring operation remains safe and predictable.
A new robot, policy or device is one implementation and one registration. The loop does not change.
The same binary in MuJoCo and on the robot
Whether you evaluate in simulation or on the real robot, it uses the same deployment executable. The MuJoCo backend takes a model file, the Unitree backend a network interface. Everything else is the same object in both runs: task code, sensor history, TensorRT policy, safety supervisor, controller bindings.
That forces a useful discipline. The MuJoCo viewer runs on its own thread and cannot feed the policy. Anything the simulator knows that the robot cannot measure stays out of goal construction. If the robot behaves differently from the simulation, the difference is physics or sensing, not code.
Switching behaviors without switching policies
Metatron is a promptable policy. Each step it reads goals and masks, where a hand should be, how fast to walk, which way to face, and fills in the rest of the body. Changing behavior means writing different fields, not loading another policy.
Each cycle the task clears every goal input and rebuilds it for the active mode: idle, steering, teleoperation or motion tracking. Sensor and action histories are kept, so the policy sees one continuous world while the operator moves from steering to a hand reach to a dance clip. The TensorRT engine and its CUDA graph never change. On the Xbox controller, A selects steering, B toggles teleoperation, LB plays the selected motion clip. A recovery owner runs after the task every cycle: if the torso drops, it masks the task, writes a get-up prompt, and hands control back once the robot is upright.
What the operator sees
The launcher publishes one JSON frame per control cycle over UDP and, on request, to a file. A separate dashboard reads it from outside the control loop: mode, compute time, hand and tracking error, motor temperatures, joint limits, timing, with a tab per task built from whatever keys arrive. Its settings tab is the one way back in. The operator can change the steering speed and turn-rate caps or pick the next reference motion mid-run, each slider marked with the range the policy was qualified for, and the launcher applies the change within a cycle. The same file replays later, labelled RECORDED.
Safety is built into the loop
None of this is specific to Metatron or the G1, and you do not write it yourself. Every cycle the runtime checks that the sensor data is fresh and complete and that the command is within limits. If a check fails, a sensor stream stalls or the gamepad disconnects, the robot goes into a damping hold instead of acting on stale data, and it stays there until a person brings it back. There is no silent fallback to the last command. The same checks run in MuJoCo, so you meet the failure modes in simulation first.
Get started
The code is in the ProtoMotions repository under deployment/, Apache-2.0. The Python exporter and its contract sidecar are the supported path. The C++ runtime under deployment/native is reference code, marked experimental, and some of its examples will move a robot, so read before you run. Export a policy, build a TensorRT plan once per GPU, run it in MuJoCo with the dashboard, then run the same binary against the robot with a different backend flag. The Metatron weights are on Hugging Face, and the project page has the recordings this runtime produced.
Before you run anything on hardware. This is research code, provided as is and without warranty of any kind, and we take no responsibility for damage, injury or loss arising from its use. Test every policy and every change in MuJoCo first. Move to the robot only after it behaves there, with the robot secured and a person at the stop button. Every launcher starts stopped and has to be armed on purpose.
If you deploy learned whole-body controllers and have opinions about any of this, we would like to hear them.