In-context learning also crosses the sim-to-real gap, zero-shot. Prompts can be formed entirely from simulated experience (e.g., from a scripted policy, an RL agent, or a human teleoperating a simulated robot) and be used to produce behaviors on a real robot. The model was not trained on the task in either the simulator or the real world.
In-context learning transfers zero-shot from simulated prompts to real robots
AISummary
Prompts built entirely from simulated experience, such as from a scripted policy, an RL agent, or a teleoperated simulated robot, can produce behaviors on a real robot. The model was not trained on the task in either the simulator or the real world, so the transfer is zero-shot.
Post on XView on X
Source: Generalist · x.comPublished · added here