Top embodied models hit 98% in sim but drop sharply on hardware
Original titleIn simulation, the best models finish easy tabletop tasks about 98% of the time.
AISummary
In simulation, the best embodied AI models complete easy tabletop tasks about 98% of the time. On physical Franka robots, their success rate falls to 24%–72% of simulated performance, and to 13%–60% in a dual-arm setup. Models that look tied in simulation can differ by 30 points on real hardware.
Source: BAAI · x.comPublished · added here