Ai2 reports mixed ToyBlocks results for robot models

Ai2 says ManiFlow, π0.5 and MolmoAct 2 generalized poorly to new spatial configurations in its initial ToyBlocks tests.

The company says controlled experiments will test training and model design changes to find which techniques improve generalization.

In a separate report, Ai2 says Molmo 2 and ManiFlow, trained only in the ToyBlocks virtual world, spelled a word with lettered blocks on a physical robot in 15 of 30 attempts. In simulation, the same models succeeded in 111 of 200 attempts. Ai2 says more trials are needed to confirm that simulation-trained models transfer to physical robots.