Robot study recovers better actions from model-generated video

Researchers report in an arXiv preprint that recovering robot actions from generated video improved a Unitree G1 humanoid's performance over the model's native action predictions. Their recovered-action method reached approximately 42% pick-and-place success, compared with 7% for native actions. AutodidactWAM estimates hand poses from generated video and translates them into robot movements. Those recovered actions then supply targets for retraining the model's action layers, without additional task-specific teleoperation after an initial adaptation to the robot. A combined training objective achieved 20% full-task success on the training object and 30% on a held-out object. A preference-only training method achieved 0% success despite 1.000 validation preference accuracy, showing that the training combination mattered.