TAPESTRY preprint reports robot control without generating video at runtime

Researchers report a robot policy that repurposes a pretrained video model to generate actions without producing video during operation. Their TAPESTRY preprint reports action chunks generated in 84 ms on consumer hardware, with experiments in simulation and on real robots. The system represents robot actions as compact tokens processed by the video model's existing transformer. Training combines video learning, joint video-and-action learning, and action-only learning. At runtime, it generates an action sequence in a single denoising step. The authors report substantially lower inference cost than comparable policies that jointly generate video and actions. The result concerns evaluated robot experiments, rather than a deployed commercial system.