Robot policy preprint reports lower latency after distilling visual models

Researchers report that All-For-One, a robot control training method, uses over 30x fewer parameters than visual foundation model encoders and achieves 50x lower latency than its own teacher policy in simulation benchmarks. The findings appear in an arXiv preprint. During training, a teacher combines camera images with outputs from multiple visual models that supply information such as depth and segmentation. A lightweight student learns from that teacher and uses only RGB images when controlling the robot. The researchers evaluated the method on two Adroit tasks and five MetaWorld tasks in simulation, plus three real-world manipulation tasks. The reported latency comparison comes from simulation.