QF3 researchers report 10x faster training for humanoid robot policies

Researchers introduce QF3, a reinforcement learning algorithm that they report trains humanoid locomotion and motion-tracking policies with a 10x wall-clock speedup over FPO++. Their preprint also reports training humanoid locomotion policies from scratch and transferring them to hardware without additional training. QF3 combines flow matching with feedback from a critic that evaluates actions. It restricts that feedback to action dimensions close to previously recorded actions, where the estimates are more reliable. The speed result pairs the algorithm with a high-throughput training recipe. Researchers also tested it for refining manipulation policies learned from demonstrations on ABC-Sim and Robomimic tasks.