RPEG preprint reports faster reasoning-model training across 256 GPUs

Researchers report that a distributed runtime called RPEG improved reasoning-model training performance by up to 2x over ReaL in tests on systems with up to 256 GH200 GPUs. Their preprint also reports up to 2.48x lower step latency than verl and up to 30% higher hardware utilization. RPEG represents a reinforcement-learning training pipeline as a graph, allowing the system to optimize both its structure and execution. It combines choices about asynchronous work and model sharing with stage fusion, scheduling and model parallelism. An analytical performance model accounts for computation, communication and memory across racks, nodes and GPUs. The reported gains come from the tested systems and comparisons in the preprint.