ALTO recipe narrows four-bit model training gap in simulated tests

Researchers report in a preprint that ALTO, an open-source training recipe for the 20-billion-parameter GPT-OSS-20B model, reduced the extra training steps needed with four-bit arithmetic. On the MLPerf Small MoE Training benchmark, the recipe cut the convergence overhead relative to the BF16 baseline from +20.0% to +5.0%. ALTO combines techniques that limit errors from representing numbers with fewer bits, including a method to reduce oscillation in model weights. The comparison measures steps to reach a validation loss of 3.34. All experiments used simulated MXFP4 kernels on AMD MI300 hardware; wall-clock speed has not been optimized.