# ALTO recipe narrows four-bit model training gap in simulated tests

_Published Monday, October 26, 2026 at 12:13 AM EDT · AI, Science · Latest · Tier 2 — Notable_

Researchers report in a preprint that ALTO, an open-source training recipe for the 20-billion-parameter GPT-OSS-20B model, reduced the extra training steps needed with four-bit arithmetic. On the MLPerf Small MoE Training benchmark, the recipe cut the convergence overhead relative to the BF16 baseline from +20.0% to +5.0%.

ALTO combines techniques that limit errors from representing numbers with fewer bits, including a method to reduce oscillation in model weights. The comparison measures steps to reach a validation loss of 3.34. All experiments used simulated MXFP4 kernels on AMD MI300 hardware; wall-clock speed has not been optimized.

## Sources

- [cs.AI updates on arXiv.org](https://arxiv.org/abs/2610.32229)

---
Canonical: https://techandbusiness.org/newswire/RfVt38ErQD6NWptkLLdxQZ
Published: 2026-10-26T04:13:35.374Z
Story chronology: 2026-10-26T04:00:00.000Z
Retrieved: 2026-10-26T07:15:16.746Z
Publisher: Tech & Business (techandbusiness.org)