Microsoft details Maia 300 chip design for larger AI inference workloads

Microsoft disclosed specifications for its future Maia 300 accelerator as the Open Compute Project Summit opened this week. The chip targets AI inference, the computation that produces model outputs, and larger workloads in which multiple models share memory and network capacity. The design moves from monolithic chips to a package of seven chips and eight high-bandwidth memory stacks, separating input-output and memory controllers from compute and cache components. It provides 288 GB of HBM4 memory and 18 TB/sec of memory bandwidth. Microsoft also changed its accelerator network to fit more devices into racks and interconnected groups. Maia 200, rather than Maia 300, is the generation ramping into Azure production this year.