Meta details Prometheus training capacity across hundreds of thousands of GPUs

Meta says its Prometheus supercluster in New Albany, Ohio, now connects hundreds of thousands of GPUs across multiple buildings and has trained its Muse foundational models. Its first phase is complete, and the company is working toward roughly 1 million GPUs at full scale. A dedicated Ethernet network connects GPUs within and between buildings, allowing the site to operate as a single training system. Meta says the capacity supports larger models and longer context windows than were previously practical. The design also reserves hardware for failures: each 72-GPU domain runs a 64-GPU training slice and keeps eight GPUs as spares. Meta says a 32,768-active-GPU training run requires provisioning 37,656 GPUs.