PyTorch reports faster Muse Spark Open inference on four TPU v7 chips

PyTorch reports serving Meta's 816B-parameter Muse Spark Open model on one four-chip TPU v7 host, with an optimized execution path delivering 1.94× faster output per token for a single request than its previous path. The system runs all 62 model layers in one program per device, overlapping weight loads, cached context reads and transfers between devices with computation. It uses SGLang's serving software through the TorchTPU backend. The single-request comparison measured 1.95 ms per output token against 3.78 ms, with both paths using speculative decoding, which checks several proposed tokens together. Those measurements used an earlier Muse Spark checkpoint; the improvement narrows as concurrent requests increase.