Open-source SGLang plugin adds inference serving support for IBM Spyre

Developers working on IBM's Spyre AI accelerator have made an open-source plugin available that connects the hardware to SGLang, software for serving AI models. In a PyTorch blog post, the team reported approximately 3.6× the throughput of its reference PyTorch inference implementation using torch-spyre. The two-person team built the integration in less than a week with AI coding agents, reusing existing PyTorch hardware support. SGLang's prefix sharing, which reuses processing for shared prompt beginnings, worked without Spyre-specific changes. Performance matched vLLM across the measured workloads for Granite 3.3 8B and Gemma 4 26B A4B; installation requires the plugin and compatibility patches.