OpenAI details Jalapeño inference chip benchmarks and deployment timeline

OpenAI presented benchmark results for its Jalapeño inference system at the Hot Chips conference, saying SemiAnalysis' InferenceX test showed more tokens per user and more throughput per kilowatt than currently available state-of-the-art inference processors. OpenAI says the design keeps model state, including the KV cache, local to reduce data movement and communication delays during inference. The company expects very small deployment volumes at the end of 2026 and more significant deployment in 2027.