# Together AI reports inference compiler gains of up to 2.6x

_Published Saturday, November 14, 2026 at 8:06 AM EST · Science, AI · Latest · Tier 2 — Notable_

Together AI researchers report in a preprint that their Together Compiler delivers up to 2.6x higher tokens per second than the best open source engines on frontier models in the InferenceX benchmark. They say the compiler serves trillions of tokens across the company's production fleet.

The system uses an agent to repeatedly optimize inference, separating engine launch overhead from computation and using a cost model to choose initial parallelism configurations. On the Llama 1B workload, it outperformed highly optimized handwritten Megakernels by up to 1.7x. The reported gains depend on the workload and comparison engine.

## Sources

- [cs.AI updates on arXiv.org](https://arxiv.org/abs/2611.14681)

---
Canonical: https://techandbusiness.org/newswire/SAIb2GVhdWuSSlEtgOsMOn
Published: 2026-11-14T13:06:42.917Z
Story chronology: 2026-11-14T05:00:00.000Z
Retrieved: 2026-11-15T00:01:46.114Z
Publisher: Tech & Business (techandbusiness.org)