# Together AI introduces compiler that uses agents to optimize inference

_Published Wednesday, November 4, 2026 at 9:18 AM EST · AI · Latest · Tier 2 — Notable_

![Together AI introduces compiler that uses agents to optimize inference — Primary](https://cdn.prod.website-files.com/69654e88dce9154b5f12070c/6aead6f8d7e2304104f29687_TKCC_OG_1200x630-dark.jpg)

Together AI introduced Together Compiler, saying it already serves trillions of tokens and achieved 1.3-3x higher tokens per second than highly optimized open-source inference configurations in its tests. The system uses AI agents to generate optimizations for specific models, hardware and request patterns.

It keeps inference engines running while loading candidate changes, avoiding repeated engine startup. Separate testing engines use different numbers of model layers to balance experiment speed against measurement fidelity.

The reported gains depend on benchmark settings: its Trainium2 evaluation used a synthetic dataset instead of AgentX because the implementation does not yet support higher concurrency within the available memory capacity.

## Sources

- [Together.ai](https://www.together.ai/blog/together-compiler-continuous-compilation-for-ai-inference)

---
Canonical: https://techandbusiness.org/newswire/pz3YkyBv3conhbx89sR6Xm
Published: 2026-11-04T14:18:03.064Z
Story chronology: 2026-11-04T12:00:00.000Z
Retrieved: 2026-11-04T22:05:15.860Z
Publisher: Tech & Business (techandbusiness.org)