Buoyant releases Kinference to manage self-hosted AI inference

Buoyant announced general availability of Kinference, open source software that routes and manages AI inference requests across models and GPUs running on Kubernetes. Available under the Apache 2.0 license, it builds on the vLLM inference engine. Platform owners can set cost and compliance policies, assign usage costs to teams, and send requests to cloud providers under defined conditions. Routing considers cached model data and queue status to reduce GPU work. Buoyant reports latency improvements against standard Kubernetes load balancing, measured on one workload type.