CarbonRouter preprint reports 16.1% lower emissions in conversation routing tests

Researchers report that CarbonRouter reduced carbon dioxide emissions by 16.1% on average compared with FairCarbon in trace-driven experiments routing AI conversations across data centers. The preprint evaluated five language models, three GPU types and four regions using two real-world conversation datasets. CarbonRouter accounts for the emissions from rebuilding conversation history when a query moves away from the deployment storing its reusable calculations. It selects a model, GPU type and region, and discourages moves likely to trigger another rebuild on the next turn. Reported savings reached 23.1% for sessions longer than 30 turns. Answer quality and latency also improved, but those measures were predicted and estimated, respectively.