Nvidia reports confidential AI inference retains more than 96% of throughput

Nvidia says confidential DeepSeek-R1 inference retained more than 96% of ordinary output-token throughput on eight B200 GPUs after TensorRT-LLM was adapted for secure execution. Across concurrency levels from one to 16, confidential computing retained 96.1% to 98.2% of baseline throughput, while mean time per output token rose 1.2% to 4.3%. The framework changes how protected host-to-device copies, timing measurements and multi-GPU communications are handled. The controlled test used a long-context, extended-generation workload at low concurrency to expose overhead; results may differ for other models, traffic patterns and configurations.