# DeepSeek releases inference framework for Nvidia and Huawei hardware

_Published Tuesday, October 20, 2026 at 7:07 AM EDT · AI, Infrastructure · Latest · Tier 2 — Notable_

DeepSeek has released DeepInfer under the Apache License 2.0, making the inference framework behind its production services available for offline use. The release supports DeepSeek-V4.1-Flash on Nvidia SM100 GPUs and Huawei Ascend 950 processors.

Features include generating draft tokens to accelerate decoding, overlapping CPU scheduling with device execution, and constraining outputs to a schema or grammar. Nvidia users can also enable consistent results across different batch compositions and deterministic sampling. The release does not include all components needed for a complete inference service: operators must supply a system for caching model state and load balancing across replicas, among other service infrastructure.

## Sources

- [GitHub](https://github.com/deepseek-ai/DeepInfer)

---
Canonical: https://techandbusiness.org/newswire/mcBzQo3bLzpXAuuqgdhtoW
Published: 2026-10-20T11:07:57.428Z
Story chronology: 2026-10-20T09:08:54.000Z
Retrieved: 2026-10-20T14:14:10.175Z
Publisher: Tech & Business (techandbusiness.org)