Latest
  1. Johns Hopkins Bayview tells 3,182 patients an employee put their health data into a public AI tool
  2. China’s AI compute access expected to double in 2027, with US-designed chips about 65 per cent
  3. UK commits £100m to national AI Biosecurity Lab
  4. Moonshot AI releases Kimi K3.1 Pro with 1-million-token context window
  5. Residents in Newport and Cardiff oppose hyperscale datacentres near homes
  6. Study: Grok 4.7 picked Republicans in 92% of forced midterm answers, GPT-6.1 Sol picked Democrats in 97%
  7. Attackers exploiting the PHP flaws hand the break-in to AI agents, Unit 42 says
  8. Elvis Presley’s rightsholders sue Anthropic over training on 274 songs

Latest / Chips, compute and energy

LaunchComputeFrontierChinaConfirmed

DeepSeek open-sources DeepInfer, the inference framework behind its production services

See how this story connects

Source: DeepSeek (GitHub)Also reported by South China Morning Post

DeepSeek published DeepInfer on GitHub on 20 October 2026 under the Apache 2.0 licence. The README calls it the inference framework behind DeepSeek’s production services and says the release provides offline inference for DeepSeek-V4.1-Flash on NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs, with the company’s own kernels (FlashMLA, DeepGEMM, DeepEP, TileKernels, DeepSelect), draft-token decoding with confidence-scheduled verification, asynchronous batch scheduling, batch-invariant inference, deterministic sampling and structured outputs. A KV cache system, prefill and decode disaggregation and load balancing across replicas are needed for a complete service and are not in the release. The framework was built on vLLM. SCMP reports that a technical article DeepSeek posted on Zhihu the same day puts the revenue from one cluster serving V4.1-Flash on DeepInfer at the equivalent of US$38 per GPU per hour at off-peak prices, and quotes the company saying the release helps the community lower compute costs and energy use.

Why it matters

Providers serving the open-weight V4.1-Flash can run DeepSeek’s own serving code on NVIDIA Blackwell or Huawei Ascend hardware, with the cache and routing layers left for them to build.

When
20 Oct 2026
ID
WTA-N-2026-0349
Where
China
Who
DeepSeek, NVIDIA, Huawei

Summary by wheresthe.ai, from DeepSeek (GitHub) and the reporting credited above. Checked against the source on 21 Oct 2026.

Read the original at DeepSeek (GitHub)

Source copy: download the page text as saved on 21 Oct 2026 (PDF). Also held by the Internet Archive.

Next stop on this Storyline: What happened nextDeepSeek and Huawei narrow AI chip gap with Nvidia, SemiAnalysis says

Cite this story

WTA-N-2026-0349: “DeepSeek open-sources DeepInfer, the inference framework behind its production services.” wheresthe.ai, 20 Oct 2026. https://wheresthe.ai/d/cn-deepseek-open-sources-deepinfer-inference-framework-2026-10/ (source: DeepSeek (GitHub)).

Spotted an error? Email [email protected].