Skip to content
Live
Trending storyPast story

DeepSeek open-sources reasoning framework DeepInfer

6 reports6 outlets20 days agoUpdate

The story so far

AI overview

DeepSeek has open-sourced its internal reasoning framework DeepInfer, released under the Apache License 2.0. The framework is used to run its production services and support DeepSeek-V4.1-Flash. The Heart of Machine reports that the accompanying token encoding and decoding components, deepseek-recipe, have also been open-sourced; DeepSeek states that future model code will also be open-sourced. The vLLM team subsequently served DeepSeek-V4.1-Flash on a single NVIDIA GB300 NVL72 rack, claiming performance matched DeepInfer and scaled to the entire rack. vLLM noted that DeepInfer is an offline engine for a single model, while vLLM's testing was for online services, using P/D separation and Wide-EP, with different configurations, so the gap between the two curves cannot be interpreted as an acceleration ratio between the engines. The Heart of Machine reported that after the framework was open-sourced, users could calculate an AI gross margin of 82%, which is the claim in the report and not a verified result.

AI-written from the reports · Updated 19 days ago

Developments

3 developments

  1. 30 Oct, 01:50 · 1 reports
    Baseten and Postman: Building reliable, cost-efficient agent
    Baseten Blog:Baseten and Postman: Building reliable, cost-efficient agents for the enterprise
  2. 23 Oct, 08:00 · 1 reports
    Serving DeepSeek-V4.1-Flash at Scale
    vLLM Blog:Serving DeepSeek-V4.1-Flash at Scale: vLLM Matches DeepInfer and Scales to GB300 NVL72
  3. 20 Oct, 22:31 · 3 reports
    DeepSeek open-sources inference framework DeepInfer
    The Decoder:Deepseek open-sources the inference engine that powers its fastest and cheapest AI model

Coverage timeline

Follow the coverage to see every side of the story.

30 Oct
  1. Baseten Blog
    Baseten and Postman: Building reliable, cost-efficient agents for the enterprise

    Postman's Agent Stack includes four products: Passport, Fabric, Astropods, and Fern, used for preparing APIs, controlling access, and governing agent behavior. Baseten, as Postman's recommended enterprise team reasoning partner, provides fast, reliable, and cost-effective reasoning infrastructure, supporting multiple models such as Beam 1.0 and DeepSeek V4.1 Flash, ranking top in throughput and latency.

23 Oct
  1. vLLM BlogFeatured
    Serving DeepSeek-V4.1-Flash at Scale: vLLM Matches DeepInfer and Scales to GB300 NVL72

    The vLLM team used P/D separation and Wide-EP to serve DeepSeek-V4.1-Flash on a single NVIDIA GB300 NVL72 rack, achieving 954K tokens/s per GPU and 196 tokens/s for single-user P90 on SemiAnalysis AgentX, which is 7.9 times the early single-node results.

22 Oct
  1. Synced
    DeepSeek open-source reasoning framework DeepInfer and token encoding/decoding component deepseek-recipe

    DeepSeek has open-sourced DeepInfer, the foundational framework supporting its large-scale reasoning service, along with the accompanying token codec component deepseek-recipe, under the Apache 2.0 license. DeepInfer is the core of DeepSeek's reasoning service; after the framework is open-sourced, users can achieve an AI gross margin of 82%.

21 Oct
  1. Latent Space
    [AINews] DeepSeek DeepInfer: Open Source SOTA Inference running 155T toks/day/cluster making $22m/day at ~80% margin for a 81 day payback

    DeepSeek releases open-source inference engine DeepInfer, supporting distributed inference, speculative decoding, etc. Single cluster processes 155T tokens daily, 96% input cache hit rate, $38/hour revenue per card, 80% profit margin, 81-day ROI. The engine is based on vLLM and adds caching optimization, but KV cache management and load balancing components are not fully open-sourced. vLLM team reports decoding throughput on GB200 similar to DeepInfer.

  2. The Decoder
    Deepseek open-sources the inference engine that powers its fastest and cheapest AI model

    DeepSeek open-sources its internal reasoning framework DeepInfer under the Apache License 2.0, a service software that supports the large-scale and fast answer generation of the V4.1-Flash model.

20 Oct
  1. MarkTechPost
    DeepSeek Open-Sources DeepInfer, the Inference Framework Behind DeepSeek-V4.1-Flash

    DeepSeek has open-sourced the DeepInfer inference framework, which is used to run its production services and support DeepSeek-V4.1-Flash.