DeepSeek released DeepInfer, the high-performance inference framework that powers its production-scale model deployments, as an open-source project under the Apache-2.0 license on October 20, 20262. The release puts the same serving software the lab runs internally into the hands of outside developers and businesses1.
DeepInfer is built specifically for DeepSeek models and currently drives the lab's DeepSeek-V4.1-Flash deployments. Bloomberg reported that the framework is designed to accelerate inference, optimize chip performance, and reduce energy and compute costs.
The framework bundles custom kernels, distributed parallelism, speculative decoding, and structured output support. Custom kernels are low-level routines tuned to extract more throughput from specific hardware. Distributed parallelism splits model workloads across many processors simultaneously, and the framework includes MoE expert parallelism suited to mixture-of-experts architectures that route queries to specialized sub-networks rather than activating the full model.
Speculative decoding lets the system predict several tokens ahead and verify them in batch; when predictions are correct, output arrives faster. Structured output support keeps responses in predictable formats such as JSON.
Alongside DeepInfer, DeepSeek released deepseek-recipe, a token encoding tool.
The release arrives the same day MiniMax shipped its 428-billion-parameter M3.1 Flash model, which scored 74 on the DeepSWE benchmark, just behind the 77 posted by DeepSeek-V4.1-Flash-1017[1]. ANALYSIS By open-sourcing the serving layer that runs DeepSeek-V4.1-Flash, DeepSeek gives third parties a path to deploy the model at production scale on their own infrastructure, extending the competitive pressure its Flash-tier models already exert on rivals through benchmark performance.
Bloomberg reported that the open-sourced software could make DeepSeek's technology cheaper and easier for businesses to operate.