vLLM has detailed Encoder / Prefill / Decode (EPD) disaggregation, an architecture that separates multimodal encoding from language-model prefill and decode into independently deployed instances, yielding latency and throughput gains across image, video, and audio workloads1.
The core capabilities are included in vLLM v0.32.0, with additional DeepSeek-V4.1-Flash metadata-only support available on main.
How EPD works
Multimodal inference requires an encoding stage — images, video frames, and audio must be processed into embeddings before the language model can consume them. Demand on this stage varies with media size, modality, and request mix, and can shift independently of prefill and decode demand. EPD breaks these stages apart so each can scale on its own hardware pool.
The post describes five engineering changes driving the work. Metadata-only requests let downstream instances consume remote embeddings without repeating media preprocessing. A dedicated Model Runner V2 path isolates encoder-only execution in a new MMEncoderModelRunner, with encoder-only CUDA Graph support and reduced checkpoint-loading I/O. On a pruned Kimi-K3 checkpoint, the encoder read 2 of 96 shards and weight loading dropped from 683 seconds to 0.31 seconds. Remote embedding transfer is handled through NIXL and Mooncake integrations, while Mooncake Store enables cross-encoder output reuse.
Performance numbers
At concurrency 64–128, EPD achieved P90 time-to-first-token (TTFT) that was 44–47% lower than an eight-GPU prefill/decode baseline, with 6.6–9.5% higher input-plus-output throughput per GPU.
In a shared-reuse experiment, P90 TTFT decreased from 1.656 to 0.891 seconds and P50 TTFT from 1.209 to 0.819 seconds. Across 128 measured requests containing 512 images, shared reuse reduced repeated image encodings from 512 to zero. That controlled experiment used six H20 GPUs, InternVL3_5-38B-HF, a 2E+4PD topology, and same-node Mooncake TCP.
In a 30-frame MMVU evaluation, end-to-end throughput improved by approximately 94–115% over three runs on separate 4-GPU GB200 nodes.
For audio, metadata-only forwarding on the MMAU test-mini benchmark reduced mean latency from 980 to 853 milliseconds and increased throughput from 62.21 to 73.13 requests per second, with accuracy changing from 75.90% to 75.65%.
Orchestration and deployment
The post describes three orchestration paths. vLLM Router supports HTTP E+PD and E/P/D deployments with metadata-only forwarding, multi-image distribution across encoders, and data-parallel prefill/decode consumers. SMG supports E/P/D deployments over gRPC, including Mooncake and NIXL embedding-transfer backends. The llm-d team has collaborated with the vLLM community to provide a Kubernetes-native deployment path, with the llm-d Router selecting encoder pods and a routing sidecar or Coordinator distributing encode requests across them for parallel multi-image processing.
Deep Seek's inference deployment describes an E/P/D architecture with a separately deployed vision encoder; vLLM has supported encoder disaggregation since a December 2025 architecture post. MiMo-V2-Omni now declares the embedding fields needed for image EPD, and XPU support includes encoder-only producer runner integration.
ANALYSIS The latency and throughput results position EPD as a practical lever for operators whose multimodal traffic is bursty or image-heavy — encoding can now autoscale without over-provisioning decode capacity. The Kubernetes-native llm-d path lowers the barrier for production clusters already running vLLM. Competing serving frameworks such as SGLang, which recently demonstrated large-scale TPU serving[3], face pressure to match this stage-level disaggregation for multimodal workloads.