Ray Serve LLM

KV cache becomes programmable infrastructure across the inference stack

vLLM consolidates as default LLM serving layer across chips, clouds, and consumer GPUs