Red Hat Benchmarks KV Cache Offloading on H100 GPUs in OpenShift AI 3.5

Red Hat published benchmarking results for tiered key-value cache offloading on NVIDIA H100 GPUs in OpenShift AI 3.5, targeting long-context enterprise workloads¹. The post identifies agentic applications, retrieval-augmented generation, and multi-turn assistants as drivers of growing context lengths, where models repeatedly process large portions of the same prompt. Technical details of the benchmarking methodology and performance results were not disclosed in the available summary.