Ollama's v0.34.2-rc2 release candidate patches a memory leak in its MLX runner that caused unbounded memory growth during speculative decoding1. The bug stemmed from freed KV-cache buffers never being released: the pool-release check fired only on exact 256-token multiples, but speculative decoding emits multiple tokens per round, causing most rounds to skip the boundary. With qwen3.8:27b-mlx at a 98k-token context on a 128 GB machine, the fix reduces peak memory from over 90 GB to a flat 30 GB.
Ollama Fixes MLX Memory Leak in Speculative Decoding
Ollama v0.34.2-rc2 patches an MLX runner memory leak during speculative decoding, cutting peak usage from over 90 GB to 30 GB at 98k-token context.