IBM Spyre Accelerator Compiler Shrinks 7x After Shift to PyTorch Inductor

IBM's torch-spyre team replaced its standalone sendnn compilation stack with an extension built on PyTorch's Inductor backend, reducing the compiler from roughly 273,000 lines of C++ and Python to approximately 38,600 lines of Python¹. The Spyre accelerator, a 75-watt PCIe-attached inference chip built from 32 cores with 2 MB scratchpads each, required out-of-tree extensions in three areas: tiling, scratchpad allocation, and work division. Optimizations reported include a 2x kernel improvement over sendnn-compiled equivalents on fused MLP-style workloads and a 27% reduction in median softmax kernel runtime from scratchpad-aware scheduling. Device-specific decomposition work contributed upstream improvements landing in PyTorch 2.12.