Google and Meta have released TorchTPU, an open-source backend that runs PyTorch natively on Google TPUs, replacing the earlier PyTorch/XLA integration that relied on Lazy Tensors and struggled to track PyTorch's rapid evolution1.
The project, started at Google in mid-2025, is already in production for inference and training workloads serving some of Google Cloud's largest customers.
Architecture and execution modes
TorchTPU supports both eager and compiled execution, including torch.compile integration. It provides three eager modalities: Strict, Debug, and DeferAndFuse. DeferAndFuse is reported to be 50% or more faster than Strict Eager while reducing peak HBM memory. The backend also supports bounded dynamism for dynamic shapes when maximum bounds are known at compile time, and offers a three-tier compilation cache spanning process-local in-memory, machine-local shared memory, and persistent remote-file-system-backed storage.
On the operator side, TorchTPU added support for roughly 450 Aten ops, covering forward and backward passes for all Core ATen ops required by major deep learning architectures. Custom kernel authoring is available through Pallas and Helion.
Compatibility and ecosystem reach
TorchTPU validated out-of-box execution on 4,917 distinct open-source models across 500-plus architectural families. It achieved a 98% Tier 1 pass rate (4,821 of 4,917 models) and a 97% Tier 2 numerical parity rate (638 of 657 models). Bounded-dynamism support passed correct numerics on 834 of 839 models.
Integration has been validated with vLLM, SGLang, TorchTitan, Ray, and TorchRec. Distributed training is supported through DTensor, FSDPv2, SimpleFSDP, and FairScale via Process Group API collectives, and multi-host inference and fine-tuning workloads have been validated with Ray-based orchestration.
Performance benchmarks
On training, TorchTPU reported 100% performance parity with Google's JAX baseline on DLRM v2, matching an 18.3 ms step latency. For Qwen3 235B MoE on 8,192 TPU v7x chips at 4K sequence length in BF16, it reported 28% MFU. Llama 3 405B on 1,024 TPU v7x chips reached 500 tokens per second per chip and above 50% MFU. A Pallas SplashAttention kernel delivered a 17.5% speedup on diffusion attention blocks.
On inference, TorchTPU achieved 1,828 tokens per second on a 64-TPU-v7 setup for Qwen3 MoE 235B, compared with 1,937 TPS for JAX/MaxText on the same configuration. Single-chip inference on a TPU v7 reached roughly 25,700 TPS with 59% MFU at sequence length 5,120. Cost benchmarks came in at 0.83 cents per million tokens on AgentX and 2.01 cents per million tokens on InferenceX 8K/1K.
TorchTPU is planned for open-source release across four repositories at the October 2026 PyTorch Conference.
ANALYSIS By delivering near-parity or matched performance against JAX on Google's own hardware, TorchTPU removes a key friction point that has kept PyTorch-native teams from adopting TPUs at scale. The breadth of validated models and framework integrations positions the backend as a drop-in option rather than a porting project.