PyTorch models can now run on the GPU inside any WebGPU-enabled browser through a new ExecuTorch backend, eliminating the need to convert models to formats such as GGUF, ONNX, or TFLite1. Developers export a model via torch.export to a single .pte file that executes in Chrome, Safari, and other WebGPU browsers as well as natively on Metal, Vulkan, and D3D12.
Performance and model breadth
On an Apple M1 Pro running Google Chrome, the WebGPU path delivered 4–7× faster prefill and 2–10× faster decode compared with ExecuTorch's own WebAssembly + XNNPACK CPU path, benchmarked on quantized Llama 3.2 1B and Qwen3-0.6B. Each data point represents the median of five runs after three warmups, across 128 to 8K tokens of context, with identical token IDs for both backends. The CPU backend uses XNNPACK's 4-bit weights with 8-bit dynamic activations, while the WebGPU backend uses 4-bit groupwise weights with fp32 activations.
The backend has been validated across a broad model roster: Llama 3.2 1B, Qwen2.5-0.5B, Qwen3-0.6B, Gemma 3 1B, SmolVLM, Depth Anything V2, SAM 2.1 Hiera Tiny, D-FINE, Real-ESRGAN, MODNet, Whisper base, and Phonon-31M-EN. That list spans LLMs, vision-language models, segmentation, depth estimation, object detection, super-resolution, and speech-to-text.
Architecture and deployment
ExecuTorch's dynamic C++ runtime compiles to WebAssembly via Emscripten, while emdawnwebgpu bridges WebGPU API calls to the browser's WebGPU JavaScript API. Models export through torch.export() and a WebGPUPartitioner, then execute via WGSL compute shaders. The runtime routes tensor shapes to specialized WGSL kernels at runtime and handles dynamic shapes through allocate-at-max buffers with per-dispatch workgroup recalculation to enable batched prefill. Pipeline compilation, memory management, and dispatch scheduling are handled through the WebGPU backend, with all data staying in GPU device memory within the browser tab.
For deployment, a developer serves the .pte file over a CDN on a web page. Each model's Hugging Face repository includes the .pte files, a model card, and the exact export recipe. An interactive ExecuTorch WebGPU Playground runs many of the listed models on the same WebAssembly + WebGPU runtime.
Privacy and reach
All prompts, photos, camera frames, and audio stay on the user's device; nothing is sent to a server. Browser-side inference also reduces network latency, enables offline use cases, and lowers server-side inference costs.
ANALYSIS By keeping the entire workflow inside the PyTorch toolchain — from training through export to browser execution — the WebGPU backend removes the conversion step that has historically pushed browser-deployed models into separate ecosystems. The 4–7× prefill and 2–10× decode speedups over the CPU path make interactive LLM inference in a browser tab materially more practical, particularly for the 1B-parameter class of models validated here.
The backend is described as still in early development.