Nvidia released Nemotron 3 Diarization on September 23, an open-weight, 100-million-parameter model that identifies up to eight speakers in live or recorded audio and outputs per-speaker activity probabilities at 10-millisecond resolution2,4. Baseten announced same-day serving support with batch, streaming, and diarized-transcription presets, each running on a single RTX PRO 6000 GPU1.
The model replaces the segmentation-plus-embedding pipeline common in prior diarization systems with a single end-to-end pass, labeling speakers at a configurable interval as low as every 320 milliseconds. It is built on Nvidia's earlier NVIDIA Streaming Sortformer architecture but doubles the speaker ceiling from four to eight, including overlapping voices. A 31-layer Transformer encoder with rotary positional embeddings processes 16 kHz single-channel audio converted into mel-spectrogram features, and a Conv1D layer upsamples predictions back to 10-millisecond resolution. Weights are released under the OpenMDW License 1.1, which permits commercial use.
Four latency profiles, one checkpoint
A single checkpoint serves requests at four algorithmic latencies ranging from 0.32 seconds to 30.4 seconds. On DIHARD III, diarization error rate (DER) rose only modestly across profiles: 12.73% offline, 13.18% low, 13.28% very low, and 13.55% ultra-low. On AISHELL-4 at the low profile, Nemotron 3 Diarization achieved a DER of 9.8%, compared with 27.2% for Streaming Sortformer v2.1. In Voice Arena's initial Diarization-Bench results, the model ranked first among 12 systems and 17 configurations, scoring 14.72% DER against 19.3% for the next-ranked system. Nemotron 3 Diarization outperformed Meta Muse Voice Transcribe on all evaluated datasets even at the ultra-low configuration.
Training combined roughly 10,000 hours of real conversations with 82,611 hours of simulated multi-talker mixtures, including real-world multi-speaker audio licensed from David AI spanning 21 languages. Adding the David AI data cut compound DER from 11.19% to 10.42%.
Serving economics and integration
Baseten's optimized serving preset delivered approximately 1.35x higher batch processing throughput than the Nvidia reference setup. A single RTX PRO 6000 can sustain over 500 concurrent hour-long diarization streams; adding transcription via NVIDIA Parakeet V2 ASR, a 600-million-parameter English speech recognition model, reduces that to 190 concurrent streams. The real-time transcription option feeds per-speaker activity vectors into Parakeet V2's initial encoder layers, transcribing overlapped audio channels in a single pass. Over HTTP, one GPU can continuously process 200 six-minute audio files per minute.
Beyond diarization, the per-speaker activity signal can inform voice-activity detection, endpointing, and turn-taking for voice agents. One developer demonstrated the model on a DGX Spark connected to a Reachy Mini robot, using one-second streaming chunks to let the robot ask a new speaker's name and remember it throughout the conversation3.
Nemotron 3 Diarization is available on Hugging Face Inc. and in the Baseten Model Library. For production deployment, Nvidia lists Baseten and DigitalOcean; on-device support is available through Argmax Pro SDK 3. The model runs on Ampere, Ada Lovelace, Hopper, or Blackwell GPUs via Nvidia NeMo. Recordings with more than eight speakers can produce missed or misassigned speech, and heavy noise, reverberation, or far-field capture can raise error rates.