NVIDIA opens the weights for Nemotron 3 Diarization to label voices

NVIDIA has released Nemotron 3 Diarization, a model for labeling audio recordings by voice, into the open. It answers the question of who spoke and when, and is equally suited to finished files and to live streams.

This kind of labeling is needed wherever there are transcripts. The words may be rendered correctly, but without attribution to a speaker it is unclear who made a commitment and who objected. In transcripts of meetings, calls, and podcasts, that is often more important than the accuracy of the text itself.

The model weighs about 100 million parameters and distinguishes up to eight participants. In offline mode it relies on a buffer of 30.4 seconds; in streaming, latency can be reduced to 0.32 seconds. Utterance boundaries are refined in 10 ms steps, and recording length is unlimited when processing in chunks.

The labels are anonymous: "speaker 2" instead of a name. The model only separates voices; linking them to specific people has to be done separately. Channels are ordered by when a voice first appears in the recording, so the numbering does not get thrown off between segments.

The weights are open, and the model can be run locally. The OpenMDW 1.1 license also permits commercial use. There is a lightweight native runtime, NeMo-Speech.cpp: a single command can label a file or add speaker labels directly into a transcript.

Integration with streaming speech recognition is described separately. In that case, speaker labels arrive together with the text rather than in a separate pass after it. This will be useful for transcription services and for analyzing calls.

Image: Kevin McCarthy CC BY 2.0 · License

Translation editor: Салтанов А.