NVIDIA has released Magpie Multilingual TTS, a 364-million parameter open-weights text-to-speech model engineered for low-latency conversational AI agents. Released under the NVIDIA Open Model License, the model is available as open checkpoints on the Hugging Face Hub and as an optimized microservice container within NVIDIA NIM.
The release expands language support to 12 languages: English, Spanish, French, German, Italian, Vietnamese, Mandarin, Hindi, Japanese, Modern Standard Arabic, Korean, and Brazilian Portuguese.
Frame-Stacked Local Transformer Architecture
In cascaded voice agent architectures, text-to-speech serves as the final pipeline stage before audio output. Magpie is structured to operate within sub-200ms end-to-end latency budgets by reducing Time to First Audio (TTFA).
To minimize generation latency without sacrificing acoustic fidelity, the architecture pairs two core techniques:
- Frame Stacking: The decoder predicts two discrete audio frames simultaneously at each decoding step. This halves the total number of autoregressive decoding iterations required per utterance.
- Local Transformer Refinement: Because simultaneous frame generation can introduce codebook token discrepancies, a localized transformer block models intra-frame dependencies and refines acoustic features before waveform reconstruction.
The model also incorporates International Phonetic Alphabet (IPA) grapheme-to-phoneme mapping and custom pronunciation dictionaries to support code-switching in mixed-language dialogues.
Hardware Benchmarks and Concurrency Scaling
According to on-premise benchmarks across NVIDIA GPU architectures, Magpie achieves the following latencies:
- Single-Stream Generation: Time to First Audio measures 32ms on Blackwell B200 (12.1x real-time throughput), 47ms on H100 (14.7x), 53ms on DGX Spark (9.8x), and 79ms on A100 (12.2x).
- Concurrent Scaling (64 Streams): Under a 64-stream concurrent load, B200 registers a TTFA of 239ms while achieving 319.8x real-time throughput. On H100, 64-stream TTFA reaches 275ms at 290.8x real-time throughput.
Acoustic evaluation benchmarks demonstrate reductions in Character Error Rates (CER) and improvements in Speaker Similarity (SSIM):
- Spanish: CER reduced from 1.14% to 0.60%, with SSIM increasing from 0.715 to 0.793.
- French: CER reduced from 2.70% to 1.54%, with SSIM improving from 0.703 to 0.747.
- New Languages: Modern Standard Arabic records a 1.62% CER baseline, Korean records 2.69%, and Brazilian Portuguese records 2.91%.
Magpie Multilingual TTS is integrated into NVIDIA's Nemotron Voice Agent reference architecture for on-premises and air-gapped enterprise deployments.
