Tencent Hunyuan has released Hy-SimulMT, an open-weight model that translates speech between Chinese and English while the speaker is still talking. It outputs translated text and, with a separate text-to-speech (TTS) service, translated speech in a clone of the speaker's voice. The model card lists the release date as October 26, 2026, and the weights and code are under the Apache-2.0 license.
Live interpretation is a hard case for voice AI. A system that waits for the end of a sentence translates better but makes the listener wait. A system that speaks early must guess where the sentence is going. Tencent's pitch is that Hy-SimulMT cuts the wait without a large loss in quality. The numbers below are Tencent's own tests, not independent results.
What Hy-SimulMT does
The model card describes Hy-SimulMT as "a real-time simultaneous translation model developed by Tencent Hunyuan, supporting bidirectional speech-to-speech translation between Chinese and English." It "continuously receives incoming audio and translates as the speaker talks, producing both translated text and speech."
Key facts from the model card, the GitHub repository and the Hugging Face file listing:
- Languages: Chinese to English and English to Chinese.
- Output: translated text, plus translated speech when a TTS service is set up.
- Voice cloning: real-time, to keep "the speaker's vocal characteristics" in the target language.
- Target uses: "meetings, speeches, and interviews," in Tencent's words.
- Size: about 30.4 billion parameters in the BF16 checkpoint, per the Hugging Face file listing.
- License: Apache-2.0 for the code, parameters and weights. The license file lists one third-party component, Qwen3-ASR-0.6B, also under Apache-2.0.
- Consumer access: Tencent also offers the model in its Hy Translate app.
Tencent published six checkpoints. Three return only the translation (BF16, FP8 and GGUF). Three "Interleave" versions return the source transcript and the translation together, in the same three formats.
The latency claim
Tencent tested the model on RealSI, an open long-form audio benchmark. On that test, Hy-SimulMT reached a StreamLAAL of "approximately 3 seconds, the lowest among the systems shown in the comparison." StreamLAAL measures how far a streaming translation lags behind the speaker. Tencent notes that it "is not the same as the time to first speech output," so 3 seconds is not the delay before a listener hears audio.
On quality in the same test, Tencent says the model's GEMBA score "exceeded those of the compared open-source models and four of the five commercial APIs." GEMBA is a translation quality score where higher is better.
Short-form benchmark scores
Tencent also ran four short-form audio benchmarks. Its table includes commercial and open systems. Selected rows, higher is better:
- Hy-SimulMT: FLEURS 90.3, CoVoST2 75.5, ACL6060 81.2, RealSI-Sentence 84.7.
- Gemini3.5-LiveTranslate: FLEURS 92.4, CoVoST2 74.4, ACL6060 82.6, RealSI-Sentence 82.2.
- Qwen3.8-LiveTranslate: FLEURS 88.4, CoVoST2 82.1, ACL6060 80.5, RealSI-Sentence 84.4.
- GPT-Realtime-Translate: FLEURS 77.7, CoVoST2 50.8, ACL6060 76.4, RealSI-Sentence 76.7.
- SeamlessStreaming: FLEURS 72.8, CoVoST2 60.9, ACL6060 49.8, RealSI-Sentence 56.9.
Hy-SimulMT's 84.7 on RealSI-Sentence is the highest in the table. It is not first everywhere: Gemini3.5-LiveTranslate leads on FLEURS and ACL6060, and Qwen3.8-LiveTranslate leads on CoVoST2.
The human interpreter test
Tencent also compared the model with human interpreters. It collected 18 Chinese and English recordings that appeared online from September 2026 onward, each about 6 minutes long, across ten domains including medicine, law and sports. Six interpreters took part: three with CATTI Level 1 certificates and three with Level 2. For each level, Tencent used the highest score of the three.
Tencent calls this a small-scale capability assessment. It says the model "achieves translation quality on par with expert human simultaneous interpreters, while often responding faster (with lower latency)." It adds its own limit: "These findings apply only to the samples, participating interpreters, and evaluation settings used in this test. They do not constitute a comprehensive assessment of speech quality, voice similarity, or other dimensions."
The card also quotes Wang Weiwei, Vice Dean of the School of Interpreting and Translation Studies at Guangdong University of Foreign Studies: "To control latency, the system starts speaking first and makes corrections afterward; these corrections sound similar to an interpreter's self-corrections." He also said AI simultaneous interpreting is moving from assisting understanding toward usable communication in low-risk settings. Tencent itself writes: "Professional interpreters remain important in scenarios requiring strict accountability, expert review, or high-stakes decisions."
What you need to run it
Self-hosting the open weights takes a GPU server. The card lists:
- A Linux machine with Python 3.12 or newer and an NVIDIA GPU with enough memory for the chosen checkpoint.
- vLLM 0.31 or newer with
HYV3ForCausalLMsupport, or the SGLang-Omni runtime. - For SGLang-Omni, the BF16 weights take "about 57 GiB before KV cache and runtime allocations, so a single H200 is sufficient." That path returns text only, and each uploaded file can be at most 30 seconds.
- llama.cpp support is a submitted pull request (#31198) that is not merged yet. It works through
llama-mtmd-clionly for now.
Tencent ships two tools in the repository. SimulInfer is the inference service, with a web page, audio file uploads and a real-time WebSocket. SimulTrain lets you keep training from a released checkpoint.
The voice part is separate. The card says "speech output and voice cloning require a configured TTS service." The SimulInfer README shows one setup: Qwen3-TTS-12Hz-0.6B-Base served with vLLM-Omni, which "clones the voice from your uploaded audio or microphone input."
Why it matters
Live voice translation is also arriving in paid products. ElevenLabs launched Eleven Realtime Translation on October 21 in alpha for enterprise API customers. Hy-SimulMT puts a comparable task into downloadable weights under a permissive license, for one language pair.
For teams building meeting tools or live captions across Chinese and English, the release gives a self-hosted option. The trade-offs are clear from Tencent's own notes: one language pair, a GPU server (the BF16 weights alone take about 57 GiB), and a TTS stack you assemble yourself. The design also differs from a single speech-to-speech model. Translation and voice are separate stages, closer to the cascaded setup we compared in speech-to-speech vs cascaded voice agents.
Voice cloning of a live speaker also raises consent questions. The model card does not describe a watermark or consent controls, so those fall to whoever deploys it.
For more on this month's voice releases, see our explainers on Mistral's Voxtral 3 voice agent stack, Eleven Realtime and the open Maya 2 Native TTS model.
