A team at Alibaba has released an open preview of TaoMate-HD, a method for making a talking digital person on video, one short chunk at a time, with speech and picture generated together. It sits on top of MiniMax H3, the open model that makes video and sound together. The paper was submitted to arXiv on November 2, 2026, and the code and preview weights were released on November 3. This post explains what the method does, what the authors measured, and what to keep in mind before you try it. Every figure below comes from the authors' own paper and repositories.
What TaoMate-HD is
TaoMate-HD is an add-on, not a full model. The Hugging Face model card says the preview "provides the inference LoRA for the MiniMax H3 FL2VA base model." A LoRA is a small set of extra weights that changes how a large model behaves without replacing it. You need the MiniMax H3 base model plus this adapter. The card credits the Alibaba TaoLive AIGC Team and says it is "Powered by MiniMax H3."
If MiniMax H3 is new to you, our explainer on what MiniMax is and what its models do is a good start. For another product built on H3, see our post on HeyGen Video 1, a MiniMax H3-based model.
The target use is a streaming digital human. The paper abstract describes systems that "produce detailed video while continuing speech as new text arrives." Think of a virtual presenter in a live product demo who keeps talking while new lines of script come in. The authors say two things make that hard: high-resolution denoising is costly, and text updates can cause repeated or interrupted speech.
How it works in plain terms
A video model like H3 builds each clip by repeatedly cleaning up noise, a process called denoising. Each pass costs compute, and passes at high resolution cost the most. TaoMate-HD cuts that cost with a coarse-to-fine plan for each chunk of video:
- Two denoising steps run at low resolution to set the rough structure of the scene.
- One final step refines the video to high resolution.
- Audio is generated along the same path, so speech and picture stay linked.
The GitHub README gives the sizes: two evaluations at 480p, then one at 768p. The paper lists the grids as 480 by 864 and 768 by 1376. Final output is 768 by 1376 at 24 frames per second with 32 kHz stereo audio.
The second idea is memory. Each stage keeps its own history of what it predicted for the previous chunk. According to the paper, that lets the next chunk start its coarse steps before the previous chunk finishes its high-resolution step. The README calls this an optional pipeline mode, where separate stages run at the same time across eight GPUs.
The third idea is about speech. When a new piece of script arrives, it may start in the middle of a sentence and include words already spoken. The authors say a naive system either repeats content or stops too early. TaoMate-HD keeps recent audio and video history and overlaps the text windows, so it can tell what has already been said. The paper says it does this without a text pointer or a separate speech recognizer.
The numbers the authors report
All of these come from the paper's own tests, run on eight NVIDIA H20 GPUs.
- Speed from the coarse-to-fine plan: mean generation time fell from 491 seconds to 307 seconds against direct 768p generation on the same inputs. The authors call this a 1.6 times speedup.
- Speed from the pipeline mode: mean time fell further, from 307 seconds to 160 seconds. The paper calls that a 1.92 times gain over the serial version. It notes this is a deployment-level comparison that bundles several changes, including overlapping stages, 8-bit projections and codec scheduling.
- Speech accuracy: the character error rate fell from 30.75 percent to 19.57 percent, a drop of 11.18 percentage points, after the authors added their continuation training.
- Length: the authors say the model supports hour-long audiovisual generation and show frames from two 60-minute streams at 1376 by 768.
The paper also compares against MiniMax H3 itself. It says H3 uses 49 denoising evaluations per five-second request, while TaoMate-HD uses three per chunk. Under the timing rules in the paper, it reports a throughput ratio of 22.4 times over H3. The paper notes that the timing intervals are not identical across methods, so treat that ratio as the authors' figure and not a neutral benchmark.
What to keep in mind
This is a research preview, and several limits are visible in the sources themselves.
- The hardware is large. The README's example command uses eight GPUs, and its 10-second example was run on eight H20 GPUs with 96 GB each. This is not a laptop tool.
- The paper reports 8.99 frames per second of generation for the best stage-parallel setup, and the output plays at 24 frames per second. By these figures, generation runs slower than playback on that hardware. Serial mode reaches 4.68.
- The test set is the authors' own. The paper says it used 100 self-constructed sequences of 1 to 10 minutes, and trained the LoRA on 10,000 self-constructed distillation pairs. No outside group has tested these claims yet in the sources we checked.
- Speech is not solved. A character error rate of 19.57 percent is better than the baseline in the paper, but it is still far from clean transcripts. The paper measures it by comparing Whisper-small transcripts with the script, and its table counts only methods with Chinese speech support.
- The release is labeled a "preview". The model card's news line says the first release is the inference code and the preview LoRA weights. We did not see training code in the sources we read.
License
The model card says the release "follows the MiniMax H3 Community License Agreement" and that "the upstream license terms apply to the base model and this adapter." The MiniMax H3 page also links a license Q&A and lists an application form "only for USA/EU/UK/South Korea." We did not read the full license text, so check it before any commercial use. For the general difference between open weights and fully open software, see our guide to open-weight versus closed models.
Why it matters
Talking-avatar video is moving from short clips to continuous streams. We recently covered Kandinsky S2V 1.0, an open speech-to-video avatar model. TaoMate-HD takes a different route. It generates speech and video together and tries to keep going as new text arrives. The two releases show that open weights are now reaching the avatar and live-presenter use case, not only short cinematic clips. Another open streaming release, Wave Forcing, builds on the small Wan 2.1 model and generates video chunk by chunk.
For creators and marketers, the practical takeaway is modest for now. The method shows that a coarse-to-fine plan can cut generation time on a large open video model, but the gains were measured on a rack of data-center GPUs. Watch for hosted versions, smaller setups and independent tests before you plan a workflow around it.
