Tencent's HY Frontier team has released UI-Mate, an open-weight foundation GUI agent designed for autonomous computer operation across desktop environments. Available in 27-billion and 9-billion parameter configurations, the models pair environment-grounded reinforcement learning with an in-context demonstration pipeline that allows agents to reference structured human workflows during execution.
Alongside model weights on Hugging Face, Tencent published a technical report, code repository, and a native Apple Silicon macOS application for local deployment.
Closed-Loop Training and In-Context Demonstrations
Standard Computer-Use Agents (CUAs) frequently struggle with complex, multi-step desktop workflows when relying solely on natural-language prompts. To address brittle instruction adherence, UI-Mate employs a two-tier operational architecture: an environment-grounded data flywheel for foundational interaction capabilities, and an in-context demonstration framework for complex procedural tasks.
The foundational training stack utilizes a closed-loop system encompassing synthetic environment construction, filtered agent rollouts, capability-tree rebalancing, supervised fine-tuning (SFT), and online reinforcement learning.
For complex workflows, the framework introduces an offline demonstration parser and an online execution harness:
- Demonstration Ingestion: A native screen recorder logs user actions alongside high-frequency before-and-after screenshots and raw input telemetry (keystrokes, pointer coordinates, text input).
- Semantic Step Structuring: A vision-language model parses recorded events into structured steps containing observation states, operational intents, discrete actions, and verification criteria.
- Subtask Grouping: Atomic actions are consolidated into milestone-driven subtasks with explicit completion conditions.
- Authoritative Runtime Execution: During live desktop execution, the agent evaluates real-time screen pixels as the authoritative ground truth while using the parsed subtask checklist for directional guidance, preventing the system from degenerating into a fragile macro replay.
Benchmark Results Across Linux and Windows Environments
The models were evaluated across standard GUI benchmarks including OSWorld-Verified, WindowsAgentArena, and the newly introduced OSWorkerBench, which comprises 100 long-horizon office workflows spanning 41 desktop applications.
| Benchmark | Metric | UI-Mate-27B | UI-Mate-9B | Kimi-2.6 (1T-A32B) | Qwen3.6-27B | ScaleCUA-Qwen3.5 (9B) | | :--- | :--- | :--- | :--- | :--- | :--- | :--- | | OSWorld-Verified | Success Rate | 77.0% | 66.2% | 73.1% | 52.5% | 68.7% | | WindowsAgentArena | Success Rate | 66.2% | 61.7% | 63.3% | 47.1% | 38.1% | | OSWorkerBench (Instruction Only) | Strict / Progress | 41.0% / 76.9% | 34.0% / 66.6% | 40.7% / 72.4% | 23.3% / 52.4% | 16.3% / 38.3% |
On OSWorld-Verified, UI-Mate-27B achieved a 77.0% success rate under instruction-only conditions, outperforming larger frontier models and open-weight alternatives. On WindowsAgentArena, the 27B model reached 66.2%, while the 9B parameter variant scored 61.7%.
In tests measuring the impact of demonstration guidance on OSWorkerBench, providing a single in-context workflow demonstration increased strict success rates from 17.2% to 35.4% on self-demonstrated tasks, showing that structured task examples mitigate state divergence in long-horizon desktop navigation.
Open-Weight Release and Ecosystem Availability
Tencent has published model checkpoints under open-weight terms on Hugging Face, covering both the 27B and 9B variants. The project repository includes the demonstration recording harness, evaluation environments, and deployment scripts. A standalone macOS client (version 0.2.4) is also available for local execution on Apple Silicon hardware.
