Alibaba's Tongyi MAI-UI team has open-sourced "OpenGCUA," the complete technology stack for a general computer-use agent: including a harness, a cross-device SDK, a native macOS app, evaluation trajectories, and a reference agent implementation
Open-source address: https://github.com/Tongyi-MAI/OpenGCUA
From "GUI Agent" to "General Computer-Use Agent"
Traditional CUA (Computer Use Agent) is basically equivalent to a GUI agent that "looks at screenshots and clicks buttons"; the GCUA (generalist computer-use agent) defined by OpenGCUA is an agent that comprehensively schedules all interfaces of the entire computer: accessibility trees, application APIs, shell commands, and GUI operations. The paper defines GCUA as "a system in which the model autonomously decides how to observe, act, and organize work, spanning visual and programmatic interfaces."
Mechanistically, at each step GCUA makes three decisions determined by the model rather than fixed rules:
1. What to observe: call screenshot() on demand, instead of forcing a screenshot every step. This is critical: it acknowledges that "taking a screenshot" is itself a costly perception action, to be saved when unnecessary (for example, intermediate steps of batch operations don't need to be viewed).
2. How to act: instead of outputting a single atomic action, it writes a Python program with loops and conditional judgments, orchestrating GUI operations, API calls, and shell commands.
3. How to organize work: the model maintains its own todo list and declares an intent at each step, shifting task state management from hard-coded harness logic to autonomous model behavior.
The two major designs, "code execution" and "harness-to-model decision transfer," are the pillars of the entire methodology. The cleverness of code execution lies in reusing the model's existing strong programming abilities, and intermediate results within a loop directly determine the next step of the code, without needing to return to the model every step; this is the source of the efficiency gains. Theoretically, pure GUI agents, CLI agents, and GUI+CLI hybrid agents (their earlier Qwen-UI-Agent) all become special cases of the GCUA framework, and this unified perspective itself has theoretical value.
Experimental Results: What the Numbers Say
Evaluation on OSWorld-v2 across 8 frontier models (GPT-6 Astra, Claude Opus 5, Gemini 3.8 Flash, GLM-5.3-Flash, Qwen3.8-Max/Flash, Kimi K3, DeepSeek-V4.1-Flash):
· Average partial score rose from 45.3 to 64.9 (+19.6 points), and average steps per task fell by 53%; GPT-6 Astra dropped from 102 steps to 34.
· Weaker models benefited most (GLM-5.3-Flash +35.2), and the gap between strong and weak models narrowed from 39.1 to 18.0. This suggests the code-execution paradigm reduces the dependence of "agent capability" on the model's fine-grained GUI operation ability; weaker models can compensate for insufficient perception-click precision by writing programs, a significant signal for the open-source ecosystem.
· Code execution adds another 6.1 points over GUI+CLI hybrid and reduces steps by a further 14.8%, validating that "orchestrating into a program" is superior to "alternately calling tools."
· Real-device macOS testing revealed a fact that benchmarks obscure: under the same harness, Qwen3.8-Max consumed 19.4 times as many reasoning tokens as GPT-6 Astra and completed only 3 of 4 tasks; the efficiency gap on real devices is far larger than benchmark scores reflect, which is the motivation for their insistence on a real-device harness.