dorsal/arxiv
View SchemaOS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agent
| Authors | Bowen Yang, Kaiming Jin, Zhenyu Wu, Zhaoyang Liu, Qiushi Sun, Zehao Li, JingJing Xie, Zhoumianze Liu, Fangzhi Xu, Kanzhi Cheng, Qingyun Li, Yian Wang, Yu Qiao, Zun Wang, Zichen Ding |
|---|---|
| Categories | |
| ArXiv ID | 2601.07779vv1 |
| URL | https://arxiv.org/abs/2601.07779 |
| License | http://arxiv.org/licenses/nonexclusive-distrib/1.0/ |
Abstract
While Vision-Language Models (VLMs) have significantly advanced Computer-Using Agents (CUAs), current frameworks struggle with robustness in long-horizon workflows and generalization in novel domains. These limitations stem from a lack of granular control over historical visual context curation and the absence of visual-aware tutorial retrieval. To bridge these gaps, we introduce OS-Symphony, a holistic framework that comprises an Orchestrator coordinating two key innovations for robust automation: (1) a Reflection-Memory Agent that utilizes milestone-driven long-term memory to enable trajectory-level self-correction, effectively mitigating visual context loss in long-horizon tasks; (2) Versatile Tool Agents featuring a Multimodal Searcher that adopts a SeeAct paradigm to navigate a browser-based sandbox to synthesize live, visually aligned tutorials, thereby resolving fidelity issues in unseen scenarios. Experimental results demonstrate that OS-Symphony delivers substantial performance gains across varying model scales, establishing new state-of-the-art results on three online benchmarks, notably achieving 65.84% on OSWorld.
{
"annotation_id": "c7de3371-98dd-4d20-bcbb-296dd918eb94",
"date_created": "2026-02-17T05:53:12.593000Z",
"date_modified": "2026-02-17T05:53:12.593000Z",
"file_hash": "55b1570ceeaa9aae69ce4f150bb0bff9347ff8e98d8c773c457820cd6c5d0f41",
"private": false,
"record": {
"abstract": "While Vision-Language Models (VLMs) have significantly advanced Computer-Using Agents (CUAs), current frameworks struggle with robustness in long-horizon workflows and generalization in novel domains. These limitations stem from a lack of granular control over historical visual context curation and the absence of visual-aware tutorial retrieval. To bridge these gaps, we introduce OS-Symphony, a holistic framework that comprises an Orchestrator coordinating two key innovations for robust automation: (1) a Reflection-Memory Agent that utilizes milestone-driven long-term memory to enable trajectory-level self-correction, effectively mitigating visual context loss in long-horizon tasks; (2) Versatile Tool Agents featuring a Multimodal Searcher that adopts a SeeAct paradigm to navigate a browser-based sandbox to synthesize live, visually aligned tutorials, thereby resolving fidelity issues in unseen scenarios. Experimental results demonstrate that OS-Symphony delivers substantial performance gains across varying model scales, establishing new state-of-the-art results on three online benchmarks, notably achieving 65.84% on OSWorld.",
"arxiv_id": "2601.07779",
"authors": [
"Bowen Yang",
"Kaiming Jin",
"Zhenyu Wu",
"Zhaoyang Liu",
"Qiushi Sun",
"Zehao Li",
"JingJing Xie",
"Zhoumianze Liu",
"Fangzhi Xu",
"Kanzhi Cheng",
"Qingyun Li",
"Yian Wang",
"Yu Qiao",
"Zun Wang",
"Zichen Ding"
],
"categories": [
"cs.MA",
"cs.AI",
"cs.CL",
"cs.CV",
"cs.HC"
],
"license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
"title": "OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agent",
"url": "https://arxiv.org/abs/2601.07779",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "1c513dc4-cd1d-4118-9f7f-1d052fad514e",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}