dorsal/arxiv
View SchemaM$^3$Searcher: Modular Multimodal Information Seeking Agency with Retrieval-Oriented Reasoning
| Authors | Xiaohan Yu, Chao Feng, Lang Mei, Chong Chen |
|---|---|
| Categories | |
| ArXiv ID | 2601.09278vv1 |
| URL | https://arxiv.org/abs/2601.09278 |
| License | http://arxiv.org/licenses/nonexclusive-distrib/1.0/ |
Abstract
Recent advances in DeepResearch-style agents have demonstrated strong capabilities in autonomous information acquisition and synthesize from real-world web environments. However, existing approaches remain fundamentally limited to text modality. Extending autonomous information-seeking agents to multimodal settings introduces critical challenges: the specialization-generalization trade-off that emerges when training models for multimodal tool-use at scale, and the severe scarcity of training data capturing complex, multi-step multimodal search trajectories. To address these challenges, we propose M$^3$Searcher, a modular multimodal information-seeking agent that explicitly decouples information acquisition from answer derivation. M$^3$Searcher is optimized with a retrieval-oriented multi-objective reward that jointly encourages factual accuracy, reasoning soundness, and retrieval fidelity. In addition, we develop MMSearchVQA, a multimodal multi-hop dataset to support retrieval centric RL training. Experimental results demonstrate that M$^3$Searcher outperforms existing approaches, exhibits strong transfer adaptability and effective reasoning in complex multimodal tasks.
{
"annotation_id": "f0a950ab-b45c-49b5-b1fa-3d378e3d01d3",
"date_created": "2026-02-17T05:53:20.459000Z",
"date_modified": "2026-02-17T05:53:20.459000Z",
"file_hash": "fb0e3e0ef1f706d1835942f13f88b4c161208243f7f2e6423945faab6063eedb",
"private": false,
"record": {
"abstract": "Recent advances in DeepResearch-style agents have demonstrated strong capabilities in autonomous information acquisition and synthesize from real-world web environments. However, existing approaches remain fundamentally limited to text modality. Extending autonomous information-seeking agents to multimodal settings introduces critical challenges: the specialization-generalization trade-off that emerges when training models for multimodal tool-use at scale, and the severe scarcity of training data capturing complex, multi-step multimodal search trajectories. To address these challenges, we propose M$^3$Searcher, a modular multimodal information-seeking agent that explicitly decouples information acquisition from answer derivation. M$^3$Searcher is optimized with a retrieval-oriented multi-objective reward that jointly encourages factual accuracy, reasoning soundness, and retrieval fidelity. In addition, we develop MMSearchVQA, a multimodal multi-hop dataset to support retrieval centric RL training. Experimental results demonstrate that M$^3$Searcher outperforms existing approaches, exhibits strong transfer adaptability and effective reasoning in complex multimodal tasks.",
"arxiv_id": "2601.09278",
"authors": [
"Xiaohan Yu",
"Chao Feng",
"Lang Mei",
"Chong Chen"
],
"categories": [
"cs.AI"
],
"license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
"title": "M$^3$Searcher: Modular Multimodal Information Seeking Agency with Retrieval-Oriented Reasoning",
"url": "https://arxiv.org/abs/2601.09278",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "33e152c8-fa63-4b61-ba98-77d86ff6236e",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}