dorsal/arxiv
View SchemaRAG-3DSG: Enhancing 3D Scene Graphs with Re-Shot Guided Retrieval-Augmented Generation
| Authors | Yue Chang, Rufeng Chen, Zhaofan Zhang, Yi Chen, Sihong Xie |
|---|---|
| Categories | |
| ArXiv ID | 2601.10168vv1 |
| URL | https://arxiv.org/abs/2601.10168 |
| License | http://arxiv.org/licenses/nonexclusive-distrib/1.0/ |
Abstract
Open-vocabulary 3D Scene Graph (3DSG) generation can enhance various downstream tasks in robotics, such as manipulation and navigation, by leveraging structured semantic representations. A 3DSG is constructed from multiple images of a scene, where objects are represented as nodes and relationships as edges. However, existing works for open-vocabulary 3DSG generation suffer from both low object-level recognition accuracy and speed, mainly due to constrained viewpoints, occlusions, and redundant surface density. To address these challenges, we propose RAG-3DSG to mitigate aggregation noise through re-shot guided uncertainty estimation and support object-level Retrieval-Augmented Generation (RAG) via reliable low-uncertainty objects. Furthermore, we propose a dynamic downsample-mapping strategy to accelerate cross-image object aggregation with adaptive granularity. Experiments on Replica dataset demonstrate that RAG-3DSG significantly improves node captioning accuracy in 3DSG generation while reducing the mapping time by two-thirds compared to the vanilla version.
{
"annotation_id": "a97c2c6e-8e3f-473c-be32-8260c8ae363b",
"date_created": "2026-02-17T05:53:24.386000Z",
"date_modified": "2026-02-17T05:53:24.386000Z",
"file_hash": "4c092f26dfb67da15d442b6c6f2f776dfe7c53f499b174b50e2a851b05b1cea7",
"private": false,
"record": {
"abstract": "Open-vocabulary 3D Scene Graph (3DSG) generation can enhance various downstream tasks in robotics, such as manipulation and navigation, by leveraging structured semantic representations. A 3DSG is constructed from multiple images of a scene, where objects are represented as nodes and relationships as edges. However, existing works for open-vocabulary 3DSG generation suffer from both low object-level recognition accuracy and speed, mainly due to constrained viewpoints, occlusions, and redundant surface density. To address these challenges, we propose RAG-3DSG to mitigate aggregation noise through re-shot guided uncertainty estimation and support object-level Retrieval-Augmented Generation (RAG) via reliable low-uncertainty objects. Furthermore, we propose a dynamic downsample-mapping strategy to accelerate cross-image object aggregation with adaptive granularity. Experiments on Replica dataset demonstrate that RAG-3DSG significantly improves node captioning accuracy in 3DSG generation while reducing the mapping time by two-thirds compared to the vanilla version.",
"arxiv_id": "2601.10168",
"authors": [
"Yue Chang",
"Rufeng Chen",
"Zhaofan Zhang",
"Yi Chen",
"Sihong Xie"
],
"categories": [
"cs.CV",
"cs.AI",
"cs.RO"
],
"license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
"title": "RAG-3DSG: Enhancing 3D Scene Graphs with Re-Shot Guided Retrieval-Augmented Generation",
"url": "https://arxiv.org/abs/2601.10168",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "bbfe54b3-0d15-47c8-ae75-858dbaf9f0b5",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}