dorsal/arxiv
View SchemaSeeing Right but Saying Wrong: Inter- and Intra-Layer Refinement in MLLMs without Training
| Authors | Shezheng Song, Shasha Li, Jie Yu |
|---|---|
| Categories | |
| ArXiv ID | 2601.07359vv1 |
| URL | https://arxiv.org/abs/2601.07359 |
| License | http://creativecommons.org/licenses/by/4.0/ |
Abstract
Multimodal Large Language Models (MLLMs) have demonstrated strong capabilities across a variety of vision-language tasks. However, their internal reasoning often exhibits a critical inconsistency: although deeper layers may attend to the correct visual regions, final predictions are frequently misled by noisy attention from earlier layers. This results in a disconnect between what the model internally understands and what it ultimately expresses, a phenomenon we describe as seeing it right but saying it wrong. To address this issue, we propose DualPD, a dual-perspective decoding refinement strategy that enhances the visual understanding without any additional training. DualPD consists of two components. (1) The layer-wise attention-guided contrastive logits module captures how the belief in the correct answer evolves by comparing output logits between layers that exhibit the largest attention shift. (2) The head-wise information filtering module suppresses low-contribution attention heads that focus on irrelevant regions, thereby improving attention quality within each layer. Experiments conducted on both the LLaVA and Qwen-VL model families across multiple multimodal benchmarks demonstrate that DualPD consistently improves accuracy without training, confirming its effectiveness and generalizability. The code will be released upon publication.
{
"annotation_id": "de400a1d-703f-4938-b6e1-f73f1db06ca2",
"date_created": "2026-02-17T05:53:11.817000Z",
"date_modified": "2026-02-17T05:53:11.817000Z",
"file_hash": "0b999da7f0427f5c255eb1eac7261054d9da6a994c899a544f281d6936fcc890",
"private": false,
"record": {
"abstract": "Multimodal Large Language Models (MLLMs) have demonstrated strong capabilities across a variety of vision-language tasks. However, their internal reasoning often exhibits a critical inconsistency: although deeper layers may attend to the correct visual regions, final predictions are frequently misled by noisy attention from earlier layers. This results in a disconnect between what the model internally understands and what it ultimately expresses, a phenomenon we describe as seeing it right but saying it wrong. To address this issue, we propose DualPD, a dual-perspective decoding refinement strategy that enhances the visual understanding without any additional training. DualPD consists of two components. (1) The layer-wise attention-guided contrastive logits module captures how the belief in the correct answer evolves by comparing output logits between layers that exhibit the largest attention shift. (2) The head-wise information filtering module suppresses low-contribution attention heads that focus on irrelevant regions, thereby improving attention quality within each layer. Experiments conducted on both the LLaVA and Qwen-VL model families across multiple multimodal benchmarks demonstrate that DualPD consistently improves accuracy without training, confirming its effectiveness and generalizability. The code will be released upon publication.",
"arxiv_id": "2601.07359",
"authors": [
"Shezheng Song",
"Shasha Li",
"Jie Yu"
],
"categories": [
"cs.CV",
"cs.AI"
],
"license": "http://creativecommons.org/licenses/by/4.0/",
"title": "Seeing Right but Saying Wrong: Inter- and Intra-Layer Refinement in MLLMs without Training",
"url": "https://arxiv.org/abs/2601.07359",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "ac5c25bd-9906-4975-b96d-16a9767e9393",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}