dorsal/arxiv
View SchemaThe AI Cognitive Trojan Horse: How Large Language Models May Bypass Human Epistemic Vigilance
| Authors | Andrew D. Maynard |
|---|---|
| Categories | |
| ArXiv ID | 2601.07085vv1 |
| URL | https://arxiv.org/abs/2601.07085 |
| License | http://creativecommons.org/licenses/by-nc-sa/4.0/ |
Abstract
Large language model (LLM)-based conversational AI systems present a challenge to human cognition that current frameworks for understanding misinformation and persuasion do not adequately address. This paper proposes that a significant epistemic risk from conversational AI may lie not in inaccuracy or intentional deception, but in something more fundamental: these systems may be configured, through optimization processes that make them useful, to present characteristics that bypass the cognitive mechanisms humans evolved to evaluate incoming information. The Cognitive Trojan Horse hypothesis draws on Sperber and colleagues' theory of epistemic vigilance -- the parallel cognitive process monitoring communicated information for reasons to doubt -- and proposes that LLM-based systems present 'honest non-signals': genuine characteristics (fluency, helpfulness, apparent disinterest) that fail to carry the information equivalent human characteristics would carry, because in humans these are costly to produce while in LLMs they are computationally trivial. Four mechanisms of potential bypass are identified: processing fluency decoupled from understanding, trust-competence presentation without corresponding stakes, cognitive offloading that delegates evaluation itself to the AI, and optimization dynamics that systematically produce sycophancy. The framework generates testable predictions, including a counterintuitive speculation that cognitively sophisticated users may be more vulnerable to AI-mediated epistemic influence. This reframes AI safety as partly a problem of calibration -- aligning human evaluative responses with the actual epistemic status of AI-generated content -- rather than solely a problem of preventing deception.
{
"annotation_id": "eed56ac5-9f7c-4885-9ab0-a8b96c9d2061",
"date_created": "2026-02-17T05:53:12.650000Z",
"date_modified": "2026-02-17T05:53:12.650000Z",
"file_hash": "221746a3cc42c2601b3b9466d2ad207c79ad687610688fdf87a4126c0d2f94b0",
"private": false,
"record": {
"abstract": "Large language model (LLM)-based conversational AI systems present a challenge to human cognition that current frameworks for understanding misinformation and persuasion do not adequately address. This paper proposes that a significant epistemic risk from conversational AI may lie not in inaccuracy or intentional deception, but in something more fundamental: these systems may be configured, through optimization processes that make them useful, to present characteristics that bypass the cognitive mechanisms humans evolved to evaluate incoming information. The Cognitive Trojan Horse hypothesis draws on Sperber and colleagues\u0027 theory of epistemic vigilance -- the parallel cognitive process monitoring communicated information for reasons to doubt -- and proposes that LLM-based systems present \u0027honest non-signals\u0027: genuine characteristics (fluency, helpfulness, apparent disinterest) that fail to carry the information equivalent human characteristics would carry, because in humans these are costly to produce while in LLMs they are computationally trivial. Four mechanisms of potential bypass are identified: processing fluency decoupled from understanding, trust-competence presentation without corresponding stakes, cognitive offloading that delegates evaluation itself to the AI, and optimization dynamics that systematically produce sycophancy. The framework generates testable predictions, including a counterintuitive speculation that cognitively sophisticated users may be more vulnerable to AI-mediated epistemic influence. This reframes AI safety as partly a problem of calibration -- aligning human evaluative responses with the actual epistemic status of AI-generated content -- rather than solely a problem of preventing deception.",
"arxiv_id": "2601.07085",
"authors": [
"Andrew D. Maynard"
],
"categories": [
"cs.HC",
"cs.AI",
"cs.CY"
],
"license": "http://creativecommons.org/licenses/by-nc-sa/4.0/",
"title": "The AI Cognitive Trojan Horse: How Large Language Models May Bypass Human Epistemic Vigilance",
"url": "https://arxiv.org/abs/2601.07085",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "8ebbe10e-9813-476b-b1a5-6f228dd46f8c",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}