dorsal/arxiv
View SchemaCategorize Early, Integrate Late: Divergent Processing Strategies in Automatic Speech Recognition
| Authors | Nathan Roll, Pranav Bhalerao, Martijn Bartelds, Arjun Pawar, Yuka Tatsumi, Tolulope Ogunremi, Chen Shani, Calbert Graham, Meghan Sumner, Dan Jurafsky |
|---|---|
| Categories | |
| ArXiv ID | 2601.06972vv1 |
| URL | https://arxiv.org/abs/2601.06972 |
| License | http://creativecommons.org/licenses/by/4.0/ |
Abstract
In speech language modeling, two architectures dominate the frontier: the Transformer and the Conformer. However, it remains unknown whether their comparable performance stems from convergent processing strategies or distinct architectural inductive biases. We introduce Architectural Fingerprinting, a probing framework that isolates the effect of architecture on representation, and apply it to a controlled suite of 24 pre-trained encoders (39M-3.3B parameters). Our analysis reveals divergent hierarchies: Conformers implement a "Categorize Early" strategy, resolving phoneme categories 29% earlier in depth and speaker gender by 16% depth. In contrast, Transformers "Integrate Late," deferring phoneme, accent, and duration encoding to deep layers (49-57%). These fingerprints suggest design heuristics: Conformers' front-loaded categorization may benefit low-latency streaming, while Transformers' deep integration may favor tasks requiring rich context and cross-utterance normalization.
{
"annotation_id": "9339f2aa-586b-4e26-b4b4-1781e0def6ef",
"date_created": "2026-02-17T05:53:08.517000Z",
"date_modified": "2026-02-17T05:53:08.517000Z",
"file_hash": "85975461343e041d3fdd68678387f1d7caada0a912befe9cd1fe4ad80c6b2139",
"private": false,
"record": {
"abstract": "In speech language modeling, two architectures dominate the frontier: the Transformer and the Conformer. However, it remains unknown whether their comparable performance stems from convergent processing strategies or distinct architectural inductive biases. We introduce Architectural Fingerprinting, a probing framework that isolates the effect of architecture on representation, and apply it to a controlled suite of 24 pre-trained encoders (39M-3.3B parameters). Our analysis reveals divergent hierarchies: Conformers implement a \"Categorize Early\" strategy, resolving phoneme categories 29% earlier in depth and speaker gender by 16% depth. In contrast, Transformers \"Integrate Late,\" deferring phoneme, accent, and duration encoding to deep layers (49-57%). These fingerprints suggest design heuristics: Conformers\u0027 front-loaded categorization may benefit low-latency streaming, while Transformers\u0027 deep integration may favor tasks requiring rich context and cross-utterance normalization.",
"arxiv_id": "2601.06972",
"authors": [
"Nathan Roll",
"Pranav Bhalerao",
"Martijn Bartelds",
"Arjun Pawar",
"Yuka Tatsumi",
"Tolulope Ogunremi",
"Chen Shani",
"Calbert Graham",
"Meghan Sumner",
"Dan Jurafsky"
],
"categories": [
"cs.CL"
],
"license": "http://creativecommons.org/licenses/by/4.0/",
"title": "Categorize Early, Integrate Late: Divergent Processing Strategies in Automatic Speech Recognition",
"url": "https://arxiv.org/abs/2601.06972",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "12f6f946-df13-4f0c-8025-917c393265a6",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}