dorsal/arxiv
View SchemaEnhancing Foundation Models in Transaction Understanding with LLM-based Sentence Embeddings
| Authors | Xiran Fan, Zhimeng Jiang, Chin-Chia Michael Yeh, Yuzhong Chen, Yingtong Dou, Menghai Pan, Yan Zheng |
|---|---|
| Categories | |
| ArXiv ID | 2601.05271vv1 |
| URL | https://arxiv.org/abs/2601.05271 |
| DOI | 10.18653/v1/2025.emnlp-industry.61 |
| Journal | Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track (EMNLP 2025), pages 903-911 |
| License | http://creativecommons.org/licenses/by/4.0/ |
Abstract
The ubiquity of payment networks generates vast transactional data encoding rich consumer and merchant behavioral patterns. Recent foundation models for transaction analysis process tabular data sequentially but rely on index-based representations for categorical merchant fields, causing substantial semantic information loss by converting rich textual data into discrete tokens. While Large Language Models (LLMs) can address this limitation through superior semantic understanding, their computational overhead challenges real-time financial deployment. We introduce a hybrid framework that uses LLM-generated embeddings as semantic initializations for lightweight transaction models, balancing interpretability with operational efficiency. Our approach employs multi-source data fusion to enrich merchant categorical fields and a one-word constraint principle for consistent embedding generation across LLM architectures. We systematically address data quality through noise filtering and context-aware enrichment. Experiments on large-scale transaction datasets demonstrate significant performance improvements across multiple transaction understanding tasks.
{
"annotation_id": "a2f5f1d6-c528-44a3-9f75-0da3bc3a6f23",
"date_created": "2026-02-17T05:53:04.386000Z",
"date_modified": "2026-02-17T05:53:04.386000Z",
"file_hash": "16d29d5a42e8195ce318d0697e1221fae0a3b800fe6a2e41ab1cb78c919fa98e",
"private": false,
"record": {
"abstract": "The ubiquity of payment networks generates vast transactional data encoding rich consumer and merchant behavioral patterns. Recent foundation models for transaction analysis process tabular data sequentially but rely on index-based representations for categorical merchant fields, causing substantial semantic information loss by converting rich textual data into discrete tokens. While Large Language Models (LLMs) can address this limitation through superior semantic understanding, their computational overhead challenges real-time financial deployment. We introduce a hybrid framework that uses LLM-generated embeddings as semantic initializations for lightweight transaction models, balancing interpretability with operational efficiency. Our approach employs multi-source data fusion to enrich merchant categorical fields and a one-word constraint principle for consistent embedding generation across LLM architectures. We systematically address data quality through noise filtering and context-aware enrichment. Experiments on large-scale transaction datasets demonstrate significant performance improvements across multiple transaction understanding tasks.",
"arxiv_id": "2601.05271",
"authors": [
"Xiran Fan",
"Zhimeng Jiang",
"Chin-Chia Michael Yeh",
"Yuzhong Chen",
"Yingtong Dou",
"Menghai Pan",
"Yan Zheng"
],
"categories": [
"cs.CL",
"cs.LG"
],
"doi": "10.18653/v1/2025.emnlp-industry.61",
"journal_ref": "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track (EMNLP 2025), pages 903-911",
"license": "http://creativecommons.org/licenses/by/4.0/",
"title": "Enhancing Foundation Models in Transaction Understanding with LLM-based Sentence Embeddings",
"url": "https://arxiv.org/abs/2601.05271",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "217b76dc-0b2c-474d-a21f-c06cac13b097",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}