dorsal/arxiv
View SchemaOpen World Knowledge Aided Single-Cell Foundation Model with Robust Cross-Modal Cell-Language Pre-training
| Authors | Haoran Wang, Xuanyi Zhang, Shuangsang Fang, Longke Ran, Ziqing Deng, Yong Zhang, Yuxiang Li, Shaoshuai Li |
|---|---|
| Categories | |
| ArXiv ID | 2601.05648vv1 |
| URL | https://arxiv.org/abs/2601.05648 |
| License | http://creativecommons.org/licenses/by/4.0/ |
Abstract
Recent advancements in single-cell multi-omics, particularly RNA-seq, have provided profound insights into cellular heterogeneity and gene regulation. While pre-trained language model (PLM) paradigm based single-cell foundation models have shown promise, they remain constrained by insufficient integration of in-depth individual profiles and neglecting the influence of noise within multi-modal data. To address both issues, we propose an Open-world Language Knowledge-Aided Robust Single-Cell Foundation Model (OKR-CELL). It is built based on a cross-modal Cell-Language pre-training framework, which comprises two key innovations: (1) leveraging Large Language Models (LLMs) based workflow with retrieval-augmented generation (RAG) enriches cell textual descriptions using open-world knowledge; (2) devising a Cross-modal Robust Alignment (CRA) objective that incorporates sample reliability assessment, curriculum learning, and coupled momentum contrastive learning to strengthen the model's resistance to noisy data. After pretraining on 32M cell-text pairs, OKR-CELL obtains cutting-edge results across 6 evaluation tasks. Beyond standard benchmarks such as cell clustering, cell-type annotation, batch-effect correction, and few-shot annotation, the model also demonstrates superior performance in broader multi-modal applications, including zero-shot cell-type annotation and bidirectional cell-text retrieval.
{
"annotation_id": "bb058451-3cd5-4a1d-8df1-8659c4eb91a9",
"date_created": "2026-02-17T05:53:03.990000Z",
"date_modified": "2026-02-17T05:53:03.990000Z",
"file_hash": "3fa3c481fbe15e348c776fead5d7c1f831eab5493ef2f23291e75ee9f3f9fe51",
"private": false,
"record": {
"abstract": "Recent advancements in single-cell multi-omics, particularly RNA-seq, have provided profound insights into cellular heterogeneity and gene regulation. While pre-trained language model (PLM) paradigm based single-cell foundation models have shown promise, they remain constrained by insufficient integration of in-depth individual profiles and neglecting the influence of noise within multi-modal data. To address both issues, we propose an Open-world Language Knowledge-Aided Robust Single-Cell Foundation Model (OKR-CELL). It is built based on a cross-modal Cell-Language pre-training framework, which comprises two key innovations: (1) leveraging Large Language Models (LLMs) based workflow with retrieval-augmented generation (RAG) enriches cell textual descriptions using open-world knowledge; (2) devising a Cross-modal Robust Alignment (CRA) objective that incorporates sample reliability assessment, curriculum learning, and coupled momentum contrastive learning to strengthen the model\u0027s resistance to noisy data. After pretraining on 32M cell-text pairs, OKR-CELL obtains cutting-edge results across 6 evaluation tasks. Beyond standard benchmarks such as cell clustering, cell-type annotation, batch-effect correction, and few-shot annotation, the model also demonstrates superior performance in broader multi-modal applications, including zero-shot cell-type annotation and bidirectional cell-text retrieval.",
"arxiv_id": "2601.05648",
"authors": [
"Haoran Wang",
"Xuanyi Zhang",
"Shuangsang Fang",
"Longke Ran",
"Ziqing Deng",
"Yong Zhang",
"Yuxiang Li",
"Shaoshuai Li"
],
"categories": [
"q-bio.GN",
"cs.AI",
"cs.CL",
"cs.LG"
],
"license": "http://creativecommons.org/licenses/by/4.0/",
"title": "Open World Knowledge Aided Single-Cell Foundation Model with Robust Cross-Modal Cell-Language Pre-training",
"url": "https://arxiv.org/abs/2601.05648",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "40f930d7-2578-494d-905c-5be0864c192d",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}