dorsal/arxiv
View SchemaMulti-Task Fine-Tuning Enables Robust Out-of-Distribution Generalization in Atomistic Models
| Authors | Chengqian Zhang, Duo Zhang, Anyang Peng, Mingyu Guo, Yuzhi Zhang, Lei Wang, Guolin Ke, Linfeng Zhang, Tiejun Li, Han Wang |
|---|---|
| Categories | |
| ArXiv ID | 2601.08486vv1 |
| URL | https://arxiv.org/abs/2601.08486 |
| License | http://creativecommons.org/licenses/by/4.0/ |
Abstract
Accurate de novo molecular and materials design requires structure-property models that generalize beyond known regimes. Although pretrained atomistic models achieve strong in-distribution accuracy after fine-tuning, their reliability under out-of-distribution (OOD) conditions remains unclear. We identify a critical failure mode in downstream adaptation: standard fine-tuning induces representation collapse, erasing pretrained chemical and structural priors and severely degrading OOD performance. To address this limitation, we propose multi-task fine-tuning (MFT), which jointly optimizes downstream property prediction with a physically grounded force-field objective inherited from pretraining. This approach preserves essential chemical priors while enabling task-specific adaptation. Across molecular and materials benchmarks, MFT consistently improves OOD generalization, approaching the theoretical limit set by in-distribution accuracy, while outperforming standard fine-tuning, training from scratch, and state-of-the-art task-specific models. These results establish safe adaptation as a central requirement for large atomistic models and position MFT as a practical and data-efficient pathway toward robust molecular and materials discovery.
{
"annotation_id": "1b6e1330-bddd-4020-8fd6-1c6107438e76",
"date_created": "2026-02-17T05:53:16.007000Z",
"date_modified": "2026-02-17T05:53:16.007000Z",
"file_hash": "d43c636ef5ec3677d39351b5534edeed021310f44357cfece9c19a9cd2fda4ec",
"private": false,
"record": {
"abstract": "Accurate de novo molecular and materials design requires structure-property models that generalize beyond known regimes. Although pretrained atomistic models achieve strong in-distribution accuracy after fine-tuning, their reliability under out-of-distribution (OOD) conditions remains unclear. We identify a critical failure mode in downstream adaptation: standard fine-tuning induces representation collapse, erasing pretrained chemical and structural priors and severely degrading OOD performance. To address this limitation, we propose multi-task fine-tuning (MFT), which jointly optimizes downstream property prediction with a physically grounded force-field objective inherited from pretraining. This approach preserves essential chemical priors while enabling task-specific adaptation. Across molecular and materials benchmarks, MFT consistently improves OOD generalization, approaching the theoretical limit set by in-distribution accuracy, while outperforming standard fine-tuning, training from scratch, and state-of-the-art task-specific models. These results establish safe adaptation as a central requirement for large atomistic models and position MFT as a practical and data-efficient pathway toward robust molecular and materials discovery.",
"arxiv_id": "2601.08486",
"authors": [
"Chengqian Zhang",
"Duo Zhang",
"Anyang Peng",
"Mingyu Guo",
"Yuzhi Zhang",
"Lei Wang",
"Guolin Ke",
"Linfeng Zhang",
"Tiejun Li",
"Han Wang"
],
"categories": [
"physics.comp-ph"
],
"license": "http://creativecommons.org/licenses/by/4.0/",
"title": "Multi-Task Fine-Tuning Enables Robust Out-of-Distribution Generalization in Atomistic Models",
"url": "https://arxiv.org/abs/2601.08486",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "cec3b077-6d3f-49e0-8cb5-93c4433f0c83",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}