dorsal/arxiv
View SchemaCAFEDistill: Learning Personalized and Dynamic Models through Federated Early-Exit Network Distillation
| Authors | Boyi Liu, Zimu Zhou, Yongxin Tong |
|---|---|
| Categories | |
| ArXiv ID | 2601.10015vv1 |
| URL | https://arxiv.org/abs/2601.10015 |
| License | http://arxiv.org/licenses/nonexclusive-distrib/1.0/ |
Abstract
Personalized Federated Learning (PFL) enables collaboratively model training on decentralized, heterogeneous data while tailoring them to each client's unique distribution. However, existing PFL methods produce static models with a fixed tradeoff between accuracy and efficiency, limiting their applicability in environments where inference requirements vary with contexts and resource availability. Early-exit networks (EENs) offer adaptive inference by attaching intermediate classifiers. Yet integrating them into PFL is challenging due to client-wise heterogeneity and depth-wise interference arising from conflicting exit objectives. Prior studies fail to resolve both conflicts simultaneously, leading to suboptimal performance. In this paper, we propose CAFEDistill, a Conflict-Aware Federated Exit Distillation framework that jointly addresses these conflicts and extends PFL to early-exit networks. Through a progressive, depth-prioritized student coordination mechanism, CAFEDistill mitigates interference among shallow and deep exits while allowing effective personalized knowledge transfer across clients. Furthermore, it reduces communication overhead via a client-decoupled formulation. Extensive evaluations show that CAFEDistill outperforms the state-of-the-arts, achieving higher accuracy and reducing inference costs by 30.79%-46.86%.
{
"annotation_id": "676f7569-692f-4dfd-80eb-8e25da0dbae2",
"date_created": "2026-02-17T05:53:23.545000Z",
"date_modified": "2026-02-17T05:53:23.545000Z",
"file_hash": "f40776622081f1bdb3bfdea314197417c2c786e4a89de85d19a1ea9428e13342",
"private": false,
"record": {
"abstract": "Personalized Federated Learning (PFL) enables collaboratively model training on decentralized, heterogeneous data while tailoring them to each client\u0027s unique distribution. However, existing PFL methods produce static models with a fixed tradeoff between accuracy and efficiency, limiting their applicability in environments where inference requirements vary with contexts and resource availability. Early-exit networks (EENs) offer adaptive inference by attaching intermediate classifiers. Yet integrating them into PFL is challenging due to client-wise heterogeneity and depth-wise interference arising from conflicting exit objectives. Prior studies fail to resolve both conflicts simultaneously, leading to suboptimal performance. In this paper, we propose CAFEDistill, a Conflict-Aware Federated Exit Distillation framework that jointly addresses these conflicts and extends PFL to early-exit networks. Through a progressive, depth-prioritized student coordination mechanism, CAFEDistill mitigates interference among shallow and deep exits while allowing effective personalized knowledge transfer across clients. Furthermore, it reduces communication overhead via a client-decoupled formulation. Extensive evaluations show that CAFEDistill outperforms the state-of-the-arts, achieving higher accuracy and reducing inference costs by 30.79%-46.86%.",
"arxiv_id": "2601.10015",
"authors": [
"Boyi Liu",
"Zimu Zhou",
"Yongxin Tong"
],
"categories": [
"cs.LG"
],
"license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
"title": "CAFEDistill: Learning Personalized and Dynamic Models through Federated Early-Exit Network Distillation",
"url": "https://arxiv.org/abs/2601.10015",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "7a413c48-a655-4581-bff2-26fbd24a3a11",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}