dorsal/arxiv
View SchemaX-SAM: Boosting Sharpness-Aware Minimization with Dominant-Eigenvector Gradient Correction
| Authors | Hongru Duan, Yongle Chen, Lei Guan |
|---|---|
| Categories | |
| ArXiv ID | 2601.10251vv1 |
| URL | https://arxiv.org/abs/2601.10251 |
| License | http://arxiv.org/licenses/nonexclusive-distrib/1.0/ |
Abstract
Sharpness-Aware Minimization (SAM) aims to improve generalization by minimizing a worst-case perturbed loss over a small neighborhood of model parameters. However, during training, its optimization behavior does not always align with theoretical expectations, since both sharp and flat regions may yield a small perturbed loss. In such cases, the gradient may still point toward sharp regions, failing to achieve the intended effect of SAM. To address this issue, we investigate SAM from a spectral and geometric perspective: specifically, we utilize the angle between the gradient and the leading eigenvector of the Hessian as a measure of sharpness. Our analysis illustrates that when this angle is less than or equal to ninety degrees, the effect of SAM's sharpness regularization can be weakened. Furthermore, we propose an explicit eigenvector-aligned SAM (X-SAM), which corrects the gradient via orthogonal decomposition along the top eigenvector, enabling more direct and efficient regularization of the Hessian's maximum eigenvalue. We prove X-SAM's convergence and superior generalization, with extensive experimental evaluations confirming both theoretical and practical advantages.
{
"annotation_id": "903433df-f2fe-4ea8-b874-7f86f354f4b9",
"date_created": "2026-02-17T05:53:23.621000Z",
"date_modified": "2026-02-17T05:53:23.621000Z",
"file_hash": "86340ce03a128c6580cb5b3c2b8612daa4bcdd699017e3432e721126d563f6e7",
"private": false,
"record": {
"abstract": "Sharpness-Aware Minimization (SAM) aims to improve generalization by minimizing a worst-case perturbed loss over a small neighborhood of model parameters. However, during training, its optimization behavior does not always align with theoretical expectations, since both sharp and flat regions may yield a small perturbed loss. In such cases, the gradient may still point toward sharp regions, failing to achieve the intended effect of SAM. To address this issue, we investigate SAM from a spectral and geometric perspective: specifically, we utilize the angle between the gradient and the leading eigenvector of the Hessian as a measure of sharpness. Our analysis illustrates that when this angle is less than or equal to ninety degrees, the effect of SAM\u0027s sharpness regularization can be weakened. Furthermore, we propose an explicit eigenvector-aligned SAM (X-SAM), which corrects the gradient via orthogonal decomposition along the top eigenvector, enabling more direct and efficient regularization of the Hessian\u0027s maximum eigenvalue. We prove X-SAM\u0027s convergence and superior generalization, with extensive experimental evaluations confirming both theoretical and practical advantages.",
"arxiv_id": "2601.10251",
"authors": [
"Hongru Duan",
"Yongle Chen",
"Lei Guan"
],
"categories": [
"cs.LG",
"cs.AI"
],
"license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
"title": "X-SAM: Boosting Sharpness-Aware Minimization with Dominant-Eigenvector Gradient Correction",
"url": "https://arxiv.org/abs/2601.10251",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "789862e6-16cc-4dfb-ae0c-1be147c86335",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}