dorsal/arxiv
View SchemaUnifying Search and Recommendation in LLMs via Gradient Multi-Subspace Tuning
| Authors | Jujia Zhao, Zihan Wang, Shuaiqun Pan, Suzan Verberne, Zhaochun Ren |
|---|---|
| Categories | |
| ArXiv ID | 2601.09496vv1 |
| URL | https://arxiv.org/abs/2601.09496 |
| License | http://arxiv.org/licenses/nonexclusive-distrib/1.0/ |
Abstract
Search and recommendation (S&R) are core to online platforms, addressing explicit intent through queries and modeling implicit intent from behaviors, respectively. Their complementary roles motivate a unified modeling paradigm. Early studies to unify S&R adopt shared encoders with task-specific heads, while recent efforts reframe item ranking in both S&R as conditional generation. The latter holds particular promise, enabling end-to-end optimization and leveraging the semantic understanding of LLMs. However, existing methods rely on full fine-tuning, which is computationally expensive and limits scalability. Parameter-efficient fine-tuning (PEFT) offers a more practical alternative but faces two critical challenges in unifying S&R: (1) gradient conflicts across tasks due to divergent optimization objectives, and (2) shifts in user intent understanding caused by overfitting to fine-tuning data, which distort general-domain knowledge and weaken LLM reasoning. To address the above issues, we propose Gradient Multi-Subspace Tuning (GEMS), a novel framework that unifies S&R with LLMs while alleviating gradient conflicts and preserving general-domain knowledge. GEMS introduces (1) \textbf{Multi-Subspace Decomposition}, which disentangles shared and task-specific optimization signals into complementary low-rank subspaces, thereby reducing destructive gradient interference, and (2) \textbf{Null-Space Projection}, which constrains parameter updates to a subspace orthogonal to the general-domain knowledge space, mitigating shifts in user intent understanding. Extensive experiments on benchmark datasets show that GEMS consistently outperforms the state-of-the-art baselines across both search and recommendation tasks, achieving superior effectiveness.
{
"annotation_id": "f899c270-8440-4b37-976b-9e778ca1993c",
"date_created": "2026-02-17T05:53:20.458000Z",
"date_modified": "2026-02-17T05:53:20.458000Z",
"file_hash": "e796f5bb72699ad8ff0d81c9c8ffa9f77d4a5bdd89fa0c33d4cea6ce50e6af8e",
"private": false,
"record": {
"abstract": "Search and recommendation (S\u0026R) are core to online platforms, addressing explicit intent through queries and modeling implicit intent from behaviors, respectively. Their complementary roles motivate a unified modeling paradigm. Early studies to unify S\u0026R adopt shared encoders with task-specific heads, while recent efforts reframe item ranking in both S\u0026R as conditional generation. The latter holds particular promise, enabling end-to-end optimization and leveraging the semantic understanding of LLMs. However, existing methods rely on full fine-tuning, which is computationally expensive and limits scalability. Parameter-efficient fine-tuning (PEFT) offers a more practical alternative but faces two critical challenges in unifying S\u0026R: (1) gradient conflicts across tasks due to divergent optimization objectives, and (2) shifts in user intent understanding caused by overfitting to fine-tuning data, which distort general-domain knowledge and weaken LLM reasoning. To address the above issues, we propose Gradient Multi-Subspace Tuning (GEMS), a novel framework that unifies S\u0026R with LLMs while alleviating gradient conflicts and preserving general-domain knowledge. GEMS introduces (1) \\textbf{Multi-Subspace Decomposition}, which disentangles shared and task-specific optimization signals into complementary low-rank subspaces, thereby reducing destructive gradient interference, and (2) \\textbf{Null-Space Projection}, which constrains parameter updates to a subspace orthogonal to the general-domain knowledge space, mitigating shifts in user intent understanding. Extensive experiments on benchmark datasets show that GEMS consistently outperforms the state-of-the-art baselines across both search and recommendation tasks, achieving superior effectiveness.",
"arxiv_id": "2601.09496",
"authors": [
"Jujia Zhao",
"Zihan Wang",
"Shuaiqun Pan",
"Suzan Verberne",
"Zhaochun Ren"
],
"categories": [
"cs.IR"
],
"license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
"title": "Unifying Search and Recommendation in LLMs via Gradient Multi-Subspace Tuning",
"url": "https://arxiv.org/abs/2601.09496",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "0b416896-9367-4e2d-9b6d-08982787da17",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}