dorsal/arxiv
View SchemaPACEvolve: Enabling Long-Horizon Progress-Aware Consistent Evolution
| Authors | Minghao Yan, Bo Peng, Benjamin Coleman, Ziqi Chen, Zhouhang Xie, Zhankui He, Noveen Sachdeva, Isabella Ye, Weili Wang, Chi Wang, Ed H. Chi, Wang-Cheng Kang, Derek Zhiyuan Cheng, Beidou Wang |
|---|---|
| Categories | |
| ArXiv ID | 2601.10657vv2 |
| URL | https://arxiv.org/abs/2601.10657 |
| License | http://creativecommons.org/licenses/by/4.0/ |
Abstract
Large Language Models (LLMs) have emerged as powerful operators for evolutionary search, yet the design of efficient search scaffolds remains ad hoc. While promising, current LLM-in-the-loop systems lack a systematic approach to managing the evolutionary process. We identify three distinct failure modes: Context Pollution, where experiment history biases future candidate generation; Mode Collapse, where agents stagnate in local minima due to poor exploration-exploitation balance; and Weak Collaboration, where rigid crossover strategies fail to leverage parallel search trajectories effectively. We introduce Progress-Aware Consistent Evolution (PACEvolve), a framework designed to robustly govern the agent's context and search dynamics, to address these challenges. PACEvolve combines hierarchical context management (HCM) with pruning to address context pollution; momentum-based backtracking (MBB) to escape local minima; and a self-adaptive sampling policy that unifies backtracking and crossover for dynamic search coordination (CE), allowing agents to balance internal refinement with cross-trajectory collaboration. We demonstrate that PACEvolve provides a systematic path to consistent, long-horizon self-improvement, achieving state-of-the-art results on LLM-SR and KernelBench, while discovering solutions surpassing the record on Modded NanoGPT.
{
"annotation_id": "d9969310-e1fa-4f52-9b21-fdc18b9a6314",
"date_created": "2026-02-17T05:53:26.429000Z",
"date_modified": "2026-02-17T05:53:26.429000Z",
"file_hash": "fdf49e2e18905a5a836faabd43a57391faf30b4650ac6569c77f7a2469e92d61",
"private": false,
"record": {
"abstract": "Large Language Models (LLMs) have emerged as powerful operators for evolutionary search, yet the design of efficient search scaffolds remains ad hoc. While promising, current LLM-in-the-loop systems lack a systematic approach to managing the evolutionary process. We identify three distinct failure modes: Context Pollution, where experiment history biases future candidate generation; Mode Collapse, where agents stagnate in local minima due to poor exploration-exploitation balance; and Weak Collaboration, where rigid crossover strategies fail to leverage parallel search trajectories effectively. We introduce Progress-Aware Consistent Evolution (PACEvolve), a framework designed to robustly govern the agent\u0027s context and search dynamics, to address these challenges. PACEvolve combines hierarchical context management (HCM) with pruning to address context pollution; momentum-based backtracking (MBB) to escape local minima; and a self-adaptive sampling policy that unifies backtracking and crossover for dynamic search coordination (CE), allowing agents to balance internal refinement with cross-trajectory collaboration. We demonstrate that PACEvolve provides a systematic path to consistent, long-horizon self-improvement, achieving state-of-the-art results on LLM-SR and KernelBench, while discovering solutions surpassing the record on Modded NanoGPT.",
"arxiv_id": "2601.10657",
"authors": [
"Minghao Yan",
"Bo Peng",
"Benjamin Coleman",
"Ziqi Chen",
"Zhouhang Xie",
"Zhankui He",
"Noveen Sachdeva",
"Isabella Ye",
"Weili Wang",
"Chi Wang",
"Ed H. Chi",
"Wang-Cheng Kang",
"Derek Zhiyuan Cheng",
"Beidou Wang"
],
"categories": [
"cs.NE",
"cs.LG"
],
"license": "http://creativecommons.org/licenses/by/4.0/",
"title": "PACEvolve: Enabling Long-Horizon Progress-Aware Consistent Evolution",
"url": "https://arxiv.org/abs/2601.10657",
"version": "v2"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "386c6dbb-ed6b-4967-ade7-051fa8a92696",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}