Evolution strategies match reinforcement learning for billion-parameter LLM fine-tuning, arXiv preprint finds

An arXiv preprint demonstrates the first successful application of evolution strategies (ES) to full-parameter fine-tuning of large language models at the billion-parameter scale without dimensionality reduction. The paper challenges the widespread assumption that ES does not scale to modern model sizes. The authors show that ES can search over extremely high-dimensional parameter spaces and outperforms established reinforcement learning (RL) implementations across multiple axes, including improved tolerance to long-horizon and delayed rewards, robustness across diverse base LLMs, reduced susceptibility to reward hacking, and improved training stability. The findings suggest ES is not merely a viable alternative to RL but a fundamentally different and powerful backpropagation-free post-training paradigm that opens a new direction for LLM fine-tuning beyond current RL-based approaches. RL has become the dominant fine-tuning paradigm underpinning many state-of-the-art LLMs, while ES has been largely overlooked due to scaling concerns. This work overturns that assumption by demonstrating ES effectiveness at billion-parameter scale for the first time.