Natural Evolution Strategies

Daan Wierstra, Tom Schaul, Jan Peters, Juergen Schmidhuber

Research output: Chapter in Book/Report/Conference proceedingConference contribution

176 Scopus citations

Abstract

This paper presents Natural Evolution Strategies (NES), a novel algorithm for performing real-valued 'black box' function optimization: optimizing an unknown objective function where algorithm-selected function measurements constitute the only information accessible to the method. Natural Evolution Strategies search the fitness landscape using a multivariate normal distribution with a self-adapting mutation matrix to generate correlated mutations in promising regions. NES shares this property with Covariance Matrix Adaption (CMA), an Evolution Strategy (ES) which has been shown to perform well on a variety of high-precision optimization tasks. The Natural Evolution Strategies algorithm, however, is simpler, less ad-hoc and more principled. Self-adaptation of the mutation matrix is derived using a Monte Carlo estimate of the natural gradient towards better expected fitness. By following the natural gradient instead of the 'vanilla' gradient, we can ensure efficient update steps while preventing early convergence due to overly greedy updates, resulting in reduced sensitivity to local suboptima. We show NES has competitive performance with CMA on unimodal tasks, while outperforming it on several multimodal tasks that are rich in deceptive local optima. © 2008 IEEE.
Original languageEnglish (US)
Title of host publication2008 IEEE Congress on Evolutionary Computation, CEC 2008
Pages3381-3387
Number of pages7
DOIs
StatePublished - Nov 14 2008
Externally publishedYes

Fingerprint

Dive into the research topics of 'Natural Evolution Strategies'. Together they form a unique fingerprint.

Cite this