Evolving deep unsupervised convolutional networks for vision-based reinforcement learning

Jan Koutník, Jürgen Schmidhuber, Faustino Gomez

Research output: Chapter in Book/Report/Conference proceedingConference contribution

87 Scopus citations


Dealing with high-dimensional input spaces, like visual input, is a challenging task for reinforcement learning (RL). Neuroevolution (NE), used for continuous RL problems, has to either reduce the problem dimensionality by (1) compressing the representation of the neural network controllers or (2) employing a pre-processor (compressor) that transforms the high-dimensional raw inputs into low-dimensional features. In this paper, we are able to evolve extremely small recurrent neural network (RNN) controllers for a task that previously required networks with over a million weights. The high-dimensional visual input, which the controller would normally receive, is first transformed into a compact feature vector through a deep, max-pooling convolutional neural network (MPCNN). Both the MPCNN preprocessor and the RNN controller are evolved successfully to control a car in the TORCS racing simulator using only visual input. This is the first use of deep learning in the context evolutionary RL. © 2014 ACM.
Original languageEnglish (US)
Title of host publicationGECCO 2014 - Proceedings of the 2014 Genetic and Evolutionary Computation Conference
PublisherAssociation for Computing Machinery
Number of pages8
ISBN (Print)9781450326629
StatePublished - Jan 1 2014
Externally publishedYes

Bibliographical note

Generated from Scopus record by KAUST IRTS on 2022-09-14


Dive into the research topics of 'Evolving deep unsupervised convolutional networks for vision-based reinforcement learning'. Together they form a unique fingerprint.

Cite this