TY - GEN

T1 - Evolving deep unsupervised convolutional networks for vision-based reinforcement learning

AU - Koutník, Jan

AU - Schmidhuber, Jürgen

AU - Gomez, Faustino

N1 - Generated from Scopus record by KAUST IRTS on 2022-09-14

PY - 2014/1/1

Y1 - 2014/1/1

N2 - Dealing with high-dimensional input spaces, like visual input, is a challenging task for reinforcement learning (RL). Neuroevolution (NE), used for continuous RL problems, has to either reduce the problem dimensionality by (1) compressing the representation of the neural network controllers or (2) employing a pre-processor (compressor) that transforms the high-dimensional raw inputs into low-dimensional features. In this paper, we are able to evolve extremely small recurrent neural network (RNN) controllers for a task that previously required networks with over a million weights. The high-dimensional visual input, which the controller would normally receive, is first transformed into a compact feature vector through a deep, max-pooling convolutional neural network (MPCNN). Both the MPCNN preprocessor and the RNN controller are evolved successfully to control a car in the TORCS racing simulator using only visual input. This is the first use of deep learning in the context evolutionary RL. © 2014 ACM.

AB - Dealing with high-dimensional input spaces, like visual input, is a challenging task for reinforcement learning (RL). Neuroevolution (NE), used for continuous RL problems, has to either reduce the problem dimensionality by (1) compressing the representation of the neural network controllers or (2) employing a pre-processor (compressor) that transforms the high-dimensional raw inputs into low-dimensional features. In this paper, we are able to evolve extremely small recurrent neural network (RNN) controllers for a task that previously required networks with over a million weights. The high-dimensional visual input, which the controller would normally receive, is first transformed into a compact feature vector through a deep, max-pooling convolutional neural network (MPCNN). Both the MPCNN preprocessor and the RNN controller are evolved successfully to control a car in the TORCS racing simulator using only visual input. This is the first use of deep learning in the context evolutionary RL. © 2014 ACM.

UR - https://dl.acm.org/doi/10.1145/2576768.2598358

UR - http://www.scopus.com/inward/record.url?scp=84905695541&partnerID=8YFLogxK

U2 - 10.1145/2576768.2598358

DO - 10.1145/2576768.2598358

M3 - Conference contribution

SN - 9781450326629

SP - 541

EP - 548

BT - GECCO 2014 - Proceedings of the 2014 Genetic and Evolutionary Computation Conference

PB - Association for Computing Machinery

ER -