Modeling systems with internal state using evolino

Daan Wierstra, Faustino J. Gomez, Jürgen Schmidhuber

Research output: Chapter in Book/Report/Conference proceedingConference contribution

39 Scopus citations

Abstract

Existing Recurrent Neural Networks (RNNs) are limited in their ability to model dynamical systems with nonlinearities and hidden internal states. Here we use our general framework for sequence learning, EVOlution of recurrent systems with LINear Outputs (Evolino), to discover good RNN hidden node weights through evolution, while using linear regression to compute an optimal linear mapping from hidden state to output. Using the Long Short-Term Memory RNN Architecture, Evolino outperforms previous state-of-the-art methods on several tasks: 1) context-sensitive languages, 2) multiple superimposed sine waves. Copyright 2005 ACM.
Original languageEnglish (US)
Title of host publicationGECCO 2005 - Genetic and Evolutionary Computation Conference
Pages1795-1802
Number of pages8
DOIs
StatePublished - Dec 1 2005
Externally publishedYes

Fingerprint

Dive into the research topics of 'Modeling systems with internal state using evolino'. Together they form a unique fingerprint.

Cite this