Learning precise timing with LSTM recurrent networks

Felix A. Gers, Nicol N. Schraudolph, Jürgen Schmidhuber

Research output: Contribution to journalArticlepeer-review

1265 Scopus citations


The temporal distance between events conveys information essential for numerous sequential tasks such as motor control and rhythm detection. While Hidden Markov Models tend to ignore this information, recurrent neural networks (RNNs) can in principle learn to make use of it. We focus on Long Short-Term Memory (LSTM) because it has been shown to outperform other RNNs on tasks involving long time lags. We find that LSTM augmented by "peephole connections" from its internal cells to its multiplicative gates can learn the fine distinction between sequences of spikes spaced either 50 or 49 time steps apart without the help of any short training exemplars. Without external resets or teacher forcing, our LSTM variant also learns to generate stable streams of precisely timed spikes and other highly nonlinear periodic patterns. This makes LSTM a promising approach for tasks that require the accurate measurement or generation of time intervals.
Original languageEnglish (US)
Pages (from-to)115-143
Number of pages29
JournalJournal of Machine Learning Research
Issue number1
StatePublished - Jan 1 2003
Externally publishedYes

Bibliographical note

Generated from Scopus record by KAUST IRTS on 2022-09-14

ASJC Scopus subject areas

  • Artificial Intelligence
  • Software
  • Statistics and Probability
  • Control and Systems Engineering


Dive into the research topics of 'Learning precise timing with LSTM recurrent networks'. Together they form a unique fingerprint.

Cite this