LSTM recurrent networks learn simple context-free and context-sensitive languages

F. A. Gers; J. Schmidhuber

doi:10.1109/72.963769

LSTM recurrent networks learn simple context-free and context-sensitive languages

F. A. Gers, J. Schmidhuber

Research output: Contribution to journal › Article › peer-review

553 Scopus citations

Abstract

Previous work on learning regular languages from exemplary training sequences showed that long short-term memory (LSTM) outperforms traditional recurrent neural networks (RNNs). Here we demonstrate LSTMs superior performance on context-free language (CFL) benchmarks for RNNs, and show that it works even better than previous hardwired or highly specialized architectures. To the best of our knowledge, LSTM variants are also the first RNNs to learn a simple context-sensitive language (CSL), namely anbncn.

Original language	English (US)
Pages (from-to)	1333-1340
Number of pages	8
Journal	IEEE Transactions on Neural Networks
Volume	12
Issue number	6
DOIs	https://doi.org/10.1109/72.963769
State	Published - Nov 1 2001
Externally published	Yes

Bibliographical note

Generated from Scopus record by KAUST IRTS on 2022-09-14

ASJC Scopus subject areas

Artificial Intelligence
Software
Computer Networks and Communications
Computer Science Applications

Access to Document

10.1109/72.963769

Cite this

@article{03d20eea9be2461290a8bbe56598f9b5,

title = "LSTM recurrent networks learn simple context-free and context-sensitive languages",

abstract = "Previous work on learning regular languages from exemplary training sequences showed that long short-term memory (LSTM) outperforms traditional recurrent neural networks (RNNs). Here we demonstrate LSTMs superior performance on context-free language (CFL) benchmarks for RNNs, and show that it works even better than previous hardwired or highly specialized architectures. To the best of our knowledge, LSTM variants are also the first RNNs to learn a simple context-sensitive language (CSL), namely anbncn.",

author = "Gers, {F. A.} and J. Schmidhuber",

note = "Generated from Scopus record by KAUST IRTS on 2022-09-14",

year = "2001",

month = nov,

day = "1",

doi = "10.1109/72.963769",

language = "English (US)",

volume = "12",

pages = "1333--1340",

journal = "IEEE Transactions on Neural Networks",

issn = "1045-9227",

publisher = "IEEE Computational Intelligence Society",

number = "6",

}

TY - JOUR

T1 - LSTM recurrent networks learn simple context-free and context-sensitive languages

AU - Gers, F. A.

AU - Schmidhuber, J.

N1 - Generated from Scopus record by KAUST IRTS on 2022-09-14

PY - 2001/11/1

Y1 - 2001/11/1

N2 - Previous work on learning regular languages from exemplary training sequences showed that long short-term memory (LSTM) outperforms traditional recurrent neural networks (RNNs). Here we demonstrate LSTMs superior performance on context-free language (CFL) benchmarks for RNNs, and show that it works even better than previous hardwired or highly specialized architectures. To the best of our knowledge, LSTM variants are also the first RNNs to learn a simple context-sensitive language (CSL), namely anbncn.

AB - Previous work on learning regular languages from exemplary training sequences showed that long short-term memory (LSTM) outperforms traditional recurrent neural networks (RNNs). Here we demonstrate LSTMs superior performance on context-free language (CFL) benchmarks for RNNs, and show that it works even better than previous hardwired or highly specialized architectures. To the best of our knowledge, LSTM variants are also the first RNNs to learn a simple context-sensitive language (CSL), namely anbncn.

UR - http://ieeexplore.ieee.org/document/963769/

UR - http://www.scopus.com/inward/record.url?scp=0035505385&partnerID=8YFLogxK

U2 - 10.1109/72.963769

DO - 10.1109/72.963769

M3 - Article

SN - 1045-9227

VL - 12

SP - 1333

EP - 1340

JO - IEEE Transactions on Neural Networks

JF - IEEE Transactions on Neural Networks

IS - 6

ER -

LSTM recurrent networks learn simple context-free and context-sensitive languages

Abstract

Bibliographical note

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this