Compete to compute

Rupesh Kumar Srivastava; Jonathan Masci; Sohrob Kazerounian; Faustino Gomez; Jürgen Schmidhuber

Compete to compute

Rupesh Kumar Srivastava, Jonathan Masci, Sohrob Kazerounian, Faustino Gomez, Jürgen Schmidhuber

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Local competition among neighboring neurons is common in biological neural networks (NNs). In this paper, we apply the concept to gradient-based, backprop-trained artificial multilayer NNs. NNs with competing linear units tend to outperform those with non-competing nonlinear units, and avoid catastrophic forgetting when training sets change over time.

Original language	English (US)
Title of host publication	Advances in Neural Information Processing Systems
Publisher	Neural information processing systems foundation
State	Published - Jan 1 2013
Externally published	Yes

Bibliographical note

Generated from Scopus record by KAUST IRTS on 2022-09-14

Cite this

@inproceedings{9d6fdfadd3c647568db4f8913bbd1f6a,

title = "Compete to compute",

abstract = "Local competition among neighboring neurons is common in biological neural networks (NNs). In this paper, we apply the concept to gradient-based, backprop-trained artificial multilayer NNs. NNs with competing linear units tend to outperform those with non-competing nonlinear units, and avoid catastrophic forgetting when training sets change over time.",

author = "Srivastava, {Rupesh Kumar} and Jonathan Masci and Sohrob Kazerounian and Faustino Gomez and J{\"u}rgen Schmidhuber",

note = "Generated from Scopus record by KAUST IRTS on 2022-09-14",

year = "2013",

month = jan,

day = "1",

language = "English (US)",

booktitle = "Advances in Neural Information Processing Systems",

publisher = "Neural information processing systems foundation",

}

TY - GEN

T1 - Compete to compute

AU - Srivastava, Rupesh Kumar

AU - Masci, Jonathan

AU - Kazerounian, Sohrob

AU - Gomez, Faustino

AU - Schmidhuber, Jürgen

N1 - Generated from Scopus record by KAUST IRTS on 2022-09-14

PY - 2013/1/1

Y1 - 2013/1/1

N2 - Local competition among neighboring neurons is common in biological neural networks (NNs). In this paper, we apply the concept to gradient-based, backprop-trained artificial multilayer NNs. NNs with competing linear units tend to outperform those with non-competing nonlinear units, and avoid catastrophic forgetting when training sets change over time.

AB - Local competition among neighboring neurons is common in biological neural networks (NNs). In this paper, we apply the concept to gradient-based, backprop-trained artificial multilayer NNs. NNs with competing linear units tend to outperform those with non-competing nonlinear units, and avoid catastrophic forgetting when training sets change over time.

UR - http://www.scopus.com/inward/record.url?scp=84898978862&partnerID=8YFLogxK

M3 - Conference contribution

BT - Advances in Neural Information Processing Systems

PB - Neural information processing systems foundation

ER -

Compete to compute

Abstract

Bibliographical note

Other files and links

Fingerprint

Cite this