A Modern Self-Referential Weight Matrix That Learns to Modify Itself

Kazuki Irie; Imanol Schlag; Róbert Csordás; Juergen Schmidhuber

A Modern Self-Referential Weight Matrix That Learns to Modify Itself

Kazuki Irie, Imanol Schlag, Róbert Csordás, Juergen Schmidhuber

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

The weight matrix (WM) of a neural network (NN) is its program. The programs of many traditional NNs are learned through gradient descent in some error function, then remain fixed. The WM of a self-referential NN, however, can keep rapidly modifying all of itself during runtime. In principle, such NNs can meta-learn to learn, and meta-meta-learn to meta-learn to learn, and so on, in the sense of recursive self-improvement. While NN architectures potentially capable of implementing such behaviour have been proposed since the '90s, there have been few if any practical studies. Here we revisit such NNs, building upon recent successes of fast weight programmers and closely related linear Transformers. We propose a scalable self-referential WM (SRWM) that learns to use outer products and the delta update rule to modify itself. We evaluate our SRWM in supervised few-shot learning and in multi-task reinforcement learning with procedurally generated game environments. Our experiments demonstrate both practical applicability and competitive performance of the proposed SRWM. Our code is public.

Original language	English (US)
Title of host publication	ICML 2022 : 39th International Conference on Machine Learning
Publisher	arXiv
State	Published - Jun 17 2022

Bibliographical note

KAUST Repository Item: Exported on 2022-12-21
Acknowledgements: We would like to thank Karl Cobbe for answering some practical questions about ProcGen. Kazuki Irie wishes to thank Anand Gopalakrishnan for letting him know about ProcGen. This research was partially funded by ERC Advanced grant no: 742870, project AlgoRNN, and by Swiss National Science Foundation grant no: 200021 192356, project NEUSYM. We are thankful for hardware donations from NVIDIA & IBM. The resources used for the project were partially provided by Swiss National Supercomputing Centre (CSCS) project d115.

Access to Document

https://repository.kaust.edu.sa/bitstream/10754/686558/1/2202.05780.pdf

Cite this

@inproceedings{b4e09b04fe7b4767a20d6aa3b56dc703,

title = "A Modern Self-Referential Weight Matrix That Learns to Modify Itself",

abstract = "The weight matrix (WM) of a neural network (NN) is its program. The programs of many traditional NNs are learned through gradient descent in some error function, then remain fixed. The WM of a self-referential NN, however, can keep rapidly modifying all of itself during runtime. In principle, such NNs can meta-learn to learn, and meta-meta-learn to meta-learn to learn, and so on, in the sense of recursive self-improvement. While NN architectures potentially capable of implementing such behaviour have been proposed since the '90s, there have been few if any practical studies. Here we revisit such NNs, building upon recent successes of fast weight programmers and closely related linear Transformers. We propose a scalable self-referential WM (SRWM) that learns to use outer products and the delta update rule to modify itself. We evaluate our SRWM in supervised few-shot learning and in multi-task reinforcement learning with procedurally generated game environments. Our experiments demonstrate both practical applicability and competitive performance of the proposed SRWM. Our code is public.",

author = "Kazuki Irie and Imanol Schlag and R{\'o}bert Csord{\'a}s and Juergen Schmidhuber",

note = "KAUST Repository Item: Exported on 2022-12-21 Acknowledgements: We would like to thank Karl Cobbe for answering some practical questions about ProcGen. Kazuki Irie wishes to thank Anand Gopalakrishnan for letting him know about ProcGen. This research was partially funded by ERC Advanced grant no: 742870, project AlgoRNN, and by Swiss National Science Foundation grant no: 200021 192356, project NEUSYM. We are thankful for hardware donations from NVIDIA & IBM. The resources used for the project were partially provided by Swiss National Supercomputing Centre (CSCS) project d115.",

year = "2022",

month = jun,

day = "17",

language = "English (US)",

booktitle = "ICML 2022 : 39th International Conference on Machine Learning",

publisher = "arXiv",

}

TY - GEN

T1 - A Modern Self-Referential Weight Matrix That Learns to Modify Itself

AU - Irie, Kazuki

AU - Schlag, Imanol

AU - Csordás, Róbert

AU - Schmidhuber, Juergen

N1 - KAUST Repository Item: Exported on 2022-12-21 Acknowledgements: We would like to thank Karl Cobbe for answering some practical questions about ProcGen. Kazuki Irie wishes to thank Anand Gopalakrishnan for letting him know about ProcGen. This research was partially funded by ERC Advanced grant no: 742870, project AlgoRNN, and by Swiss National Science Foundation grant no: 200021 192356, project NEUSYM. We are thankful for hardware donations from NVIDIA & IBM. The resources used for the project were partially provided by Swiss National Supercomputing Centre (CSCS) project d115.

PY - 2022/6/17

Y1 - 2022/6/17

N2 - The weight matrix (WM) of a neural network (NN) is its program. The programs of many traditional NNs are learned through gradient descent in some error function, then remain fixed. The WM of a self-referential NN, however, can keep rapidly modifying all of itself during runtime. In principle, such NNs can meta-learn to learn, and meta-meta-learn to meta-learn to learn, and so on, in the sense of recursive self-improvement. While NN architectures potentially capable of implementing such behaviour have been proposed since the '90s, there have been few if any practical studies. Here we revisit such NNs, building upon recent successes of fast weight programmers and closely related linear Transformers. We propose a scalable self-referential WM (SRWM) that learns to use outer products and the delta update rule to modify itself. We evaluate our SRWM in supervised few-shot learning and in multi-task reinforcement learning with procedurally generated game environments. Our experiments demonstrate both practical applicability and competitive performance of the proposed SRWM. Our code is public.

AB - The weight matrix (WM) of a neural network (NN) is its program. The programs of many traditional NNs are learned through gradient descent in some error function, then remain fixed. The WM of a self-referential NN, however, can keep rapidly modifying all of itself during runtime. In principle, such NNs can meta-learn to learn, and meta-meta-learn to meta-learn to learn, and so on, in the sense of recursive self-improvement. While NN architectures potentially capable of implementing such behaviour have been proposed since the '90s, there have been few if any practical studies. Here we revisit such NNs, building upon recent successes of fast weight programmers and closely related linear Transformers. We propose a scalable self-referential WM (SRWM) that learns to use outer products and the delta update rule to modify itself. We evaluate our SRWM in supervised few-shot learning and in multi-task reinforcement learning with procedurally generated game environments. Our experiments demonstrate both practical applicability and competitive performance of the proposed SRWM. Our code is public.

UR - http://hdl.handle.net/10754/686558

UR - https://arxiv.org/pdf/2202.05780.pdf

M3 - Conference contribution

BT - ICML 2022 : 39th International Conference on Machine Learning

PB - arXiv

ER -

A Modern Self-Referential Weight Matrix That Learns to Modify Itself

Abstract

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this