Learning One Abstract Bit at a Time Through Self-invented Experiments Encoded as Neural Networks

Vincent Herrmann; Louis Kirsch; Jürgen Schmidhuber

doi:10.1007/978-3-031-47958-8_16

Learning One Abstract Bit at a Time Through Self-invented Experiments Encoded as Neural Networks

Vincent Herrmann^*, Louis Kirsch, Jürgen Schmidhuber

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

There are two important things in science: (A) Finding answers to given questions, and (B) Coming up with good questions. Our artificial scientists not only learn to answer given questions, but also continually invent new questions, by proposing hypotheses to be verified or falsified through potentially complex and time-consuming experiments, including thought experiments akin to those of mathematicians. While an artificial scientist expands its knowledge, it remains biased towards the simplest, least costly experiments that still have surprising outcomes, until they become boring. We present an empirical analysis of the automatic generation of interesting experiments. In the first setting, we investigate self-invented experiments in a reinforcement-providing environment and show that they lead to effective exploration. In the second setting, pure thought experiments are implemented as the weights of recurrent neural networks generated by a neural experiment generator. Initially interesting thought experiments may become boring over time.

Original language	English (US)
Title of host publication	Active Inference - 4th International Workshop, IWAI 2023, Revised Selected Papers
Editors	Christopher L. Buckley, Daniela Cialfi, Pablo Lanillos, Maxwell Ramstead, Tim Verbelen, Maxwell Ramstead, Noor Sajid, Hideaki Shimazaki, Martijn Wisse
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	254-274
Number of pages	21
ISBN (Print)	9783031479571
DOIs	https://doi.org/10.1007/978-3-031-47958-8_16
State	Published - 2024
Event	4th International Workshop on Active Inference, IWAI 2023 - Ghent, Belgium Duration: Sep 13 2023 → Sep 15 2023

Publication series

Name	Communications in Computer and Information Science
Volume	1915 CCIS
ISSN (Print)	1865-0929
ISSN (Electronic)	1865-0937

Conference

Conference	4th International Workshop on Active Inference, IWAI 2023
Country/Territory	Belgium
City	Ghent
Period	09/13/23 → 09/15/23

Bibliographical note

Publisher Copyright:
© 2024, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Keywords

Exploration
Reinforcement Learning

ASJC Scopus subject areas

General Computer Science
General Mathematics

Access to Document

10.1007/978-3-031-47958-8_16

Cite this

Herrmann, V., Kirsch, L., & Schmidhuber, J. (2024). Learning One Abstract Bit at a Time Through Self-invented Experiments Encoded as Neural Networks. In C. L. Buckley, D. Cialfi, P. Lanillos, M. Ramstead, T. Verbelen, M. Ramstead, N. Sajid, H. Shimazaki, & M. Wisse (Eds.), Active Inference - 4th International Workshop, IWAI 2023, Revised Selected Papers (pp. 254-274). (Communications in Computer and Information Science; Vol. 1915 CCIS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-47958-8_16

Herrmann, Vincent ; Kirsch, Louis ; Schmidhuber, Jürgen. / Learning One Abstract Bit at a Time Through Self-invented Experiments Encoded as Neural Networks. Active Inference - 4th International Workshop, IWAI 2023, Revised Selected Papers. editor / Christopher L. Buckley ; Daniela Cialfi ; Pablo Lanillos ; Maxwell Ramstead ; Tim Verbelen ; Maxwell Ramstead ; Noor Sajid ; Hideaki Shimazaki ; Martijn Wisse. Springer Science and Business Media Deutschland GmbH, 2024. pp. 254-274 (Communications in Computer and Information Science).

@inproceedings{4513ad3dcdba4aa5a82c713e1b0ce78f,

title = "Learning One Abstract Bit at a Time Through Self-invented Experiments Encoded as Neural Networks",

abstract = "There are two important things in science: (A) Finding answers to given questions, and (B) Coming up with good questions. Our artificial scientists not only learn to answer given questions, but also continually invent new questions, by proposing hypotheses to be verified or falsified through potentially complex and time-consuming experiments, including thought experiments akin to those of mathematicians. While an artificial scientist expands its knowledge, it remains biased towards the simplest, least costly experiments that still have surprising outcomes, until they become boring. We present an empirical analysis of the automatic generation of interesting experiments. In the first setting, we investigate self-invented experiments in a reinforcement-providing environment and show that they lead to effective exploration. In the second setting, pure thought experiments are implemented as the weights of recurrent neural networks generated by a neural experiment generator. Initially interesting thought experiments may become boring over time.",

keywords = "Exploration, Reinforcement Learning",

author = "Vincent Herrmann and Louis Kirsch and J{\"u}rgen Schmidhuber",

note = "Publisher Copyright: {\textcopyright} 2024, The Author(s), under exclusive license to Springer Nature Switzerland AG.; 4th International Workshop on Active Inference, IWAI 2023 ; Conference date: 13-09-2023 Through 15-09-2023",

year = "2024",

doi = "10.1007/978-3-031-47958-8_16",

language = "English (US)",

isbn = "9783031479571",

series = "Communications in Computer and Information Science",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "254--274",

editor = "Buckley, {Christopher L.} and Daniela Cialfi and Pablo Lanillos and Maxwell Ramstead and Tim Verbelen and Maxwell Ramstead and Noor Sajid and Hideaki Shimazaki and Martijn Wisse",

booktitle = "Active Inference - 4th International Workshop, IWAI 2023, Revised Selected Papers",

address = "Germany",

}

Herrmann, V, Kirsch, L & Schmidhuber, J 2024, Learning One Abstract Bit at a Time Through Self-invented Experiments Encoded as Neural Networks. in CL Buckley, D Cialfi, P Lanillos, M Ramstead, T Verbelen, M Ramstead, N Sajid, H Shimazaki & M Wisse (eds), Active Inference - 4th International Workshop, IWAI 2023, Revised Selected Papers. Communications in Computer and Information Science, vol. 1915 CCIS, Springer Science and Business Media Deutschland GmbH, pp. 254-274, 4th International Workshop on Active Inference, IWAI 2023, Ghent, Belgium, 09/13/23. https://doi.org/10.1007/978-3-031-47958-8_16

Learning One Abstract Bit at a Time Through Self-invented Experiments Encoded as Neural Networks. / Herrmann, Vincent; Kirsch, Louis; Schmidhuber, Jürgen.
Active Inference - 4th International Workshop, IWAI 2023, Revised Selected Papers. ed. / Christopher L. Buckley; Daniela Cialfi; Pablo Lanillos; Maxwell Ramstead; Tim Verbelen; Maxwell Ramstead; Noor Sajid; Hideaki Shimazaki; Martijn Wisse. Springer Science and Business Media Deutschland GmbH, 2024. p. 254-274 (Communications in Computer and Information Science; Vol. 1915 CCIS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Learning One Abstract Bit at a Time Through Self-invented Experiments Encoded as Neural Networks

AU - Herrmann, Vincent

AU - Kirsch, Louis

AU - Schmidhuber, Jürgen

PY - 2024

Y1 - 2024

N2 - There are two important things in science: (A) Finding answers to given questions, and (B) Coming up with good questions. Our artificial scientists not only learn to answer given questions, but also continually invent new questions, by proposing hypotheses to be verified or falsified through potentially complex and time-consuming experiments, including thought experiments akin to those of mathematicians. While an artificial scientist expands its knowledge, it remains biased towards the simplest, least costly experiments that still have surprising outcomes, until they become boring. We present an empirical analysis of the automatic generation of interesting experiments. In the first setting, we investigate self-invented experiments in a reinforcement-providing environment and show that they lead to effective exploration. In the second setting, pure thought experiments are implemented as the weights of recurrent neural networks generated by a neural experiment generator. Initially interesting thought experiments may become boring over time.

AB - There are two important things in science: (A) Finding answers to given questions, and (B) Coming up with good questions. Our artificial scientists not only learn to answer given questions, but also continually invent new questions, by proposing hypotheses to be verified or falsified through potentially complex and time-consuming experiments, including thought experiments akin to those of mathematicians. While an artificial scientist expands its knowledge, it remains biased towards the simplest, least costly experiments that still have surprising outcomes, until they become boring. We present an empirical analysis of the automatic generation of interesting experiments. In the first setting, we investigate self-invented experiments in a reinforcement-providing environment and show that they lead to effective exploration. In the second setting, pure thought experiments are implemented as the weights of recurrent neural networks generated by a neural experiment generator. Initially interesting thought experiments may become boring over time.

KW - Exploration

KW - Reinforcement Learning

UR - http://www.scopus.com/inward/record.url?scp=85177830238&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-47958-8_16

DO - 10.1007/978-3-031-47958-8_16

M3 - Conference contribution

AN - SCOPUS:85177830238

SN - 9783031479571

T3 - Communications in Computer and Information Science

SP - 254

EP - 274

BT - Active Inference - 4th International Workshop, IWAI 2023, Revised Selected Papers

A2 - Buckley, Christopher L.

A2 - Cialfi, Daniela

A2 - Lanillos, Pablo

A2 - Ramstead, Maxwell

A2 - Verbelen, Tim

A2 - Ramstead, Maxwell

A2 - Sajid, Noor

A2 - Shimazaki, Hideaki

A2 - Wisse, Martijn

PB - Springer Science and Business Media Deutschland GmbH

T2 - 4th International Workshop on Active Inference, IWAI 2023

Y2 - 13 September 2023 through 15 September 2023

ER -

Herrmann V, Kirsch L, Schmidhuber J. Learning One Abstract Bit at a Time Through Self-invented Experiments Encoded as Neural Networks. In Buckley CL, Cialfi D, Lanillos P, Ramstead M, Verbelen T, Ramstead M, Sajid N, Shimazaki H, Wisse M, editors, Active Inference - 4th International Workshop, IWAI 2023, Revised Selected Papers. Springer Science and Business Media Deutschland GmbH. 2024. p. 254-274. (Communications in Computer and Information Science). doi: 10.1007/978-3-031-47958-8_16