Confidence-based progress-driven self-generated goals for skill acquisition in developmental robots

Hung Ngo, Matthew Luciw, Alexander Förster, Jürgen Schmidhuber

Research output: Contribution to journalArticlepeer-review

10 Scopus citations

Abstract

A reinforcement learning agent that autonomously explores its environment can utilize a curiosity drive to enable continual learning of skills, in the absence of any external rewards. We formulate curiosity-driven exploration, and eventual skill acquisition, as a selective sampling problem. Each environment setting provides the agent with a stream of instances. An instance is a sensory observation that, when queried, causes an outcome that the agent is trying to predict. After an instance is observed, a query condition, derived herein, tells whether its outcome is statistically known or unknown to the agent, based on the confidence interval of an online linear classifier. Upon encountering the first unknown instance, the agent "queries" the environment to observe the outcome, which is expected to improve its confidence in the corresponding predictor. If the environment is in a setting where all instances are known, the agent generates a plan of actions to reach a new setting, where an unknown instance is likely to be encountered. The desired setting is a self-generated goal, and the plan of action, essentially a program to solve a problem, is a skill. The success of the plan depends on the quality of the agent's predictors, which are improved as mentioned above. For validation, this method is applied to both a simulated and real Katana robot arm in its "blocks-world" environment. Results show that the proposed method generates sample-efficient curious exploration behavior, which exhibits developmental stages, continual learning, and skill acquisition, in an intrinsically-motivated playful agent. © 2013 Ngo, Luciw, Förster and Schmidhuber.
Original languageEnglish (US)
JournalFrontiers in Psychology
Volume4
Issue numberNOV
DOIs
StatePublished - Dec 12 2013
Externally publishedYes

ASJC Scopus subject areas

  • Psychology(all)

Fingerprint

Dive into the research topics of 'Confidence-based progress-driven self-generated goals for skill acquisition in developmental robots'. Together they form a unique fingerprint.

Cite this