Low complexity proto-value function learning from sensory observations with incremental slow feature analysis

Matthew Luciw, Juergen Schmidhuber

Research output: Chapter in Book/Report/Conference proceedingConference contribution

13 Scopus citations

Abstract

We show that Incremental Slow Feature Analysis (IncSFA) provides a low complexity method for learning Proto-Value Functions (PVFs). It has been shown that a small number of PVFs provide a good basis set for linear approximation of value functions in reinforcement environments. Our method learns PVFs from a high-dimensional sensory input stream, as the agent explores its world, without building a transition model, adjacency matrix, or covariance matrix. A temporal-difference based reinforcement learner improves a value function approximation upon the features, and the agent uses the value function to achieve rewards successfully. The algorithm is local in space and time, furthering the biological plausibility and applicability of PVFs. © 2012 Springer-Verlag.
Original languageEnglish (US)
Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Pages279-287
Number of pages9
DOIs
StatePublished - Oct 25 2012
Externally publishedYes

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'Low complexity proto-value function learning from sensory observations with incremental slow feature analysis'. Together they form a unique fingerprint.

Cite this