Optimal control using the transport equation: The Liouville machine

Research output: Contribution to journalArticlepeer-review

2 Scopus citations

Abstract

Transport theory describes the scattering behavior of physical particles such as photons. Here we show how to connect this theory to optimal control theory and to adaptive behavior of agents embedded in an environment. Environments and tasks are defined by physical boundary conditions. Given some task, we compute a set of probability densities on continuous state and action and time. From these densities we derive an optimal policy such that for all states the most likely action maximizes the probability of reaching a predefined goal state. Liouville’s conservation theorem tells us that the conditional density al time t, state s, and action a must equal the density at t+ dt, s+ ds, a+ da. Discretization yields a linear system that can be solved directly and whose solution corresponds to an optimal policy. Discounted reward schemes are incorporated naturally by taking the Laplace transform of the equations. The Liouville machine quickly solves rather complex maze problems.
Original languageEnglish (US)
Pages (from-to)105-118
Number of pages14
JournalAdaptive Behavior
Volume9
Issue number2
DOIs
StatePublished - Jan 1 2001
Externally publishedYes

Bibliographical note

Generated from Scopus record by KAUST IRTS on 2022-09-14

ASJC Scopus subject areas

  • Behavioral Neuroscience
  • Experimental and Cognitive Psychology

Fingerprint

Dive into the research topics of 'Optimal control using the transport equation: The Liouville machine'. Together they form a unique fingerprint.

Cite this