Hierarchical controller learning in a first-person shooter

Niels Van Hoorn, Julian Togelius, Jürgen Schmidhuber

Research output: Chapter in Book/Report/Conference proceedingConference contribution

37 Scopus citations


We describe the architecture of a hierarchical learning-based controller for bots in the First-Person Shooter (FPS) game Unreal Tournament 2004. The controller is inspired by the subsumption architecture commonly used in behaviour-based robotics. A behaviour selector decides which of three sub-controllers gets to control the bot at each time step. Each controller is implemented as a recurrent neural network, and trained with artificial evolution to perform respectively combat, exploration and path following. The behaviour selector is trained with a multiobjective evolutionary algorithm to achieve an effective balancing of the lower-level behaviours. We argue that FPS games provide good environments for studying the learning of complex behaviours, and that the methods proposed here can help developing interesting opponents for games. ©2009 IEEE.
Original languageEnglish (US)
Title of host publicationCIG2009 - 2009 IEEE Symposium on Computational Intelligence and Games
Number of pages8
StatePublished - Dec 14 2009
Externally publishedYes

Bibliographical note

Generated from Scopus record by KAUST IRTS on 2022-09-14


Dive into the research topics of 'Hierarchical controller learning in a first-person shooter'. Together they form a unique fingerprint.

Cite this