Learning Team Strategies: Soccer Case Studies

Rafał P. Sałustowicz, Marco A. Wiering, Jürgen Schmidhuber

Research output: Contribution to journalArticlepeer-review

38 Scopus citations


We use simulated soccer to study multiagent learning. Each team's players (agents) share action set and policy, but may behave differently due to position-dependent inputs. All agents making up a team are rewarded or punished collectively in case of goals. We conduct simulations with varying team sizes, and compare several learning algorithms: TD-Q learning with linear neural networks (TD-Q), Probabilistic Incremental Program Evolution (PIPE), and a PIPE version that learns by coevolution (CO-PIPE). TD-Q is based on learning evaluation functions (EFs) mapping input/action pairs to expected reward. PIPE and CO-PIPE search policy space directly. They use adaptive probability distributions to synthesize programs that calculate action probabilities from current inputs. Our results show that linear TD-Q encounters several difficulties in learning appropriate shared EFs. PIPE and CO-PIPE, however, do not depend on EFs and find good policies faster and more reliably. This suggests that in some multiagent learning scenarios direct search in policy space can offer advantages over EF-based approaches.
Original languageEnglish (US)
Pages (from-to)263-282
Number of pages20
JournalMachine Learning
Issue number4
StatePublished - Jan 1 1998
Externally publishedYes

Bibliographical note

Generated from Scopus record by KAUST IRTS on 2022-09-14

ASJC Scopus subject areas

  • Artificial Intelligence
  • Software


Dive into the research topics of 'Learning Team Strategies: Soccer Case Studies'. Together they form a unique fingerprint.

Cite this