Learning-based importance sampling via stochastic optimal control for stochastic reaction networks

Chiheb Ben Hammouda, Nadhir Ben Rached, Raul Tempone, Sophia Wiechert

Research output: Contribution to journalArticlepeer-review

1 Scopus citations


We explore efficient estimation of statistical quantities, particularly rare event probabilities, for stochastic reaction networks. Consequently, we propose an importance sampling (IS) approach to improve the Monte Carlo (MC) estimator efficiency based on an approximate tau-leap scheme. The crucial step in the IS framework is choosing an appropriate change of probability measure to achieve substantial variance reduction. This task is typically challenging and often requires insights into the underlying problem. Therefore, we propose an automated approach to obtain a highly efficient path-dependent measure change based on an original connection in the stochastic reaction network context between finding optimal IS parameters within a class of probability measures and a stochastic optimal control formulation. Optimal IS parameters are obtained by solving a variance minimization problem. First, we derive an associated dynamic programming equation. Analytically solving this backward equation is challenging, hence we propose an approximate dynamic programming formulation to find near-optimal control parameters. To mitigate the curse of dimensionality, we propose a learning-based method to approximate the value function using a neural network, where the parameters are determined via a stochastic optimization algorithm. Our analysis and numerical experiments verify that the proposed learning-based IS approach substantially reduces MC estimator variance, resulting in a lower computational complexity in the rare event regime, compared with standard tau-leap MC estimators.
Original languageEnglish (US)
JournalStatistics and Computing
Issue number3
StatePublished - Mar 28 2023

Bibliographical note

KAUST Repository Item: Exported on 2023-04-12
Acknowledged KAUST grant number(s): OSR-2019-CRG8-4033
Acknowledgements: This publication is based upon work supported by the King Abdullah University of Science and Technology (KAUST) Office of Sponsored Research (OSR) under Award No. OSR-2019-CRG8-4033. This work was partially performed as part of the Helmholtz School for Data Science in Life, Earth and Energy (HDS-LEE) and received funding from the Helmholtz Association of German Research Centres and the Alexander von Humboldt Foundation. Open Access funding enabled and organized by Projekt DEAL.

ASJC Scopus subject areas

  • Computational Theory and Mathematics
  • Theoretical Computer Science
  • Statistics and Probability
  • Statistics, Probability and Uncertainty


Dive into the research topics of 'Learning-based importance sampling via stochastic optimal control for stochastic reaction networks'. Together they form a unique fingerprint.

Cite this