The Method of Manufactured Universes for validating uncertainty quantification methods

H.F. Stripling, M.L. Adams, R.G. McClarren, B.K. Mallick

Research output: Contribution to journalArticlepeer-review

16 Scopus citations


The Method of Manufactured Universes is presented as a validation framework for uncertainty quantification (UQ) methodologies and as a tool for exploring the effects of statistical and modeling assumptions embedded in these methods. The framework calls for a manufactured reality from which experimental data are created (possibly with experimental error), an imperfect model (with uncertain inputs) from which simulation results are created (possibly with numerical error), the application of a system for quantifying uncertainties in model predictions, and an assessment of how accurately those uncertainties are quantified. The application presented in this paper manufactures a particle-transport universe, models it using diffusion theory with uncertain material parameters, and applies both Gaussian process and Bayesian MARS algorithms to make quantitative predictions about new experiments within the manufactured reality. The results of this preliminary study indicate that, even in a simple problem, the improper application of a specific UQ method or unrealized effects of a modeling assumption may produce inaccurate predictions. We conclude that the validation framework presented in this paper is a powerful and flexible tool for the investigation and understanding of UQ methodologies. © 2011 Elsevier Ltd. All rights reserved.
Original languageEnglish (US)
Pages (from-to)1242-1256
Number of pages15
JournalReliability Engineering & System Safety
Issue number9
StatePublished - Sep 2011
Externally publishedYes

Bibliographical note

KAUST Repository Item: Exported on 2020-10-01
Acknowledged KAUST grant number(s): KUS-C1-016-04
Acknowledgements: We thank Derek Bingham (Simon Fraser University) for helpful conversations and for sharing some helpful UQ software. Work by the first author was supported by the Department of Energy Computational Science Graduate Fellowship program under Grant no. DE-FG02-97ER25308. Work by other authors was partially supported by the Predictive Sciences Academic Alliances Program in DOE NNSA-ASC under Grant DE-FC52-08NA28616, and the Lawrence Livermore National Laboratory. This publication is based in part on work supported by Award No. KUS-C1-016-04 made by the King Abdullah University of Science and Technology (KAUST). Finally, we are grateful for the effort of one journal referee, whose comments and suggestions greatly strengthened the content of this paper.
This publication acknowledges KAUST support, but has no KAUST affiliated authors.


Dive into the research topics of 'The Method of Manufactured Universes for validating uncertainty quantification methods'. Together they form a unique fingerprint.

Cite this