Differentiable modelling to unify machine learning and physical models for geosciences

Chaopeng Shen, Alison P. Appling, Pierre Gentine, Toshiyuki Bandai, Hoshin Gupta, Alexandre Tartakovsky, Marco Baity-Jesi, Fabrizio Fenicia, Daniel Kifer, Li Li, Xiaofeng Liu, Wei Ren, Yi Zheng, Ciaran J. Harman, Martyn Clark, Matthew Farthing, Dapeng Feng, Praveen Kumar, Doaa Aboelyazeed, Farshid RahmaniYalan Song, Hylke Beck, Tadd Bindas, Dipankar Dwivedi, Kuai Fang, Marvin Höge, Chris Rackauckas, Binayak Mohanty, Tirthankar Roy, Chonggang Xu, Kathryn Lawson

Research output: Contribution to journalArticlepeer-review

19 Scopus citations

Abstract

Process-based modelling offers interpretability and physical consistency in many domains of geosciences but struggles to leverage large datasets efficiently. Machine-learning methods, especially deep networks, have strong predictive skills yet are unable to answer specific scientific questions. In this Perspective, we explore differentiable modelling as a pathway to dissolve the perceived barrier between process-based modelling and machine learning in the geosciences and demonstrate its potential with examples from hydrological modelling. ‘Differentiable’ refers to accurately and efficiently calculating gradients with respect to model variables or parameters, enabling the discovery of high-dimensional unknown relationships. Differentiable modelling involves connecting (flexible amounts of) prior physical knowledge to neural networks, pushing the boundary of physics-informed machine learning. It offers better interpretability, generalizability, and extrapolation capabilities than purely data-driven machine learning, achieving a similar level of accuracy while requiring less training data. Additionally, the performance and efficiency of differentiable models scale well with increasing data volumes. Under data-scarce scenarios, differentiable models have outperformed machine-learning models in producing short-term dynamics and decadal-scale trends owing to the imposed physical constraints. Differentiable modelling approaches are primed to enable geoscientists to ask questions, test hypotheses, and discover unrecognized physical relationships. Future work should address computational challenges, reduce uncertainty, and verify the physical significance of outputs.
Original languageEnglish (US)
JournalNature Reviews Earth and Environment
DOIs
StatePublished - Jul 11 2023

Bibliographical note

KAUST Repository Item: Exported on 2023-07-24
Acknowledgements: The authors are grateful for the discussion at the HydroML symposium, University Park, PA, May 2022, https://bit.ly/3g3DQNX , sponsored by National Science Foundation EAR #2015680 and Penn State Institute for Computational and Data Sciences. C.S., Y.S. and T.B. were supported by National Science Foundation Award EAR-2221880, Office of Science, US Department of Energy under award DE-SC0016605, and Cooperative Institute for Research to Operations in Hydrology (CIROH), award number A22-0307-S003, respectively. P.G. acknowledges funding from the National Science Foundational Science and Technology Center, Learning the Earth with Artificial intelligence and Physics (LEAP), award #2019625, and USMILE European Research Council grant. M. Wernimont at the US Geological Survey (USGS) greatly improved the presentation of Figs. and . A.P.A. was supported by the USGS Water Mission Area, Water Availability and Use Science Program. Any use of trade, firm or product names is for descriptive purposes only and does not imply endorsement by the US Government.

Fingerprint

Dive into the research topics of 'Differentiable modelling to unify machine learning and physical models for geosciences'. Together they form a unique fingerprint.

Cite this