Extreme Scale Multi-Physics Simulations of the Tsunamigenic 2004 Sumatra Megathrust Earthquake

Carsten Uphoff; Sebastian Rettenberger; Michael Bader; Elizabeth H. Madden; Thomas Ulrich; Stephanie Wollherr; Alice-Agnes Gabriel

doi:10.1145/3126908.3126948

Extreme Scale Multi-Physics Simulations of the Tsunamigenic 2004 Sumatra Megathrust Earthquake

Carsten Uphoff, Sebastian Rettenberger, Michael Bader, Elizabeth H. Madden, Thomas Ulrich, Stephanie Wollherr, Alice-Agnes Gabriel

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

62 Scopus citations

Abstract

We present a high-resolution simulation of the 2004 Sumatra-Andaman earthquake, including non-linear frictional failure on a megathrust-splay fault system. Our method exploits unstructured meshes capturing the complicated geometries in subduction zones that are crucial to understand large earthquakes and tsunami generation. These up-to-date largest and longest dynamic rupture simulations enable analysis of dynamic source effects on the seafloor displacements. To tackle the extreme size of this scenario an end-to-end optimization of the simulation code SeisSol was necessary. We implemented a new cache-aware wave propagation scheme and optimized the dynamic rupture kernels using code generation. We established a novel clustered local-time-stepping scheme for dynamic rupture. In total, we achieved a speed-up of 13.6 compared to the previous implementation. For the Sumatra scenario with 221 million elements this reduced the time-to-solution to 13.9 hours on 86,016 Haswell cores. Furthermore, we used asynchronous output to overlap I/O and compute time.

Original language	English (US)
Title of host publication	Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis
Publisher	ACM
ISBN (Print)	9781450351140
DOIs	https://doi.org/10.1145/3126908.3126948
State	Published - Nov 12 2017
Externally published	Yes

Bibliographical note

KAUST Repository Item: Exported on 2022-06-23
Acknowledgements: The work presented in this paper was supported by the Volkswagen Foundation (project ASCETE - Advanced Simulation of Coupled Earthquake-Tsunami Events, grant no. 88479), by the German Research Foundation (DFG) (project no. KA 2281/4-1, AOBJ 584936/TG-92), by the Bavarian Competence Network for Technical and Scientific High Performance Computing (KONWIHR) (project GeoPF - Geophysics for PetaFlop Computing), and by Intel as part of the Intel Parallel Computing Center ExScaMIC-KNL. Computing resources were provided by the Leibniz Supercomputing Centre (LRZ, project no. pr45f and h019z, on SuperMUC), by P. Martin Mai, King Abdullah University of Science and Technology (KAUST, on Shaheen-II) and by the National Energy Research Scientific Computing Center (NERSC, on Cori). We especially thank Nicolay Hammer (LRZ), as well as Richard Gerber and Jack Deslippe (NERSC) for their highly valuable support.
This publication acknowledges KAUST support, but has no KAUST affiliated authors.

Access to Document

10.1145/3126908.3126948

Cite this

@inproceedings{1ede1fdacede4d9b9c03490efb96e473,

title = "Extreme Scale Multi-Physics Simulations of the Tsunamigenic 2004 Sumatra Megathrust Earthquake",

abstract = "We present a high-resolution simulation of the 2004 Sumatra-Andaman earthquake, including non-linear frictional failure on a megathrust-splay fault system. Our method exploits unstructured meshes capturing the complicated geometries in subduction zones that are crucial to understand large earthquakes and tsunami generation. These up-to-date largest and longest dynamic rupture simulations enable analysis of dynamic source effects on the seafloor displacements. To tackle the extreme size of this scenario an end-to-end optimization of the simulation code SeisSol was necessary. We implemented a new cache-aware wave propagation scheme and optimized the dynamic rupture kernels using code generation. We established a novel clustered local-time-stepping scheme for dynamic rupture. In total, we achieved a speed-up of 13.6 compared to the previous implementation. For the Sumatra scenario with 221 million elements this reduced the time-to-solution to 13.9 hours on 86,016 Haswell cores. Furthermore, we used asynchronous output to overlap I/O and compute time.",

author = "Carsten Uphoff and Sebastian Rettenberger and Michael Bader and Madden, {Elizabeth H.} and Thomas Ulrich and Stephanie Wollherr and Alice-Agnes Gabriel",

note = "KAUST Repository Item: Exported on 2022-06-23 Acknowledgements: The work presented in this paper was supported by the Volkswagen Foundation (project ASCETE - Advanced Simulation of Coupled Earthquake-Tsunami Events, grant no. 88479), by the German Research Foundation (DFG) (project no. KA 2281/4-1, AOBJ 584936/TG-92), by the Bavarian Competence Network for Technical and Scientific High Performance Computing (KONWIHR) (project GeoPF - Geophysics for PetaFlop Computing), and by Intel as part of the Intel Parallel Computing Center ExScaMIC-KNL. Computing resources were provided by the Leibniz Supercomputing Centre (LRZ, project no. pr45f and h019z, on SuperMUC), by P. Martin Mai, King Abdullah University of Science and Technology (KAUST, on Shaheen-II) and by the National Energy Research Scientific Computing Center (NERSC, on Cori). We especially thank Nicolay Hammer (LRZ), as well as Richard Gerber and Jack Deslippe (NERSC) for their highly valuable support. This publication acknowledges KAUST support, but has no KAUST affiliated authors.",

year = "2017",

month = nov,

day = "12",

doi = "10.1145/3126908.3126948",

language = "English (US)",

isbn = "9781450351140",

booktitle = "Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis",

publisher = "ACM",

}

Extreme Scale Multi-Physics Simulations of the Tsunamigenic 2004 Sumatra Megathrust Earthquake. / Uphoff, Carsten; Rettenberger, Sebastian; Bader, Michael et al.
Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. ACM, 2017.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Extreme Scale Multi-Physics Simulations of the Tsunamigenic 2004 Sumatra Megathrust Earthquake

AU - Uphoff, Carsten

AU - Rettenberger, Sebastian

AU - Bader, Michael

AU - Madden, Elizabeth H.

AU - Ulrich, Thomas

AU - Wollherr, Stephanie

AU - Gabriel, Alice-Agnes

N1 - KAUST Repository Item: Exported on 2022-06-23 Acknowledgements: The work presented in this paper was supported by the Volkswagen Foundation (project ASCETE - Advanced Simulation of Coupled Earthquake-Tsunami Events, grant no. 88479), by the German Research Foundation (DFG) (project no. KA 2281/4-1, AOBJ 584936/TG-92), by the Bavarian Competence Network for Technical and Scientific High Performance Computing (KONWIHR) (project GeoPF - Geophysics for PetaFlop Computing), and by Intel as part of the Intel Parallel Computing Center ExScaMIC-KNL. Computing resources were provided by the Leibniz Supercomputing Centre (LRZ, project no. pr45f and h019z, on SuperMUC), by P. Martin Mai, King Abdullah University of Science and Technology (KAUST, on Shaheen-II) and by the National Energy Research Scientific Computing Center (NERSC, on Cori). We especially thank Nicolay Hammer (LRZ), as well as Richard Gerber and Jack Deslippe (NERSC) for their highly valuable support. This publication acknowledges KAUST support, but has no KAUST affiliated authors.

PY - 2017/11/12

Y1 - 2017/11/12

N2 - We present a high-resolution simulation of the 2004 Sumatra-Andaman earthquake, including non-linear frictional failure on a megathrust-splay fault system. Our method exploits unstructured meshes capturing the complicated geometries in subduction zones that are crucial to understand large earthquakes and tsunami generation. These up-to-date largest and longest dynamic rupture simulations enable analysis of dynamic source effects on the seafloor displacements. To tackle the extreme size of this scenario an end-to-end optimization of the simulation code SeisSol was necessary. We implemented a new cache-aware wave propagation scheme and optimized the dynamic rupture kernels using code generation. We established a novel clustered local-time-stepping scheme for dynamic rupture. In total, we achieved a speed-up of 13.6 compared to the previous implementation. For the Sumatra scenario with 221 million elements this reduced the time-to-solution to 13.9 hours on 86,016 Haswell cores. Furthermore, we used asynchronous output to overlap I/O and compute time.

AB - We present a high-resolution simulation of the 2004 Sumatra-Andaman earthquake, including non-linear frictional failure on a megathrust-splay fault system. Our method exploits unstructured meshes capturing the complicated geometries in subduction zones that are crucial to understand large earthquakes and tsunami generation. These up-to-date largest and longest dynamic rupture simulations enable analysis of dynamic source effects on the seafloor displacements. To tackle the extreme size of this scenario an end-to-end optimization of the simulation code SeisSol was necessary. We implemented a new cache-aware wave propagation scheme and optimized the dynamic rupture kernels using code generation. We established a novel clustered local-time-stepping scheme for dynamic rupture. In total, we achieved a speed-up of 13.6 compared to the previous implementation. For the Sumatra scenario with 221 million elements this reduced the time-to-solution to 13.9 hours on 86,016 Haswell cores. Furthermore, we used asynchronous output to overlap I/O and compute time.

UR - http://hdl.handle.net/10754/679279

UR - https://dl.acm.org/doi/10.1145/3126908.3126948

UR - http://www.scopus.com/inward/record.url?scp=85040180617&partnerID=8YFLogxK

U2 - 10.1145/3126908.3126948

DO - 10.1145/3126908.3126948

M3 - Conference contribution

SN - 9781450351140

BT - Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

PB - ACM

ER -

Extreme Scale Multi-Physics Simulations of the Tsunamigenic 2004 Sumatra Megathrust Earthquake

Abstract

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this