Abstract
Todays high-end parallel clusters are architecturally very complex. Most large scale applications nowadays are utilizing multiple parallel programming paradigms to achieve the required scalability, with MPI+threads being the most common approach. Yet, as of today, there is no parallel I/O library that matches this hybrid programming model. File I/O operations are typically executed by a single thread for each process. This paper explores multi-Threaded optimizations for individual MPI I/O operations, an important step towards matching the execution model of modern parallel applications. We describe the changes necessary to the internal processing in the MPI I/O library as well as to the file access phase. We demonstrate the performance improvement of the redesigned functions using multiple benchmarks and on multiple platforms for many scenarios over the original, single-Threaded version.
| Original language | English (US) |
|---|---|
| Title of host publication | 2022 30th Euromicro International Conference on Parallel, Distributed and Network-based Processing (PDP) |
| Publisher | IEEE |
| Pages | 122-126 |
| Number of pages | 5 |
| ISBN (Print) | 9781665469586 |
| DOIs | |
| State | Published - Apr 18 2022 |
| Externally published | Yes |
Bibliographical note
KAUST Repository Item: Exported on 2022-06-21Acknowledgements: Partial support for this work was provided by the National Science Foundation under Award No. SI2-SSI 1663887. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. The authors would also like to thank the Supercomputing Laboratory at King Abdullah University of Science & Technology (KAUST) for providing compute time on the Shaheen II supercomputer for this project.
This publication acknowledges KAUST support, but has no KAUST affiliated authors.