Design and Evaluation of Multi-Threaded Optimizations for Individual MPI I/O Operations

Raafat Feki, Edgar Gabriel

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

Todays high-end parallel clusters are architecturally very complex. Most large scale applications nowadays are utilizing multiple parallel programming paradigms to achieve the required scalability, with MPI+threads being the most common approach. Yet, as of today, there is no parallel I/O library that matches this hybrid programming model. File I/O operations are typically executed by a single thread for each process. This paper explores multi-Threaded optimizations for individual MPI I/O operations, an important step towards matching the execution model of modern parallel applications. We describe the changes necessary to the internal processing in the MPI I/O library as well as to the file access phase. We demonstrate the performance improvement of the redesigned functions using multiple benchmarks and on multiple platforms for many scenarios over the original, single-Threaded version.
Original languageEnglish (US)
Title of host publication2022 30th Euromicro International Conference on Parallel, Distributed and Network-based Processing (PDP)
PublisherIEEE
Pages122-126
Number of pages5
ISBN (Print)9781665469586
DOIs
StatePublished - Apr 18 2022
Externally publishedYes

Bibliographical note

KAUST Repository Item: Exported on 2022-06-21
Acknowledgements: Partial support for this work was provided by the National Science Foundation under Award No. SI2-SSI 1663887. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. The authors would also like to thank the Supercomputing Laboratory at King Abdullah University of Science & Technology (KAUST) for providing compute time on the Shaheen II supercomputer for this project.
This publication acknowledges KAUST support, but has no KAUST affiliated authors.

Fingerprint

Dive into the research topics of 'Design and Evaluation of Multi-Threaded Optimizations for Individual MPI I/O Operations'. Together they form a unique fingerprint.

Cite this