Design and Evaluation of Multi-Threaded Optimizations for Individual MPI I/O Operations

Raafat Feki, Edgar Gabriel

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Todays high-end parallel clusters are architecturally very complex. Most large scale applications nowadays are utilizing multiple parallel programming paradigms to achieve the required scalability, with MPI+threads being the most common approach. Yet, as of today, there is no parallel I/O library that matches this hybrid programming model. File I/O operations are typically executed by a single thread for each process. This paper explores multi-Threaded optimizations for individual MPI I/O operations, an important step towards matching the execution model of modern parallel applications. We describe the changes necessary to the internal processing in the MPI I/O library as well as to the file access phase. We demonstrate the performance improvement of the redesigned functions using multiple benchmarks and on multiple platforms for many scenarios over the original, single-Threaded version.
Original languageEnglish (US)
Title of host publication2022 30th Euromicro International Conference on Parallel, Distributed and Network-based Processing (PDP)
PublisherIEEE
Pages122-126
Number of pages5
ISBN (Print)9781665469586
DOIs
StatePublished - Apr 18 2022
Externally publishedYes

Fingerprint

Dive into the research topics of 'Design and Evaluation of Multi-Threaded Optimizations for Individual MPI I/O Operations'. Together they form a unique fingerprint.

Cite this