Abstract
Exploiting data sparsity in dense matrices is an algorithmic bridge between architectures that are increasingly memory-austere on a per-core basis and extreme-scale applications. In this work, we leverage the Hierarchical matrix Computations on Manycore Architectures (HiCMA) library in order to tackle this challenging problem by achieving significant reductions in time to solution and memory footprint, while preserving a specified accuracy requirement of the application. We have extended HiCMA to provide a high-performance implementation on distributed-memory systems of one of the most widely used matrix factorization in large-scale scientific applications, i.e., the Cholesky factorization. It employs the tile low-rank data format to compress the dense data-sparse off-diagonal tiles of the matrix. It then decomposes the matrix computations into interdependent tasks and relies on the dynamic runtime system StarPU for asynchronous out-of-order scheduling, while allowing high user productivity. Performance comparisons and memory footprint on matrix dimensions up to eleven million show a performance gain and memory saving of more than an order of magnitude for both metrics on thousands of cores, against state-of-the-art open-source and vendor optimized numerical libraries. This represents an important milestone in enabling large-scale matrix computations toward solving big data problems in geospatial statistics for climate/weather forecasting applications.
Original language | English (US) |
---|---|
Title of host publication | Euro-Par 2018 |
Subtitle of host publication | Parallel Processing - 24th International Conference on Parallel and Distributed Computing, Proceedings |
Editors | Massimo Torquati, Marco Aldinucci, Luca Padovani |
Publisher | Springer Verlag |
Pages | 721-734 |
Number of pages | 14 |
ISBN (Print) | 9783319969824 |
DOIs | |
State | Published - 2018 |
Event | 24th International European Conference on Parallel and Distributed Computing, Euro-Par 2018 - Turin, Italy Duration: Aug 27 2018 → Aug 31 2018 |
Publication series
Name | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
---|---|
Volume | 11014 LNCS |
ISSN (Print) | 0302-9743 |
ISSN (Electronic) | 1611-3349 |
Conference
Conference | 24th International European Conference on Parallel and Distributed Computing, Euro-Par 2018 |
---|---|
Country/Territory | Italy |
City | Turin |
Period | 08/27/18 → 08/31/18 |
Bibliographical note
Publisher Copyright:© 2018, Springer International Publishing AG, part of Springer Nature.
ASJC Scopus subject areas
- Theoretical Computer Science
- General Computer Science