Automatic tuning of bag-of-tasks applications

Majed Sahli; Essam Mansour; Tariq Lutfallah Mohammed Alturkestani; Panos Kalnis

doi:10.1109/ICDE.2015.7113338

Automatic tuning of bag-of-tasks applications

Majed Sahli, Essam Mansour, Tariq Lutfallah Mohammed Alturkestani, Panos Kalnis

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

7 Scopus citations

Abstract

This paper presents APlug, a framework for automatic tuning of large scale applications of many independent tasks. APlug suggests the best decomposition of the original computation into smaller tasks and the best number of CPUs to use, in order to meet user-specific constraints. We show that the problem is not trivial because there is large variability in the execution time of tasks, and it is possible for a task to occupy a CPU by performing useless computations. APlug collects a sample of task execution times and builds a model, which is then used by a discrete event simulator to calculate the optimal parameters. We provide a C++ API and a stand-alone implementation of APlug, and we integrate it with three typical applications from computational chemistry, bioinformatics, and data mining. A scenario for optimizing resources utilization is used to demonstrate our framework. We run experiments on 16,384 CPUs on a supercomputer, 480 cores on a Linux cluster and 80 cores on Amazon EC2, and show that APlug is very accurate with minimal overhead.

Original language	English (US)
Title of host publication	2015 IEEE 31st International Conference on Data Engineering
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
Pages	843-854
Number of pages	12
ISBN (Print)	9781479979646
DOIs	https://doi.org/10.1109/ICDE.2015.7113338
State	Published - Apr 2015

Bibliographical note

KAUST Repository Item: Exported on 2021-09-14
Acknowledgements: For computer time, this research used the resources of the Supercomputing Laboratory at King Abdullah University of
Science & Technology (KAUST) in Thuwal, Saudi Arabia

Access to Document

10.1109/ICDE.2015.7113338

Cite this

@inproceedings{a52372c4f6494c278bf454c835e8b792,

title = "Automatic tuning of bag-of-tasks applications",

abstract = "This paper presents APlug, a framework for automatic tuning of large scale applications of many independent tasks. APlug suggests the best decomposition of the original computation into smaller tasks and the best number of CPUs to use, in order to meet user-specific constraints. We show that the problem is not trivial because there is large variability in the execution time of tasks, and it is possible for a task to occupy a CPU by performing useless computations. APlug collects a sample of task execution times and builds a model, which is then used by a discrete event simulator to calculate the optimal parameters. We provide a C++ API and a stand-alone implementation of APlug, and we integrate it with three typical applications from computational chemistry, bioinformatics, and data mining. A scenario for optimizing resources utilization is used to demonstrate our framework. We run experiments on 16,384 CPUs on a supercomputer, 480 cores on a Linux cluster and 80 cores on Amazon EC2, and show that APlug is very accurate with minimal overhead.",

author = "Majed Sahli and Essam Mansour and Alturkestani, {Tariq Lutfallah Mohammed} and Panos Kalnis",

note = "KAUST Repository Item: Exported on 2021-09-14 Acknowledgements: For computer time, this research used the resources of the Supercomputing Laboratory at King Abdullah University of Science & Technology (KAUST) in Thuwal, Saudi Arabia",

year = "2015",

month = apr,

doi = "10.1109/ICDE.2015.7113338",

language = "English (US)",

isbn = "9781479979646",

pages = "843--854",

booktitle = "2015 IEEE 31st International Conference on Data Engineering",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

}

TY - GEN

T1 - Automatic tuning of bag-of-tasks applications

AU - Sahli, Majed

AU - Mansour, Essam

AU - Alturkestani, Tariq Lutfallah Mohammed

AU - Kalnis, Panos

N1 - KAUST Repository Item: Exported on 2021-09-14 Acknowledgements: For computer time, this research used the resources of the Supercomputing Laboratory at King Abdullah University of Science & Technology (KAUST) in Thuwal, Saudi Arabia

PY - 2015/4

Y1 - 2015/4

N2 - This paper presents APlug, a framework for automatic tuning of large scale applications of many independent tasks. APlug suggests the best decomposition of the original computation into smaller tasks and the best number of CPUs to use, in order to meet user-specific constraints. We show that the problem is not trivial because there is large variability in the execution time of tasks, and it is possible for a task to occupy a CPU by performing useless computations. APlug collects a sample of task execution times and builds a model, which is then used by a discrete event simulator to calculate the optimal parameters. We provide a C++ API and a stand-alone implementation of APlug, and we integrate it with three typical applications from computational chemistry, bioinformatics, and data mining. A scenario for optimizing resources utilization is used to demonstrate our framework. We run experiments on 16,384 CPUs on a supercomputer, 480 cores on a Linux cluster and 80 cores on Amazon EC2, and show that APlug is very accurate with minimal overhead.

AB - This paper presents APlug, a framework for automatic tuning of large scale applications of many independent tasks. APlug suggests the best decomposition of the original computation into smaller tasks and the best number of CPUs to use, in order to meet user-specific constraints. We show that the problem is not trivial because there is large variability in the execution time of tasks, and it is possible for a task to occupy a CPU by performing useless computations. APlug collects a sample of task execution times and builds a model, which is then used by a discrete event simulator to calculate the optimal parameters. We provide a C++ API and a stand-alone implementation of APlug, and we integrate it with three typical applications from computational chemistry, bioinformatics, and data mining. A scenario for optimizing resources utilization is used to demonstrate our framework. We run experiments on 16,384 CPUs on a supercomputer, 480 cores on a Linux cluster and 80 cores on Amazon EC2, and show that APlug is very accurate with minimal overhead.

UR - http://hdl.handle.net/10754/656029

UR - https://ieeexplore.ieee.org/document/7113338/

UR - http://www.scopus.com/inward/record.url?scp=84940868788&partnerID=8YFLogxK

U2 - 10.1109/ICDE.2015.7113338

DO - 10.1109/ICDE.2015.7113338

M3 - Conference contribution

AN - SCOPUS:84940868788

SN - 9781479979646

SP - 843

EP - 854

BT - 2015 IEEE 31st International Conference on Data Engineering

PB - Institute of Electrical and Electronics Engineers (IEEE)

ER -

Automatic tuning of bag-of-tasks applications

Abstract

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this