Continuous Integration for HPC with Github Actions and Tapis

Benjamin Pachev, Georgia Stuart, Clint Dawson

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Continuous integration and deployment (CICD) are fundamental to modern software development. While many platforms such as GitHub and Atlassian provide cloud solutions for CICD, these solutions don’t fully meet the unique needs of high performance computing (HPC) applications. These needs include, but are not limited to, testing distributed memory and scaling studies, both of which require an HPC environment. We propose a novel framework for running CICD workflows on supercomputing resources. Our framework directly integrates with GitHub Actions and leverages TACC’s Tapis API for communication with HPC resources. The framework is demonstrated for PYthon Ocean PArticle TRAcking (PYOPATRA), an HPC application for Lagrangian particle tracking.
Original languageEnglish (US)
Title of host publicationPractice and Experience in Advanced Research Computing
PublisherACM
DOIs
StatePublished - Jul 8 2022
Externally publishedYes

Bibliographical note

KAUST Repository Item: Exported on 2022-09-14
Acknowledgements: This work used the Extreme Science and Engineering Discovery Environment (XSEDE) [18], which is supported by National Science Foundation grant number ACI-1548562. The authors would like to gratefully acknowledge the use of XSEDE allocation TGDMS080016N, which provided access to the Jetstream and Stampede2 supercomputers at the Texas Advanced Computing Center and Indiana University. This project was supported by a Competitive Research Grant (CRG) from King Abdullah University of Science and Technology (KAUST) entitled “Detecting, tracking and forecasting for hazardous spills in the Red Sea” In addition, we thank Dr. Christopher Simmons for many conversations on CI/CD in HPC.
This publication acknowledges KAUST support, but has no KAUST affiliated authors.

Fingerprint

Dive into the research topics of 'Continuous Integration for HPC with Github Actions and Tapis'. Together they form a unique fingerprint.

Cite this