Abstract
Continuous integration and deployment (CICD) are fundamental to modern software development. While many platforms such as GitHub and Atlassian provide cloud solutions for CICD, these solutions don’t fully meet the unique needs of high performance computing (HPC) applications. These needs include, but are not limited to, testing distributed memory and scaling studies, both of which require an HPC environment. We propose a novel framework for running CICD workflows on supercomputing resources. Our framework directly integrates with GitHub Actions and leverages TACC’s Tapis API for communication with HPC resources. The framework is demonstrated for PYthon Ocean PArticle TRAcking (PYOPATRA), an HPC application for Lagrangian particle tracking.
| Original language | English (US) |
|---|---|
| Title of host publication | Practice and Experience in Advanced Research Computing |
| Publisher | ACM |
| DOIs | |
| State | Published - Jul 8 2022 |
| Externally published | Yes |
Bibliographical note
KAUST Repository Item: Exported on 2022-09-14Acknowledgements: This work used the Extreme Science and Engineering Discovery Environment (XSEDE) [18], which is supported by National Science Foundation grant number ACI-1548562. The authors would like to gratefully acknowledge the use of XSEDE allocation TGDMS080016N, which provided access to the Jetstream and Stampede2 supercomputers at the Texas Advanced Computing Center and Indiana University. This project was supported by a Competitive Research Grant (CRG) from King Abdullah University of Science and Technology (KAUST) entitled “Detecting, tracking and forecasting for hazardous spills in the Red Sea” In addition, we thank Dr. Christopher Simmons for many conversations on CI/CD in HPC.
This publication acknowledges KAUST support, but has no KAUST affiliated authors.