Abstract
Large-scale graph processing, with its massive data sets, requires distributed processing. However, conventional frameworks for distributed graph processing, such as Pregel, use non-traditional programming models that are well-suited for parallelism and scalability but inconvenient for implementing non-trivial graph algorithms. In this paper, we use Green-Marl, a Domain-Specific Language for graph analysis, to intuitively describe graph algorithms and extend its compiler to generate equivalent Pregel implementations. Using the semantic information captured by Green-Marl, the compiler applies a set of transformation rules that convert imperative graph algorithms into Pregel's programming model. Our experiments show that the Pregel programs generated by the Green-Marl compiler perform similarly to manually coded Pregel implementations of the same algorithms. The compiler is even able to generate a Pregel implementation of a complicated graph algorithm for which a manual Pregel implementation is very challenging.
| Original language | English (US) |
|---|---|
| Title of host publication | Proceedings of Annual IEEE/ACM International Symposium on Code Generation and Optimization - CGO '14 |
| Publisher | Association for Computing Machinery (ACM) |
| ISBN (Print) | 9781450326704 |
| DOIs | |
| State | Published - 2014 |
| Externally published | Yes |
Bibliographical note
KAUST Repository Item: Exported on 2020-10-01Acknowledgements: This work was funded by DARPA Contract, Xgraphs; Languageand Algorithms for Heterogeneous Graph Streams, FA8750-12-2-0335; Army contract AHPCRC W911NF-07-2-0027-1; the Na-tional Science Foundation (IIS-0904497) and a KAUST researchgrant; Stanford PPL affiliates program, Pervasive Parallelism Lab:Oracle, AMD, Intel, NVIDIA, and Huawei. Authors also acknowl-edge additional support from Oracle.
This publication acknowledges KAUST support, but has no KAUST affiliated authors.