Simplifying Scalable Graph Processing with a Domain-Specific Language

Sungpack Hong, Semih Salihoglu, Jennifer Widom, Kunle Olukotun

Research output: Chapter in Book/Report/Conference proceedingConference contribution

38 Scopus citations


Large-scale graph processing, with its massive data sets, requires distributed processing. However, conventional frameworks for distributed graph processing, such as Pregel, use non-traditional programming models that are well-suited for parallelism and scalability but inconvenient for implementing non-trivial graph algorithms. In this paper, we use Green-Marl, a Domain-Specific Language for graph analysis, to intuitively describe graph algorithms and extend its compiler to generate equivalent Pregel implementations. Using the semantic information captured by Green-Marl, the compiler applies a set of transformation rules that convert imperative graph algorithms into Pregel's programming model. Our experiments show that the Pregel programs generated by the Green-Marl compiler perform similarly to manually coded Pregel implementations of the same algorithms. The compiler is even able to generate a Pregel implementation of a complicated graph algorithm for which a manual Pregel implementation is very challenging.
Original languageEnglish (US)
Title of host publicationProceedings of Annual IEEE/ACM International Symposium on Code Generation and Optimization - CGO '14
PublisherAssociation for Computing Machinery (ACM)
ISBN (Print)9781450326704
StatePublished - 2014
Externally publishedYes

Bibliographical note

KAUST Repository Item: Exported on 2020-10-01
Acknowledgements: This work was funded by DARPA Contract, Xgraphs; Languageand Algorithms for Heterogeneous Graph Streams, FA8750-12-2-0335; Army contract AHPCRC W911NF-07-2-0027-1; the Na-tional Science Foundation (IIS-0904497) and a KAUST researchgrant; Stanford PPL affiliates program, Pervasive Parallelism Lab:Oracle, AMD, Intel, NVIDIA, and Huawei. Authors also acknowl-edge additional support from Oracle.
This publication acknowledges KAUST support, but has no KAUST affiliated authors.


Dive into the research topics of 'Simplifying Scalable Graph Processing with a Domain-Specific Language'. Together they form a unique fingerprint.

Cite this