Graph Neural Networks (GNNs) have been drawing significant attention to representation learning on graphs. Recent works developed frameworks to train very deep GNNs and showed impressive results in tasks like point cloud learning and protein interaction prediction. In this work, we study the performance of such deep models in large-scale graphs. In particular, we look at the effect of adequately choosing an aggregation function on deep models. We find that GNNs are very sensitive to the choice of aggregation functions (e.g. mean, max, and sum) when applied to different datasets. We systematically study and propose to alleviate this issue by introducing a novel class of aggregation functions named Generalized Aggregation Functions. The proposed functions extend beyond commonly used aggregation functions to a wide range of new permutation-invariant functions. Generalized Aggregation Functions are fully differentiable, where their parameters can be learned in an end-to-end fashion to yield a suitable aggregation function for each task. We show that equipped with the proposed aggregation functions, deep residual GNNs outperform state-of-the-art in several benchmarks from Open Graph Benchmark (OGB) across tasks and domains.
|Original language||English (US)|
|Number of pages||12|
|Journal||IEEE Transactions on Pattern Analysis and Machine Intelligence|
|State||Published - Aug 21 2023|
Bibliographical noteKAUST Repository Item: Exported on 2023-08-31
Acknowledgements: This work was supported by the King Abdullah University of Science and Technology (KAUST) Office of Sponsored Research through the Visual Computing Center (VCC) funding.
ASJC Scopus subject areas
- Artificial Intelligence
- Computational Theory and Mathematics
- Applied Mathematics
- Computer Vision and Pattern Recognition