High-performance Graph Convolutional Networks Training on Fugaku and ABCI Supercomputers

• Item type: SIG Technical Reports(1)
• 公開日: 2024-08-01
• タイトル: High-performance Graph Convolutional Networks Training on Fugaku and ABCI Supercomputers
• タイトル / 言語: en
• タイトル: High-performance Graph Convolutional Networks Training on Fugaku and ABCI Supercomputers
• 言語: eng
• キーワード / 主題Scheme: Other
• キーワード / 主題: 深層学習
• 資源タイプ / 資源タイプ識別子: http://purl.org/coar/resource_type/c_18gh
• 資源タイプ: technical report
• 著者所属: Tokyo Institute of Technology／RIKEN Center for Computational Science
• 著者所属: National Institute of Advanced Industrial Science and Technology (AIST)／RIKEN Center for Computational Science
• 著者所属: National Institute of Advanced Industrial Science and Technology (AIST)
• 著者所属: Tokyo Institute of Technology
• 著者所属: Tokyo Institute of Technology
• 著者所属: Tokyo Institute of Technology／RIKEN Center for Computational Science
• 著者所属: RIKEN Center for Computational Science
• 著者所属(英): en
• 著者所属(英): Tokyo Institute of Technology / RIKEN Center for Computational Science
• 著者所属(英): en
• 著者所属(英): National Institute of Advanced Industrial Science and Technology (AIST) / RIKEN Center for Computational Science
• 著者所属(英): en
• 著者所属(英): National Institute of Advanced Industrial Science and Technology (AIST)
• 著者所属(英): en
• 著者所属(英): Tokyo Institute of Technology
• 著者所属(英): en
• 著者所属(英): Tokyo Institute of Technology
• 著者所属(英): en
• 著者所属(英): Tokyo Institute of Technology / RIKEN Center for Computational Science
• 著者所属(英): en
• 著者所属(英): RIKEN Center for Computational Science
• 著者名: Chen, Zhuang; Peng, Chen; Xin, Liu; Rio, Yokota; Toshio, Endo; Satoshi, Matsuoka; Mohamed, Wahib
• 著者名(英): Chen, Zhuang; Peng, Chen; Xin, Liu; Rio, Yokota; Toshio, Endo; Satoshi, Matsuoka; Mohamed, Wahib
• 論文抄録 / 内容記述タイプ: Other
• 論文抄録 / 内容記述: Graph Convolutional Networks (GCNs) are widely used in various domains. However, training distributed full-batch GCNs on large-scale graphs poses challenges due to high communication overhead. This paper presents a hybrid pre-post-aggregation approach to reduce communication volume. Additionally, we employ an integer quantization method to compress the communication data, thus reducing communication costs further. Combining these techniques, we develop an efficient and scalable distributed GCN training framework, SuperGNN, for CPU-powered supercomputers, Fugaku and ABCI. Experimental results on multiple large graph datasets show that our method achieves a speedup of up to 6× compared with the state-of-the-art implementations, and scales to 1000s of HPC-grade CPUs, without sacrificing model convergence and accuracy. Our framework achieves performance on CPU-powered supercomputers comparable to that of GPU-powered supercomputers, with a fraction of the cost and power budget.
• 論文抄録(英) / 内容記述タイプ: Other
• 論文抄録(英) / 内容記述: Graph Convolutional Networks (GCNs) are widely used in various domains. However, training distributed full-batch GCNs on large-scale graphs poses challenges due to high communication overhead. This paper presents a hybrid pre-post-aggregation approach to reduce communication volume. Additionally, we employ an integer quantization method to compress the communication data, thus reducing communication costs further. Combining these techniques, we develop an efficient and scalable distributed GCN training framework, SuperGNN, for CPU-powered supercomputers, Fugaku and ABCI. Experimental results on multiple large graph datasets show that our method achieves a speedup of up to 6× compared with the state-of-the-art implementations, and scales to 1000s of HPC-grade CPUs, without sacrificing model convergence and accuracy. Our framework achieves performance on CPU-powered supercomputers comparable to that of GPU-powered supercomputers, with a fraction of the cost and power budget.
• 書誌レコードID / 収録物識別子タイプ: NCID
• 書誌レコードID / 収録物識別子: AN10463942
• 書誌情報: 研究報告ハイパフォーマンスコンピューティング（HPC）; 巻 2024-HPC-195, 号 14, p. 1-8, 発行日 2024-08-01
• ISSN / 収録物識別子タイプ: ISSN
• ISSN / 収録物識別子: 2188-8841
• Notice: SIG Technical Reports are nonrefereed and hence may later appear in any journals, conferences, symposia, etc.
• 出版者 / 言語: ja
• 出版者: 情報処理学会