Exploring Approaches to Improve Accuracy in SplitFed Learning on Non-IID Data

• Item type: Symposium(1)
• 公開日: 2024-12-27
• タイトル: Exploring Approaches to Improve Accuracy in SplitFed Learning on Non-IID Data
• タイトル / 言語: en
• タイトル: Exploring Approaches to Improve Accuracy in SplitFed Learning on Non-IID Data
• 言語: eng
• 資源タイプ / 資源タイプ識別子: http://purl.org/coar/resource_type/c_5794
• 資源タイプ: conference paper
• 著者所属: Saitama University
• 著者所属: Saitama University
• 著者所属(英): en
• 著者所属(英): Saitama University
• 著者所属(英): en
• 著者所属(英): Saitama University
• 著者名: Hinata, Tomimori; Takuya, Azumi
• 著者名(英): Hinata, Tomimori; Takuya, Azumi
• 論文抄録 / 内容記述タイプ: Other
• 論文抄録 / 内容記述: Distributed Machine Learning (DML) approaches, particularly Split Learning (SL) and Federated Learning (FL), have gained attention for enabling collaborative model training while protecting data privacy. SplitFed Learning (SFL) has emerged as a hybrid of SL and FL, offering low computationally intensive for the client and high parallelism. However, the performance of SFL can degrade significantly when dealing with Non-Independent and Identically Distributed (Non-IID) data, a common scenario in real-world applications. This paper explores the impact of Non-IID data on SFL through evaluations with various data distributions, and proposes approaches to mitigate accuracy degradation. Specifically, the effectiveness of existing techniques, such as Prototypical Contrastive Learning (PCL), Model Contrastive Learning (MOON), and Mutual Knowledge Distillation (MKD), when applied to SFL is evaluated. The findings demonstrate that while these approaches can enhance accuracy on Non-IID data, challenges are also introduced, particularly in terms of computational complexity and client drift. Based on these results, the discussion focuses on the direction of SFL approaches for Non-IID data, aiming to make SFL a more feasible solution in large-scale, resource-constrained environments such as Internet of Things (IoT) networks.
• 論文抄録(英) / 内容記述タイプ: Other
• 論文抄録(英) / 内容記述: Distributed Machine Learning (DML) approaches, particularly Split Learning (SL) and Federated Learning (FL), have gained attention for enabling collaborative model training while protecting data privacy. SplitFed Learning (SFL) has emerged as a hybrid of SL and FL, offering low computationally intensive for the client and high parallelism. However, the performance of SFL can degrade significantly when dealing with Non-Independent and Identically Distributed (Non-IID) data, a common scenario in real-world applications. This paper explores the impact of Non-IID data on SFL through evaluations with various data distributions, and proposes approaches to mitigate accuracy degradation. Specifically, the effectiveness of existing techniques, such as Prototypical Contrastive Learning (PCL), Model Contrastive Learning (MOON), and Mutual Knowledge Distillation (MKD), when applied to SFL is evaluated. The findings demonstrate that while these approaches can enhance accuracy on Non-IID data, challenges are also introduced, particularly in terms of computational complexity and client drift. Based on these results, the discussion focuses on the direction of SFL approaches for Non-IID data, aiming to make SFL a more feasible solution in large-scale, resource-constrained environments such as Internet of Things (IoT) networks.
• 書誌情報: Proceedings of Asia Pacific Conference on Robot IoT System Development and Platform; 巻 2024, p. 7-14, 発行日 2024-12-27
• 出版者 / 言語: ja
• 出版者: 情報処理学会