A Study of Deep Reinforcement Learning Based Resource Allocation Strategy in Multi-access Edge Computing Environment

• Item type: SIG Technical Reports(1)
• 公開日: 2024-06-03
• タイトル: A Study of Deep Reinforcement Learning Based Resource Allocation Strategy in Multi-access Edge Computing Environment
• タイトル / 言語: en
• タイトル: A Study of Deep Reinforcement Learning Based Resource Allocation Strategy in Multi-access Edge Computing Environment
• 言語: eng
• キーワード / 主題Scheme: Other
• キーワード / 主題: 低消費電力
• 資源タイプ / 資源タイプ識別子: http://purl.org/coar/resource_type/c_18gh
• 資源タイプ: technical report
• 著者所属: The University of Tokyo
• 著者所属: The University of Tokyo
• 著者所属: The University of Tokyo
• 著者所属(英): en
• 著者所属(英): The University of Tokyo
• 著者所属(英): en
• 著者所属(英): The University of Tokyo
• 著者所属(英): en
• 著者所属(英): The University of Tokyo
• 著者名: Huilin, Li; Hiroshi, Nakamura; Hideki, Takase
• 著者名(英): Huilin, Li; Hiroshi, Nakamura; Hideki, Takase
• 論文抄録 / 内容記述タイプ: Other
• 論文抄録 / 内容記述: In order to help user terminal devices (UTDs) efficiently handle computation-intensive and delay-sensitive computing tasks, the use of multi-access edge computing (MEC) has been proposed. However, due to shortcomings in the performance of UTDs and limited resources in networks, the optimization problem of the energy consumption of UTDs under the constraint of a certain delay condition is still a focus of research. This paper studies the optimization strategy of resource allocation for the federated learning system in the MEC network in multi-user and single-server scenarios. Firstly, we formulate the optimization problem as a constrained nonlinear programming (CNLP) problem to minimize the sum of time delay and energy consumption of UTDs under a certain delay constraint, by adjusting the local computing frequency, channel selection and power allocation dynamically. Secondly, we propose a reinforcement learning based algorithm, which uses an actor network to predict the next action based on the current state and a critic network to evaluate the quality of the predicted value. Both actor and critic networks use the gradient descent method to reduce the loss and update their network parameters. Within a limited number of training episodes, we can obtain a stable convergence solution to the original optimization problem. Finally, the performance of the proposed algorithm is compared with other classical methods.
• 論文抄録(英) / 内容記述タイプ: Other
• 論文抄録(英) / 内容記述: In order to help user terminal devices (UTDs) efficiently handle computation-intensive and delay-sensitive computing tasks, the use of multi-access edge computing (MEC) has been proposed. However, due to shortcomings in the performance of UTDs and limited resources in networks, the optimization problem of the energy consumption of UTDs under the constraint of a certain delay condition is still a focus of research. This paper studies the optimization strategy of resource allocation for the federated learning system in the MEC network in multi-user and single-server scenarios. Firstly, we formulate the optimization problem as a constrained nonlinear programming (CNLP) problem to minimize the sum of time delay and energy consumption of UTDs under a certain delay constraint, by adjusting the local computing frequency, channel selection and power allocation dynamically. Secondly, we propose a reinforcement learning based algorithm, which uses an actor network to predict the next action based on the current state and a critic network to evaluate the quality of the predicted value. Both actor and critic networks use the gradient descent method to reduce the loss and update their network parameters. Within a limited number of training episodes, we can obtain a stable convergence solution to the original optimization problem. Finally, the performance of the proposed algorithm is compared with other classical methods.
• 書誌レコードID / 収録物識別子タイプ: NCID
• 書誌レコードID / 収録物識別子: AN10096105
• 書誌情報: 研究報告システム・アーキテクチャ（ARC）; 巻 2024-ARC-257, 号 3, p. 1-8, 発行日 2024-06-03
• ISSN / 収録物識別子タイプ: ISSN
• ISSN / 収録物識別子: 2188-8574
• Notice: SIG Technical Reports are nonrefereed and hence may later appear in any journals, conferences, symposia, etc.
• 出版者 / 言語: ja
• 出版者: 情報処理学会