Testing-based GPU-Memory Consumption Estimation for Deep Learning

• Item type: Symposium(1)
• 公開日: 2021-08-30
• タイトル: Testing-based GPU-Memory Consumption Estimation for Deep Learning
• タイトル / 言語: en
• タイトル: Testing-based GPU-Memory Consumption Estimation for Deep Learning
• 言語: eng
• キーワード / 主題Scheme: Other
• キーワード / 主題: ソフトウェアテスト
• 資源タイプ / 資源タイプ識別子: http://purl.org/coar/resource_type/c_5794
• 資源タイプ: conference paper
• 著者所属: Graduate School of Advanced Science and Engineering, Hiroshima University
• 著者所属: Graduate School of Advanced Science and Engineering, Hiroshima University
• 著者所属: Graduate School of Advanced Science and Engineering, Hiroshima University
• 著者所属(英): en
• 著者所属(英): Graduate School of Advanced Science and Engineering, Hiroshima University
• 著者所属(英): en
• 著者所属(英): Graduate School of Advanced Science and Engineering, Hiroshima University
• 著者所属(英): en
• 著者所属(英): Graduate School of Advanced Science and Engineering, Hiroshima University
• 著者名: Haiyi, Liu; Shaoying, Liu; Ai, Liu
• 著者名(英): Haiyi, Liu; Shaoying, Liu; Ai, Liu
• 論文抄録 / 内容記述タイプ: Other
• 論文抄録 / 内容記述: Deep learning(DL) has been successfully applied in many software systems and deployed to a variety of server. The training of DL needs a lot of GPU computing resources. However, it is difficult for developers to accurately calculate the GPU resources that the model may consume before running the model, which brings great inconvenience to the development of DL system. Especially nowadays, a lot of model training runs on cloud services. Therefore, it is very important to estimate the GPU-memory resources that any model may use in a certain computing framework. The existing work mainly focuses on the static analysis method to evaluate the GPU-memory consumption, which is highly coupled with the framework implementation, and lacks the research on the software testing and evaluation method of GPU-memory consumption (It does not depend on the framework implementation itself). In this paper, we propose a new method to estimate the memory consumption of DL framework. The method is based on software testing and static analysis estimation. Firstly, heuristic random search algorithm is used to explore the real GPU-memory consumption of different DL models at runtime. At the same time, the software static analysis method is used to evaluate the theoretical memory consumption of DL model, which can reduce the number of model testing (because the testing model requires computing resources). Finally, the known data is modeled to estimate the real memory consumption of different models in different computing frameworks. To evaluate the effectiveness of our proposed method, we apply it to the mainstream computing framework, i.e., TensorFlow, Pytorch. The results show that our method can achieve more accurate evaluation of GPU-memory consumption without knowing the operation mechanism of the framework.
• 論文抄録(英) / 内容記述タイプ: Other
• 論文抄録(英) / 内容記述: Deep learning(DL) has been successfully applied in many software systems and deployed to a variety of server. The training of DL needs a lot of GPU computing resources. However, it is difficult for developers to accurately calculate the GPU resources that the model may consume before running the model, which brings great inconvenience to the development of DL system. Especially nowadays, a lot of model training runs on cloud services. Therefore, it is very important to estimate the GPU-memory resources that any model may use in a certain computing framework. The existing work mainly focuses on the static analysis method to evaluate the GPU-memory consumption, which is highly coupled with the framework implementation, and lacks the research on the software testing and evaluation method of GPU-memory consumption (It does not depend on the framework implementation itself). In this paper, we propose a new method to estimate the memory consumption of DL framework. The method is based on software testing and static analysis estimation. Firstly, heuristic random search algorithm is used to explore the real GPU-memory consumption of different DL models at runtime. At the same time, the software static analysis method is used to evaluate the theoretical memory consumption of DL model, which can reduce the number of model testing (because the testing model requires computing resources). Finally, the known data is modeled to estimate the real memory consumption of different models in different computing frameworks. To evaluate the effectiveness of our proposed method, we apply it to the mainstream computing framework, i.e., TensorFlow, Pytorch. The results show that our method can achieve more accurate evaluation of GPU-memory consumption without knowing the operation mechanism of the framework.
• 書誌情報: ソフトウェアエンジニアリングシンポジウム2021論文集; 巻 2021, p. 196-199, 発行日 2021-08-30
• 出版者 / 言語: ja
• 出版者: 情報処理学会