Developing Value Networks for Game 2048 with Reinforcement Learning

• Item type: Journal(1)
• 公開日: 2021-04-15
• タイトル: Developing Value Networks for Game 2048 with Reinforcement Learning
• タイトル / 言語: en
• タイトル: Developing Value Networks for Game 2048 with Reinforcement Learning
• 言語: eng
• キーワード / 主題Scheme: Other
• キーワード / 主題: [一般論文] game 2048, neural network, reinforcement learning, stochastic game, single-player game
• 資源タイプ / 資源タイプ識別子: http://purl.org/coar/resource_type/c_6501
• 資源タイプ: journal article
• 著者所属: School of Information, Kochi University of Technology
• 著者所属(英): en
• 著者所属(英): School of Information, Kochi University of Technology
• 著者名: Kiminori, Matsuzaki
• 著者名(英): Kiminori, Matsuzaki
• 論文抄録 / 内容記述タイプ: Other
• 論文抄録 / 内容記述: The game 2048 is a stochastic single-player game and several computer players have been developed in not only research work but also student projects. Among them, the most successful approach is based on N-tuple networks trained by reinforcement learning methods. Though there have been several works on computer players with deep neural networks, their performance were not as good in most cases. In our previous work, we designed policy networks and applied supervised learning, which resulted in an average score of 215,802. In this study, we tackle the problem with value networks and reinforcement learning methods, since value networks are important to combine with game-tree search methods. We investigate the training methods in several aspects, including batches of training, use of symmetry, network structures, and use of game-specific tricks. We then conduct a training for 240 hours with the best configuration. With the best value network obtained, we achieved an average score of 228,100 with the greedy (1-ply search) play, and furthermore an average score of 406,927 by combining it with the 3-ply expectimax search.; ------------------------------; This is a preprint of an article intended for publication Journal of; Information Processing(JIP). This preprint should not be cited. This; article should be cited as: Journal of Information Processing Vol.29(2021) (online); DOI http://dx.doi.org/10.2197/ipsjjip.29.336; ------------------------------
• 論文抄録(英) / 内容記述タイプ: Other
• 論文抄録(英) / 内容記述: The game 2048 is a stochastic single-player game and several computer players have been developed in not only research work but also student projects. Among them, the most successful approach is based on N-tuple networks trained by reinforcement learning methods. Though there have been several works on computer players with deep neural networks, their performance were not as good in most cases. In our previous work, we designed policy networks and applied supervised learning, which resulted in an average score of 215,802. In this study, we tackle the problem with value networks and reinforcement learning methods, since value networks are important to combine with game-tree search methods. We investigate the training methods in several aspects, including batches of training, use of symmetry, network structures, and use of game-specific tricks. We then conduct a training for 240 hours with the best configuration. With the best value network obtained, we achieved an average score of 228,100 with the greedy (1-ply search) play, and furthermore an average score of 406,927 by combining it with the 3-ply expectimax search.; ------------------------------; This is a preprint of an article intended for publication Journal of; Information Processing(JIP). This preprint should not be cited. This; article should be cited as: Journal of Information Processing Vol.29(2021) (online); DOI http://dx.doi.org/10.2197/ipsjjip.29.336; ------------------------------
• 書誌レコードID / 収録物識別子タイプ: NCID
• 書誌レコードID / 収録物識別子: AN00116647
• 書誌情報: 情報処理学会論文誌; 巻 62, 号 4, 発行日 2021-04-15
• ISSN / 収録物識別子タイプ: ISSN
• ISSN / 収録物識別子: 1882-7764