Modeling Pooling Layers For CNN Initialization

• Item type: Trans(1)
• 公開日: 2022-07-26
• タイトル: Modeling Pooling Layers For CNN Initialization
• タイトル / 言語: en
• タイトル: Modeling Pooling Layers For CNN Initialization
• 言語: eng
• キーワード / 主題Scheme: Other
• キーワード / 主題: [オリジナル論文] deep neural network, convolutional neural network, initialization, signal variance
• 資源タイプ / 資源タイプ識別子: http://purl.org/coar/resource_type/c_6501
• 資源タイプ: journal article
• 著者所属: Graduate School of Science and Technology, Gunma University
• 著者所属: Graduate School of Science and Technology, Gunma University
• 著者所属(英): en
• 著者所属(英): Graduate School of Science and Technology, Gunma University
• 著者所属(英): en
• 著者所属(英): Graduate School of Science and Technology, Gunma University
• 著者名: Takahiko, Henmi; Tsuyoshi, Kato
• 著者名(英): Takahiko, Henmi; Tsuyoshi, Kato
• 論文抄録 / 内容記述タイプ: Other
• 論文抄録 / 内容記述: Deep convolutional neural networks (CNNs) have achieved consistent excellent performance on image processing tasks. The CNN architecture consists of different types of layers, including the convolution layer and the max pooling layer. It is widely understood among CNN practitioners that the stability of learning depends on the initialization of the model parameters in each layer. Currently, the de facto standard scheme for initialization is the Kaiming initialization, which was developed by He et al. The Kaiming scheme was derived from a much simpler model than the currently used CNN structure that evolved since the emergence of the Kaiming scheme. It consists only of the convolution and fully connected layers, and does not include the max pooling or global average pooling layers. In this study, we derive an initialization scheme, not from the simplified Kaiming model, but from modern CNN architectures consisting not only of the convolution and the fully connected layers but also of the max pooling and the global average pooling. Furthermore, the new model expresses the padding which is not considered in the existing models. We empirically investigate the performance of the new initialization methods compared to the de facto standard methods that are widely used today. Source code: https://github.com/hecwitane/ASV-pub/
• 論文抄録(英) / 内容記述タイプ: Other
• 論文抄録(英) / 内容記述: Deep convolutional neural networks (CNNs) have achieved consistent excellent performance on image processing tasks. The CNN architecture consists of different types of layers, including the convolution layer and the max pooling layer. It is widely understood among CNN practitioners that the stability of learning depends on the initialization of the model parameters in each layer. Currently, the de facto standard scheme for initialization is the Kaiming initialization, which was developed by He et al. The Kaiming scheme was derived from a much simpler model than the currently used CNN structure that evolved since the emergence of the Kaiming scheme. It consists only of the convolution and fully connected layers, and does not include the max pooling or global average pooling layers. In this study, we derive an initialization scheme, not from the simplified Kaiming model, but from modern CNN architectures consisting not only of the convolution and the fully connected layers but also of the max pooling and the global average pooling. Furthermore, the new model expresses the padding which is not considered in the existing models. We empirically investigate the performance of the new initialization methods compared to the de facto standard methods that are widely used today. Source code: https://github.com/hecwitane/ASV-pub/
• 書誌レコードID / 収録物識別子タイプ: NCID
• 書誌レコードID / 収録物識別子: AA11464803
• 書誌情報: 情報処理学会論文誌数理モデル化と応用（TOM）; 巻 15, 号 3, p. 29-37, 発行日 2022-07-26
• ISSN / 収録物識別子タイプ: ISSN
• ISSN / 収録物識別子: 1882-7780
• 出版者 / 言語: ja
• 出版者: 情報処理学会