Distilling Knowledge from a Multi-scale Deep CNN Ensemble for Robust and Light-weight Acoustic Modeling

• Item type: SIG Technical Reports(1)
• 公開日: 2018-02-13
• タイトル: Distilling Knowledge from a Multi-scale Deep CNN Ensemble for Robust and Light-weight Acoustic Modeling
• タイトル / 言語: en
• タイトル: Distilling Knowledge from a Multi-scale Deep CNN Ensemble for Robust and Light-weight Acoustic Modeling
• 言語: eng
• 資源タイプ / 資源タイプ識別子: http://purl.org/coar/resource_type/c_18gh
• 資源タイプ: technical report
• 著者所属: Nara Institute of Science and Technology
• 著者所属: IBM Research AI
• 著者所属: IBM Research AI
• 著者所属: IBM Research AI
• 著者所属: Nara Institute of Science and Technology
• 著者所属(英): en
• 著者所属(英): Nara Institute of Science and Technology
• 著者所属(英): en
• 著者所属(英): IBM Research AI
• 著者所属(英): en
• 著者所属(英): IBM Research AI
• 著者所属(英): en
• 著者所属(英): IBM Research AI
• 著者所属(英): en
• 著者所属(英): Nara Institute of Science and Technology
• 著者名: Michael, Heck; Masayuki, Suzuki; Takashi, Fukuda; Gakuto, Kurata; Satoshi, Nakamura
• 著者名(英): Michael, Heck; Masayuki, Suzuki; Takashi, Fukuda; Gakuto, Kurata; Satoshi, Nakamura
• 論文抄録 / 内容記述タイプ: Other
• 論文抄録 / 内容記述: This paper presents our work on constructing a multi-scale deep convolutional neural network (CNN) ensemble for robust and light-weight acoustic modeling. Several VGG nets are used that differ solely in the kernel size of the convolutional layers. The ensemble serves as teacher for distilling knowledge into a much simpler student CNN. We compare the performance of the distilled CNN model with the results of system combination. We show that the knowledge distillation from a multi-scale ensemble yields equal performance with the best conventional combination methods, with a much simpler system architecture and decoding pipeline.
• 論文抄録(英) / 内容記述タイプ: Other
• 論文抄録(英) / 内容記述: This paper presents our work on constructing a multi-scale deep convolutional neural network (CNN) ensemble for robust and light-weight acoustic modeling. Several VGG nets are used that differ solely in the kernel size of the convolutional layers. The ensemble serves as teacher for distilling knowledge into a much simpler student CNN. We compare the performance of the distilled CNN model with the results of system combination. We show that the knowledge distillation from a multi-scale ensemble yields equal performance with the best conventional combination methods, with a much simpler system architecture and decoding pipeline.
• 書誌レコードID / 収録物識別子タイプ: NCID
• 書誌レコードID / 収録物識別子: AN10438388
• 書誌情報: 研究報告音楽情報科学（MUS）; 巻 2018-MUS-118, 号 1, p. 1-4, 発行日 2018-02-13
• ISSN / 収録物識別子タイプ: ISSN
• ISSN / 収録物識別子: 2188-8752
• Notice: SIG Technical Reports are nonrefereed and hence may later appear in any journals, conferences, symposia, etc.
• 出版者 / 言語: ja
• 出版者: 情報処理学会