A Study on Sparsity Learning for Neural Network Acceleration

• Item type: SIG Technical Reports(1)
• 公開日: 2020-02-20
• タイトル: A Study on Sparsity Learning for Neural Network Acceleration
• タイトル / 言語: en
• タイトル: A Study on Sparsity Learning for Neural Network Acceleration
• 言語: eng
• 資源タイプ / 資源タイプ識別子: http://purl.org/coar/resource_type/c_18gh
• 資源タイプ: technical report
• 著者所属: KDDI Research, Inc.
• 著者所属: KDDI Research, Inc.
• 著者所属(英): en
• 著者所属(英): KDDI Research, Inc.
• 著者所属(英): en
• 著者所属(英): KDDI Research, Inc.
• 著者名: Jianfeng, Xu; Kazuyuki, Tasaka
• 著者名(英): Jianfeng, Xu; Kazuyuki, Tasaka
• 論文抄録 / 内容記述タイプ: Other
• 論文抄録 / 内容記述: A sparsity learning framework is effective as they learn and prune the models in an end-to-end data-driven manner. However, existing works impose the same sparsity regularization on all filters indiscriminately, which can hardly result in an optimal structure-sparse network. In this paper, we propose a Saliency-Adaptive Sparsity Learning (SASL) approach for further optimization. The saliency of each filter is measured from two aspects: the importance for the prediction performance and the consumed computational resources. During sparsity learning, the regularization is adjusted according to the saliency, so our optimized format can better preserve the prediction performance while zeroing out more computation-heavy filters. The calculation for saliency introduces minimum overhead to the training process, which means our SASL is very efficient. During the pruning phase, in order to optimize the proposed data-dependent criterion, a hard sample mining strategy is utilized, which shows higher effectiveness and efficiency. Extensive experiments demonstrate the superior performance of our method.
• 論文抄録(英) / 内容記述タイプ: Other
• 論文抄録(英) / 内容記述: A sparsity learning framework is effective as they learn and prune the models in an end-to-end data-driven manner. However, existing works impose the same sparsity regularization on all filters indiscriminately, which can hardly result in an optimal structure-sparse network. In this paper, we propose a Saliency-Adaptive Sparsity Learning (SASL) approach for further optimization. The saliency of each filter is measured from two aspects: the importance for the prediction performance and the consumed computational resources. During sparsity learning, the regularization is adjusted according to the saliency, so our optimized format can better preserve the prediction performance while zeroing out more computation-heavy filters. The calculation for saliency introduces minimum overhead to the training process, which means our SASL is very efficient. During the pruning phase, in order to optimize the proposed data-dependent criterion, a hard sample mining strategy is utilized, which shows higher effectiveness and efficiency. Extensive experiments demonstrate the superior performance of our method.
• 書誌レコードID / 収録物識別子タイプ: NCID
• 書誌レコードID / 収録物識別子: AN10438399
• 書誌情報: 研究報告オーディオビジュアル複合情報処理（AVM）; 巻 2020-AVM-108, 号 10, p. 1-6, 発行日 2020-02-20
• ISSN / 収録物識別子タイプ: ISSN
• ISSN / 収録物識別子: 2188-8582
• Notice: SIG Technical Reports are nonrefereed and hence may later appear in any journals, conferences, symposia, etc.
• 出版者 / 言語: ja
• 出版者: 情報処理学会