Enhancing Dysarthric Speech Recognition with Auxiliary Feature Fusion Module: Exploring Articulatory-related Features from Foundation Models

• Item type: SIG Technical Reports(1)
• 公開日: 2023-11-25
• タイトル: Enhancing Dysarthric Speech Recognition with Auxiliary Feature Fusion Module: Exploring Articulatory-related Features from Foundation Models
• タイトル / 言語: en
• タイトル: Enhancing Dysarthric Speech Recognition with Auxiliary Feature Fusion Module: Exploring Articulatory-related Features from Foundation Models
• 言語: eng
• キーワード / 主題Scheme: Other
• キーワード / 主題: ポスター
• 資源タイプ / 資源タイプ識別子: http://purl.org/coar/resource_type/c_18gh
• 資源タイプ: technical report
• 著者所属: Tianjin Key Laboratory of Cognitive Computing and Application, College of Intelligence and Computing, Tianjin University／Graduate School of Engineering, The University of Tokyo
• 著者所属: Tianjin Key Laboratory of Cognitive Computing and Application, College of Intelligence and Computing, Tianjin University
• 著者所属: Tianjin Key Laboratory of Cognitive Computing and Application, College of Intelligence and Computing, Tianjin University
• 著者所属: Graduate School of Engineering, The University of Tokyo
• 著者所属: Tianjin Key Laboratory of Cognitive Computing and Application, College of Intelligence and Computing, Tianjin University
• 著者所属(英): en
• 著者所属(英): Tianjin Key Laboratory of Cognitive Computing and Application, College of Intelligence and Computing, Tianjin University / Graduate School of Engineering, The University of Tokyo
• 著者所属(英): en
• 著者所属(英): Tianjin Key Laboratory of Cognitive Computing and Application, College of Intelligence and Computing, Tianjin University
• 著者所属(英): en
• 著者所属(英): Tianjin Key Laboratory of Cognitive Computing and Application, College of Intelligence and Computing, Tianjin University
• 著者所属(英): en
• 著者所属(英): Graduate School of Engineering, The University of Tokyo
• 著者所属(英): en
• 著者所属(英): Tianjin Key Laboratory of Cognitive Computing and Application, College of Intelligence and Computing, Tianjin University
• 著者名: Yuqin, Lin; Longbiao, Wang; Jianwu, Dang; Nobuaki, Minematsu
• 著者名(英): Yuqin, Lin; Longbiao, Wang; Jianwu, Dang; Nobuaki, Minematsu
• 論文抄録 / 内容記述タイプ: Other
• 論文抄録 / 内容記述: Addressing dysarthric speech variability in Automatic Speech Recognition (ASR) is crucial for improving human-computer interactions for everyone. This paper proposes the Auxiliary Features Fusion (AFFusion) module, which leverages phonetic and articulatory-related features from models like wav2vec to compensate for distorted acoustics in dysarthric ASR. Experimental results using AFFusion with various feature models demonstrate its effectiveness on dysarthric databases. Interestingly, the analysis suggests that AFFusion shares similarities with human speech perception processes, offering potential insights into addressing fuzzy recognition in dysarthric ASR based on the motor theory of speech perception.
• 論文抄録(英) / 内容記述タイプ: Other
• 論文抄録(英) / 内容記述: Addressing dysarthric speech variability in Automatic Speech Recognition (ASR) is crucial for improving human-computer interactions for everyone. This paper proposes the Auxiliary Features Fusion (AFFusion) module, which leverages phonetic and articulatory-related features from models like wav2vec to compensate for distorted acoustics in dysarthric ASR. Experimental results using AFFusion with various feature models demonstrate its effectiveness on dysarthric databases. Interestingly, the analysis suggests that AFFusion shares similarities with human speech perception processes, offering potential insights into addressing fuzzy recognition in dysarthric ASR based on the motor theory of speech perception.
• 書誌レコードID / 収録物識別子タイプ: NCID
• 書誌レコードID / 収録物識別子: AN10115061
• 書誌情報: 研究報告自然言語処理（NL）; 巻 2023-NL-258, 号 14, p. 1-6, 発行日 2023-11-25
• ISSN / 収録物識別子タイプ: ISSN
• ISSN / 収録物識別子: 2188-8779
• Notice: SIG Technical Reports are nonrefereed and hence may later appear in any journals, conferences, symposia, etc.
• 出版者 / 言語: ja
• 出版者: 情報処理学会