Bilingual Corpus Mining and Multistage Fine-tuning for Improving Machine Translation of Lecture Transcripts

• Item type: Journal(1)
• 公開日: 2024-08-15
• タイトル: Bilingual Corpus Mining and Multistage Fine-tuning for Improving Machine Translation of Lecture Transcripts
• タイトル / 言語: en
• タイトル: Bilingual Corpus Mining and Multistage Fine-tuning for Improving Machine Translation of Lecture Transcripts
• 言語: eng
• キーワード / 主題Scheme: Other
• キーワード / 主題: [一般論文] lecture translation, parallel corpus mining, machine translation, sentence alignment
• 資源タイプ / 資源タイプ識別子: http://purl.org/coar/resource_type/c_6501
• 資源タイプ: journal article
• 著者所属: National Institute of Information and Communications Technology／Kyoto University
• 著者所属: National Institute of Information and Communications Technology
• 著者所属: Kyoto University
• 著者所属: National Institute of Information and Communications Technology
• 著者所属: National Institute of Informatics／Kyoto University
• 著者所属(英): en
• 著者所属(英): National Institute of Information and Communications Technology / Kyoto University
• 著者所属(英): en
• 著者所属(英): National Institute of Information and Communications Technology
• 著者所属(英): en
• 著者所属(英): Kyoto University
• 著者所属(英): en
• 著者所属(英): National Institute of Information and Communications Technology
• 著者所属(英): en
• 著者所属(英): National Institute of Informatics / Kyoto University
• 著者名: Haiyue, Song; Raj, Dabre; Chenhui, Chu; Atsushi, Fujita; Sadao, Kurohashi
• 著者名(英): Haiyue, Song; Raj, Dabre; Chenhui, Chu; Atsushi, Fujita; Sadao, Kurohashi
• 論文抄録 / 内容記述タイプ: Other
• 論文抄録 / 内容記述: Lecture transcript translation helps learners understand online courses; however, building a high-quality lecture machine translation system lacks publicly available parallel corpora. To address this, we examine a framework for parallel corpus mining, which provides a quick and effective way to mine a parallel corpus from publicly available lectures on Coursera. To create the parallel corpora, we propose a dynamic programming based sentence alignment algorithm which leverages the cosine similarity of machine-translated sentences. The sentence alignment F1 score reaches 96%, which is higher than using the BERTScore, LASER, or sentBERT methods. For both English-Japanese and English-Chinese lecture translations, we extracted parallel corpora of approximately 50,000 lines and created development and test sets through manual filtering for benchmarking translation performance. Through machine translation experiments, we show that the mined corpora enhance the quality of lecture transcript translation when used in conjunction with out-of-domain parallel corpora via multistage fine-tuning. Furthermore, this study also suggests guidelines for gathering and cleaning corpora, mining parallel sentences, cleaning noise in the mined data, and creating high-quality evaluation splits. For the sake of reproducibility, we have released the corpora as well as the code to create them.; ------------------------------; This is a preprint of an article intended for publication Journal of; Information Processing(JIP). This preprint should not be cited. This; article should be cited as: Journal of Information Processing Vol.32(2024) (online); DOI http://dx.doi.org/10.2197/ipsjjip.32.628; ------------------------------
• 論文抄録(英) / 内容記述タイプ: Other
• 論文抄録(英) / 内容記述: Lecture transcript translation helps learners understand online courses; however, building a high-quality lecture machine translation system lacks publicly available parallel corpora. To address this, we examine a framework for parallel corpus mining, which provides a quick and effective way to mine a parallel corpus from publicly available lectures on Coursera. To create the parallel corpora, we propose a dynamic programming based sentence alignment algorithm which leverages the cosine similarity of machine-translated sentences. The sentence alignment F1 score reaches 96%, which is higher than using the BERTScore, LASER, or sentBERT methods. For both English-Japanese and English-Chinese lecture translations, we extracted parallel corpora of approximately 50,000 lines and created development and test sets through manual filtering for benchmarking translation performance. Through machine translation experiments, we show that the mined corpora enhance the quality of lecture transcript translation when used in conjunction with out-of-domain parallel corpora via multistage fine-tuning. Furthermore, this study also suggests guidelines for gathering and cleaning corpora, mining parallel sentences, cleaning noise in the mined data, and creating high-quality evaluation splits. For the sake of reproducibility, we have released the corpora as well as the code to create them.; ------------------------------; This is a preprint of an article intended for publication Journal of; Information Processing(JIP). This preprint should not be cited. This; article should be cited as: Journal of Information Processing Vol.32(2024) (online); DOI http://dx.doi.org/10.2197/ipsjjip.32.628; ------------------------------
• 書誌レコードID / 収録物識別子タイプ: NCID
• 書誌レコードID / 収録物識別子: AN00116647
• 書誌情報: 情報処理学会論文誌; 巻 65, 号 8, 発行日 2024-08-15
• ISSN / 収録物識別子タイプ: ISSN
• ISSN / 収録物識別子: 1882-7764
• 公開者 / 言語: ja
• 公開者 / 出版者: 情報処理学会