Research on Video Captioning with a Late Fusion Based Multimodal Transformer Network

• Item type: National Convention(1)
• 公開日: 2023-02-16
• タイトル: Research on Video Captioning with a Late Fusion Based Multimodal Transformer Network
• 言語: eng
• キーワード / 主題Scheme: Other
• キーワード / 主題: 人工知能と認知科学
• 資源タイプ / 資源タイプ識別子: http://purl.org/coar/resource_type/c_5794
• 資源タイプ: conference paper
• 著者所属: 早大
• 著者所属: 早大
• 著者所属: 早大
• 著者名: 鮑, 飛; 石川, 孝明; 渡辺, 裕
• 論文抄録 / 内容記述タイプ: Other
• 論文抄録 / 内容記述: Video captioning is a task that aims to generate natural language descriptions of a given video, which has drawn increasing attention in recent years. As the video is a combination of different modalities of data, multimodal learning has become relevant in the video captioning area. One of the multimodal fusion strategies, early-fusion, which involves simply concatenating multiple modalities before inputting them into the model, is a general operation used by most methods. However, such a naive operation may lead to potential representations being ignored by the model and usually suffers from a high computational cost, even a quadratic cost with regard to the length of input information in Transformer. Therefore, we propose a method that integrates different modalities in a late-fusion way, which reduces the computational complexity and increases the evaluation metric CIDEr by 1.22.
• 書誌レコードID / 収録物識別子タイプ: NCID
• 書誌レコードID / 収録物識別子: AN00349328
• 書誌情報: 第85回全国大会講演論文集; 巻 2023, 号 1, p. 195-196, 発行日 2023-02-16
• 出版者 / 言語: ja
• 出版者: 情報処理学会