Generating Emotional Region and Text Representations for Joint Visual Textual Sentiment Analysis

• Item type: SIG Technical Reports(1)
• 公開日: 2019-12-16
• タイトル: Generating Emotional Region and Text Representations for Joint Visual Textual Sentiment Analysis
• タイトル / 言語: en
• タイトル: Generating Emotional Region and Text Representations for Joint Visual Textual Sentiment Analysis
• 言語: eng
• 資源タイプ / 資源タイプ識別子: http://purl.org/coar/resource_type/c_18gh
• 資源タイプ: technical report
• 著者所属: Graduate School of Information, Production and Systems, Waseda University
• 著者所属(英): en
• 著者所属(英): Graduate School of Information, Production and Systems, Waseda University
• 著者名: Jiayi, Zhao; Mizuho, Iwaihara
• 著者名(英): Jiayi, Zhao; Mizuho, Iwaihara
• 論文抄録 / 内容記述タイプ: Other
• 論文抄録 / 内容記述: Recently, social media users are increasingly using images and texts to express their opinions and share their experiences. Sentiment analysis of images and texts can help better extract users' sentiments toward events or topic, where prediction of sentiment is an important task. Traditional sentiment analysis on images with textual description only considers the whole image and the whole sentence to predict the result (positive or negative). However, both the whole image and local emotional regions convey significant sentiment information and the emotional textual fragment of the description expresses sentiment as well. Therefore, we propose a deep fusion network to detect the most emotional region in the image and associate the emotional region with the most emotional textual fragment in the description. For predicting the sentiment of the whole image, we consider synthetically integrating the four parts: Textual description, emotional region, and most matching emotional textual fragment. At last, we build a fusion layer to combine the output of the representations to predict the overall sentiment. We carry out experiments on a subset selected from dataset Visual Sentiment Ontology (VSO). The results show that our proposed method can effectively predict sentiment and performs competitively to other methods.
• 論文抄録(英) / 内容記述タイプ: Other
• 論文抄録(英) / 内容記述: Recently, social media users are increasingly using images and texts to express their opinions and share their experiences. Sentiment analysis of images and texts can help better extract users' sentiments toward events or topic, where prediction of sentiment is an important task. Traditional sentiment analysis on images with textual description only considers the whole image and the whole sentence to predict the result (positive or negative). However, both the whole image and local emotional regions convey significant sentiment information and the emotional textual fragment of the description expresses sentiment as well. Therefore, we propose a deep fusion network to detect the most emotional region in the image and associate the emotional region with the most emotional textual fragment in the description. For predicting the sentiment of the whole image, we consider synthetically integrating the four parts: Textual description, emotional region, and most matching emotional textual fragment. At last, we build a fusion layer to combine the output of the representations to predict the overall sentiment. We carry out experiments on a subset selected from dataset Visual Sentiment Ontology (VSO). The results show that our proposed method can effectively predict sentiment and performs competitively to other methods.
• 書誌レコードID / 収録物識別子タイプ: NCID
• 書誌レコードID / 収録物識別子: AN10112482
• 書誌情報: 研究報告データベースシステム（DBS）; 巻 2019-DBS-170, 号 6, p. 1-6, 発行日 2019-12-16
• ISSN / 収録物識別子タイプ: ISSN
• ISSN / 収録物識別子: 2188-871X
• Notice: SIG Technical Reports are nonrefereed and hence may later appear in any journals, conferences, symposia, etc.
• 出版者 / 言語: ja
• 出版者: 情報処理学会