Selecting Iconic Gesture Forms Based on Typical Entity Images

• Item type: Journal(1)
• 公開日: 2024-02-15
• タイトル: Selecting Iconic Gesture Forms Based on Typical Entity Images
• タイトル / 言語: en
• タイトル: Selecting Iconic Gesture Forms Based on Typical Entity Images
• 言語: eng
• キーワード / 主題Scheme: Other
• キーワード / 主題: [特集:インタラクションの理解および基盤・応用技術] gesture generation, gesture form, iconic gesture, image representation, deep neural network
• 資源タイプ / 資源タイプ識別子: http://purl.org/coar/resource_type/c_6501
• 資源タイプ: journal article
• 著者所属: Faculty of Science and Technology, Seikei University
• 著者所属: Human Informatics Laboratories, NTT Corporation
• 著者所属: Human Informatics Laboratories, NTT Corporation
• 著者所属: Human Informatics Laboratories, NTT Corporation
• 著者所属(英): en
• 著者所属(英): Faculty of Science and Technology, Seikei University
• 著者所属(英): en
• 著者所属(英): Human Informatics Laboratories, NTT Corporation
• 著者所属(英): en
• 著者所属(英): Human Informatics Laboratories, NTT Corporation
• 著者所属(英): en
• 著者所属(英): Human Informatics Laboratories, NTT Corporation
• 著者名: Yukiko, I. Nakano; Fumio, Nihei; Ryo, Ishii; Ryuichiro, Higashinaka
• 著者名(英): Yukiko, I. Nakano; Fumio, Nihei; Ryo, Ishii; Ryuichiro, Higashinaka
• 論文抄録 / 内容記述タイプ: Other
• 論文抄録 / 内容記述: Hand gestures are communication signals that emphasize an important part of an utterance and express the concept of emphasized words. Iconic gestures are hand gestures that depict concrete actions, objects, or events mentioned in speech. In this study, assuming that gesture forms of iconic gestures are determined based on the image of a given object in the speaker's mind, we propose a method for selecting iconic gesture forms based on the image representation obtained from a set of pictures of an object. First, we asked annotators to select a gesture form that best expresses the meaning of a given word based on a typical image and concept in their minds. We also collected a set of pictures of each entity from the web and created an average image representation from them. We then created a Deep Neural Network (DNN) model that takes a set of pictures of objects as input and predicts the typical gesture form that originates from the human mind. In the model evaluation experiment, our two-step gesture form selection method successfully classified seven types of gesture forms with an accuracy of over 62%. Furthermore, we created character animations that performed selected gestures and conducted a preliminary perception study to examine how human users perceive animated iconic gestures.; ------------------------------; This is a preprint of an article intended for publication Journal of; Information Processing(JIP). This preprint should not be cited. This; article should be cited as: Journal of Information Processing Vol.32(2024) (online); DOI http://dx.doi.org/10.2197/ipsjjip.32.196; ------------------------------
• 論文抄録(英) / 内容記述タイプ: Other
• 論文抄録(英) / 内容記述: Hand gestures are communication signals that emphasize an important part of an utterance and express the concept of emphasized words. Iconic gestures are hand gestures that depict concrete actions, objects, or events mentioned in speech. In this study, assuming that gesture forms of iconic gestures are determined based on the image of a given object in the speaker's mind, we propose a method for selecting iconic gesture forms based on the image representation obtained from a set of pictures of an object. First, we asked annotators to select a gesture form that best expresses the meaning of a given word based on a typical image and concept in their minds. We also collected a set of pictures of each entity from the web and created an average image representation from them. We then created a Deep Neural Network (DNN) model that takes a set of pictures of objects as input and predicts the typical gesture form that originates from the human mind. In the model evaluation experiment, our two-step gesture form selection method successfully classified seven types of gesture forms with an accuracy of over 62%. Furthermore, we created character animations that performed selected gestures and conducted a preliminary perception study to examine how human users perceive animated iconic gestures.; ------------------------------; This is a preprint of an article intended for publication Journal of; Information Processing(JIP). This preprint should not be cited. This; article should be cited as: Journal of Information Processing Vol.32(2024) (online); DOI http://dx.doi.org/10.2197/ipsjjip.32.196; ------------------------------
• 書誌レコードID / 収録物識別子タイプ: NCID
• 書誌レコードID / 収録物識別子: AN00116647
• 書誌情報: 情報処理学会論文誌; 巻 65, 号 2, 発行日 2024-02-15
• ISSN / 収録物識別子タイプ: ISSN
• ISSN / 収録物識別子: 1882-7764
• 公開者 / 言語: ja
• 公開者 / 出版者: 情報処理学会