A Proposal of Emotion Estimation Method in Social Robots using Facial Expression Recognition Model with Graph-based Techniques

• Item type: Symposium(1)
• 公開日: 2023-12-20
• タイトル: A Proposal of Emotion Estimation Method in Social Robots using Facial Expression Recognition Model with Graph-based Techniques
• タイトル / 言語: en
• タイトル: A Proposal of Emotion Estimation Method in Social Robots using Facial Expression Recognition Model with Graph-based Techniques
• 言語: eng
• 資源タイプ / 資源タイプ識別子: http://purl.org/coar/resource_type/c_5794
• 資源タイプ: conference paper
• 著者所属: King Mongkut's University of Technology Thonburi
• 著者所属: Shibaura Institute of Technology
• 著者所属: Shibaura Institute of Technology
• 著者所属: Shibaura Institute of Technology
• 著者所属: Shibaura Institute of Technology
• 著者所属(英): en
• 著者所属(英): King Mongkut's University of Technology Thonburi
• 著者所属(英): en
• 著者所属(英): Shibaura Institute of Technology
• 著者所属(英): en
• 著者所属(英): Shibaura Institute of Technology
• 著者所属(英): en
• 著者所属(英): Shibaura Institute of Technology
• 著者所属(英): en
• 著者所属(英): Shibaura Institute of Technology
• 著者名: Nopphakorn, Subsa-ard; Felipe, Yudi Fulini; Tipporn, Laohakangvalvit; Kaoru, Suzuki; Midori, Sugaya
• 著者名(英): Nopphakorn, Subsa-ard; Felipe, Yudi Fulini; Tipporn, Laohakangvalvit; Kaoru, Suzuki; Midori, Sugaya
• 論文抄録 / 内容記述タイプ: Other
• 論文抄録 / 内容記述: Social robots play a crucial role in human-robot interaction, offering potential benefits in various applications, including nursing care and home assistance. Our previous study proposed “a prototype of multi-modal interaction robot based on emotion estimation method using physiological signals”, which presented our implementation on using electroencephalography (EEG) and Heart Rate Variability (HRV) as physiological signals to estimate the emotional states of human. However, wearable sensing devices could be inconvenient to use in the real world for nursing care robots or home-use robots because they may irritate wearing. To tackle this problem, we proposed a new approach for emotion estimation method. Various studies have proposed various methods to estimate human emotions such as recognition of facial expressions, postures, tone of voice, and speech, including the integration of those techniques for further classification. In this study, we focus on the facial expression recognition because facial expression can be captured by a camera instead of any wearable devices, which is easy to use and deploy. Our proposed method employed facial expression recognition with a graph-based technique using an open-source MediaPipe framework to extract face landmarks from photos and used Deep Learning technique to build emotion classification from face mesh data. To validate our proposed method, we used FER2013 dataset from Microsoft as a benchmark dataset. Finally, we implemented our method to the prototype of robot. The results show that our proposed method effectively classify emotions from facial expression captured by web camera, and the robot can interact according to the estimated emotions continuously over time.
• 論文抄録(英) / 内容記述タイプ: Other
• 論文抄録(英) / 内容記述: Social robots play a crucial role in human-robot interaction, offering potential benefits in various applications, including nursing care and home assistance. Our previous study proposed “a prototype of multi-modal interaction robot based on emotion estimation method using physiological signals”, which presented our implementation on using electroencephalography (EEG) and Heart Rate Variability (HRV) as physiological signals to estimate the emotional states of human. However, wearable sensing devices could be inconvenient to use in the real world for nursing care robots or home-use robots because they may irritate wearing. To tackle this problem, we proposed a new approach for emotion estimation method. Various studies have proposed various methods to estimate human emotions such as recognition of facial expressions, postures, tone of voice, and speech, including the integration of those techniques for further classification. In this study, we focus on the facial expression recognition because facial expression can be captured by a camera instead of any wearable devices, which is easy to use and deploy. Our proposed method employed facial expression recognition with a graph-based technique using an open-source MediaPipe framework to extract face landmarks from photos and used Deep Learning technique to build emotion classification from face mesh data. To validate our proposed method, we used FER2013 dataset from Microsoft as a benchmark dataset. Finally, we implemented our method to the prototype of robot. The results show that our proposed method effectively classify emotions from facial expression captured by web camera, and the robot can interact according to the estimated emotions continuously over time.
• 書誌情報: Proceedings of Asia Pacific Conference on Robot IoT System Development and Platform; 巻 2023, p. 40-41, 発行日 2023-12-20
• 出版者 / 言語: ja
• 出版者: 情報処理学会