2026-04-17T16:54:34Z https://ipsj.ixsq.nii.ac.jp/oai

oai:ipsj.ixsq.nii.ac.jp:00232611 2025-01-19T10:23:28Z 1164:4061:11479:11513

Detecting Distress Variations Using Multimodal Data Obtained through Interaction with A Smart Speaker Detecting Distress Variations Using Multimodal Data Obtained through Interaction with A Smart Speaker Chingyuan, Lin Yuki, Matsuda Hirohiko, Suwa Keiichi, Yasumoto Chingyuan, Lin Yuki, Matsuda Hirohiko, Suwa Keiichi, Yasumoto センシング Mental health significantly affects people, with excessive stress potentially causing depression, low productivity, and suicidal thoughts. It can also harm physical health, impacting appetite and sleep, and may lead to other diseases. In most cases, individuals do not notice stress buildup until their health severely deteriorates. Thus, daily monitoring of stress levels is essential. In this study, we aim to realize a method to estimate people’s distress levels in everyday life through conversation with a smart speaker. We set up a smart speaker in the bedrooms of participants to simulate a home environment and recorded their interactions with it using a webcam. These recordings allowed us to analyze facial expressions, voice, and heart rate data. We processed these features and predicted levels of Happiness, Depression, and Anxiety. Participants completed questionnaires using the Depression and Anxiety Mood Scale (DAMS) after each session, providing emotion labels with scores from 0 to 18. In a 14-day experiment involving seven participants aged 22-24, the MAE for Happiness, Depression, and Anxiety levels were 2.04, 2.59, and 2.31, respectively, while the RMSE for these distress levels were 2.63, 3.20, and 2.91. Mental health significantly affects people, with excessive stress potentially causing depression, low productivity, and suicidal thoughts. It can also harm physical health, impacting appetite and sleep, and may lead to other diseases. In most cases, individuals do not notice stress buildup until their health severely deteriorates. Thus, daily monitoring of stress levels is essential. In this study, we aim to realize a method to estimate people’s distress levels in everyday life through conversation with a smart speaker. We set up a smart speaker in the bedrooms of participants to simulate a home environment and recorded their interactions with it using a webcam. These recordings allowed us to analyze facial expressions, voice, and heart rate data. We processed these features and predicted levels of Happiness, Depression, and Anxiety. Participants completed questionnaires using the Depression and Anxiety Mood Scale (DAMS) after each session, providing emotion labels with scores from 0 to 18. In a 14-day experiment involving seven participants aged 22-24, the MAE for Happiness, Depression, and Anxiety levels were 2.04, 2.59, and 2.31, respectively, while the RMSE for these distress levels were 2.63, 3.20, and 2.91. 情報処理学会 2024-02-22 eng technical report https://ipsj.ixsq.nii.ac.jp/records/232611 2188-8698 AA11838947 研究報告ユビキタスコンピューティングシステム（UBI） 2024-UBI-81 7 1 6 https://ipsj.ixsq.nii.ac.jp/record/232611/files/IPSJ-UBI24081007.pdf application/pdf 2.1 MB