Multimodal fusion diagnosis of depression and anxiety based on face video

作者：Chen Wang, Lizhong Liang, Xiaofeng Liu, Yao Lu, Jihong Shen, Hui Luo, Wanqing Xie · 年份：2021 · DOI：10.1109/icmipe53131.2021.9698881 · 被引用次数：10 · 研究领域：Emotion and Mood Recognition、Mental Health via Writing、Digital Mental Health Interventions

In order to diagnose depression and anxiety, clinicians will conduct interviews with subjects. If large-scale screening is carried out, this method is too costly and difficult to implement. Because facial expressions play an important role in the diagnosis of clinicians, this provides an opportunity to solve this problem. Therefore, we recorded 303 subjects who answered the self-rated anxiety scale (SAS) and the self-rated depression scale (SDS) Video. Based on Convolutional Neural Networks (CNN) and Long Short-Term Memory Networks (LSTM), by using either of these two types of videos alone as a binary classification experiment, the accuracy of the diagnosis of depression is 72.53%, and the diagnosis of anxiety is 72.08%. In addition, by fusing the two types of videos to diagnose anxiety, depression, and normal in three categories, the accuracy of the model is 80.22%. Through the comparison of the results, the multimodal fusion diagnosis can not only diagnose the three categories but also has the highest accuracy. This model can be deployed on smartphones, not only for large-scale screening but also to assist doctors in diagnosis.