Deep skin diseases diagnostic system with Dual-channel Image and Extracted Text

作者：Huanyu Li, Peng Zhang, Zikun Wei, Tian Qian, Yiqi Tang, Kun Hu, Xianqiong Huang, Xinxin Xia, Yishuang Zhang, Haixing Cheng, Fubing Yu, Wenjia Zhang, Kena Dan, Xuan Liu, Shujun Ye, Guangqiao He, Xia Jiang, Liwei Liu, Yukun Fan, Tingting Song, Guomin Zhou, Ziyi Wang, Dao‐Jun Zhang, Junwei Lv · 发表于：Frontiers in Artificial Intelligence · 年份：2023 · DOI：10.3389/frai.2023.1213620 · 被引用次数：11 · 研究领域：Cutaneous Melanoma Detection and Management、Dermatological and COVID-19 studies、Dermatology and Skin Diseases

Background: Due to the lower reliability of laboratory tests, skin diseases are more suitable for diagnosis with AI models. There are limited AI dermatology diagnostic models combining images and text; few of these are for Asian populations, and few cover the most common types of diseases. Methods: Leveraging a dataset sourced from Asia comprising over 200,000 images and 220,000 medical records, we explored a deep learning-based system for Dual-channel images and extracted text for the diagnosis of skin diseases model DIET-AI to diagnose 31 skin diseases, which covers the majority of common skin diseases. From 1 September to 1 December 2021, we prospectively collected images from 6,043 cases and medical records from 15 hospitals in seven provinces in China. Then the performance of DIET-AI was compared with that of six doctors of different seniorities in the clinical dataset. Results: The average performance of DIET-AI in 31 diseases was not less than that of all the doctors of different seniorities. By comparing the area under the curve, sensitivity, and specificity, we demonstrate that the DIET-AI model is effective in clinical scenarios. In addition, medical records affect the performance of DIET-AI and physicians to varying degrees. Conclusion: This is the largest dermatological dataset for the Chinese demographic. For the first time, we built a Dual-channel image classification model on a non-cancer dermatitis dataset with both images and medical records and achieved comp...