Sampling inequalities affect generalization of neuroimaging-based diagnostic classifiers in psychiatry

作者：Zhiyi Chen, Bowen Hu, Xuerong Liu, Benjamin Becker, Simon B. Eickhoff, Kuan Miao, Xingmei Gu, Yancheng Tang, Xin Dai, Chao Li, Artemiy Leonov, Zhibing Xiao, Zhengzhi Feng, Chen Ji, Hu Chuan-Peng · 发表于：BMC Medicine · 年份：2023 · DOI：10.1186/s12916-023-02941-4 · 被引用次数：34 · 研究领域：Functional Brain Connectivity Studies、Digital Mental Health Interventions、Mental Health Research Topics

Abstract Background The development of machine learning models for aiding in the diagnosis of mental disorder is recognized as a significant breakthrough in the field of psychiatry. However, clinical practice of such models remains a challenge, with poor generalizability being a major limitation. Methods Here, we conducted a pre-registered meta-research assessment on neuroimaging-based models in the psychiatric literature, quantitatively examining global and regional sampling issues over recent decades, from a view that has been relatively underexplored. A total of 476 studies ( n = 118,137) were included in the current assessment. Based on these findings, we built a comprehensive 5-star rating system to quantitatively evaluate the quality of existing machine learning models for psychiatric diagnoses. Results A global sampling inequality in these models was revealed quantitatively (sampling Gini coefficient ( G ) = 0.81, p < .01), varying across different countries (regions) (e.g., China, G = 0.47; the USA, G = 0.58; Germany, G = 0.78; the UK, G = 0.87). Furthermore, the severity of this sampling inequality was significantly predicted by national economic levels ( β = − 2.75, p < .001, R 2 adj = 0.40; r = − .84, 95% CI: − .41 to − .97), and was plausibly predictable for model performance, with higher sampling inequality for reporting higher classification accuracy. Further analyses showed that lack of independent testing (84.24% of models, 95% CI: 81.0–87.5%), improper ...