Assessments of Credibility in the Social and Behavioral Sciences

作者：Anna Lou Abatayo, Titipat Achakulvisut, Daniel E. Acuña, Balazs Aczel, Laxmaan Balaji, Anita Bandrowski, D. M. Benjamin, Michaël Bishop, Gary L. Brase, Andrew William Brown, Martin Bush, James Caverlee, Tatiana Chakravorti, Yiling Chen, Macie Daley, Morteza Dehghani, Mirka Dirzo, Anna Dreber, Peter Eckmann, Timothy M. Errington, Qizhang Feng, Fiona Fidler, Samuel Field, Nicholas William Fox, Robert Fraleigh, Aaron Frank, Hannah Fraser, James E. Gentile, C Lee Giles, Brandon Goldfedder, Phil Gooch, Michael Gordon, Elliot Gould, Christopher Griffin, Timothy R. Gulden, Noah Haber, Krystal Hahn, Felix Holzmeister, Xia Hu, Yuzhong Huang, Magnus Johannesson, Brendan Kennedy, Melissa Kline Struhl, Anthony M. Kwasnica, Dong-Ho Lee, Kristina Lerman, Yang Liu, Allegra E. Pearce, Isabella Mandema, Alexandru Marcoci, Brinna Mawhinney, Souad McIntosh, Michael Mclaughlin, Arjun Menon, Olivia Miske, Fallon Mody, Fred Morstatter, Nishanth Sridhar Nakshatri, Brian A. Nosek, Michele B. Nuijten, David Pennock, Thomas Pfeiffer, Darien Pipkin, Jay Pujara, Sarah Rajtmajer, Martijn Roelandse, Adam Russell, Priya Silverstein, Vaibhav Singh, Courtney K. Soderberg, Anna Ms Squicciarini, Theresa Stankov, Jordan W Suchow, Barnabas Szaszi, Louisa Tran, Peter A. Vesk, Tim Vines, Colby J. Vorland, Juntao Wang, Zhuoer Wang, David P. Wilkinson, Bonnie C. Wintle, Jian Wu · 年份：2026 · DOI：10.31222/osf.io/7u58q_v1 · 被引用次数：2 · 研究领域：Meta-analysis and systematic reviews、Reliability and Agreement in Measurement、Academic integrity and plagiarism

Credibility assessment — determining whether research findings are trustworthy or believable — is essential to the research process. One aspect of credibility is repeatability, which includes assessing whether consistent results are obtained when using new data to answer the same question (replicability), when repeating the original analyses with the original data (reproducibility), or when conducting alternative analyses about the same question with the original data (robustness). These features of repeatability differ in the resources required to investigate them, and it is unknown how they relate with one another and with other features of credibility. We investigated relationships among credibility measures in a stratified random sample of claims made across the social and behavioral sciences. Measures of repeatability were modestly correlated with each other (r’s = 0.30, -0.04, -0.23) though the correlation between robustness and reproducibility is likely misestimated because selecting claims for robustness testing was partly contingent on reproducibility success. Replicability and human and machine predictions of replicability were modestly correlated (Median r = 0.23; Range = -0.10 to 0.47). Though estimated with substantial uncertainty in some cases, no discipline showed consistently higher repeatability than other disciplines across measures. For example, Education had the highest replicability estimate (0.63, 95% CI [.32 - .86]) and the lowest reproducibility estima...