Examining the generalizability of research findings from archival data

作者：Andrew Delios, Elena Giulia Clemente, Tao Wu, Hongbin Tan, Yong Wang, Michael Gordon, Domenico Viganola, Zhaowei Chen, Anna Dreber, Magnus Johannesson, Thomas Pfeiffer, Eric Luis Uhlmann, Ahmad M. Abd Al-Aziz, Ajay T. Abraham, Jais Trojan, Matúš Adamkovič, Елена Агадуллина, Jungsoo Ahn, Çinla Akinci, Handan Akkaş, David Albrecht, Shilaan Alzahawi, Marcio Alves Amaral-Baptista, Rahul Anand, Kevin Francis U. Ang, Frederik Anseel, John Jamir Benzon R. Aruta, Mujeeba Ashraf, Bradley J. Baker, Xueqi Bao, Ernest Baskin, Hanoku Bathula, Christopher W. Bauman, Jozef Bavoľár, Seçil Bayraktar, S. Beckman, Aaron S. Benjamin, Stephanie Brown, Jeffrey Buckley, Ricardo E. Buitrago R., Jefferson Luiz Bution, Nick Byrd, Clara Carrera, Eugene M. Caruso, Minxia Chen, Lin Chen, Eyyub Ensari Cicerali, Eric David Cohen, Marcus Credé, J. David Cummins, Linus Dahlander, David P. Daniels, Lea Liat Daskalo, Ian Dawson, Martin V. Day, Erik Dietl, Artur Domurat, Jacinta Dsilva, Christilene du Plessis, Dmitrii Dubrov, Sarah Edris, Christian T. Elbaek, Mahmoud Medhat Elsherif, Thomas Rhys Evans, Martin R. Fellenz, Susann Fiedler, Mustafa Firat, Raquel Meister Ko. Freitag, Rémy A. Furrer, Richa Gautam, Dhruba Kumar Gautam, Brian Gearin, Stephan Gerschewski, Omid Ghasemi, Zohreh Ghasemi, Anindya Ghosh, Cinzia Giani, Matthew H. Goldberg, Manisha Goswami, Lorenz Graf‐Vlachy, Jennifer A. Griffith, Dmitry Grigoryev, Jingyang Gu, H Rajeshwari, Allègre L. Hadida, Andrew Hafenbrack, Sebastian Hafenbrädl, Jonathan Hammersley, Hyemin Han, Jason L. Harman, Andree Hartanto, Alexander P. Henkel, Yen-Chen Ho, Benjamin C. Holding, Felix Holzmeister, Alexandra Horobeţ, Tina Huang, Yiming Huang, Jeffrey R. Huntsinger, Katarzyna Idzikowska, Hirotaka Imada, Rabia Imran, Michael J. Ingels, Bastian Jaeger, Steve M. J. Janssen, Fanli Jia, Alfredo Jiménez, Jason Lu Jin, Niklas Johannes, Daniel Jolles, Bibiána Jozefiaková, Pavol Kačmár, Tamara Kalandadze, Kyriaki Kalimeri, Polly Kang, Jarosław Kantorowicz, Didar Karadağ, Hamid Karimi-Rouzbahani, Daisy Mui Hung Kee, Lucas Keller, Haider A. Khan, Mikael Knutsson, Olga Kombeiz, Aleksey Korniychuk, Marta Kowal, Johannes Leder, Liang Wenhao Liang, Taegyeong Liew, Fangwen Lin, Chengwei Liu, Bin Liu, Maria Cristina Longo, Andrey Lovakov, Mei Peng Low, Gerardus J. M. Lucas, Oliver Lukason, Albert L. Ly, Zegang Ma, Alexander Mafael, Elizabeth A. Mahar, Soheil Mahmoudkalayeh, David Manheim, Alfred Marcus, Melvin S. Marsh, Jolie M. Martin, Luis Martínez, Mario Martinoli, Marcel Martončik, Theodore C. Masters-Waage, Rui Mata, Hamid Mazloomi Khamseh, Randy J. McCarthy, Philip Millroth, Mahima Mishra, Supriti Mishra, Alexander T. Mohr, David Moreau, Annalisa Myer, Amos Nadler, Sudhir Nair, Gustav Nilsonne, Paweł Niszczota, Aoife O’Mahony, Marc Oberhauser, Tomasz Obłój, Mehmet Orhan, Flora Oswald, Tobias Otterbring, Philipp Otto, Ivar Padrón-Hernández, Alan J. Pan, Mariola Paruzel‐Czachura, Gerit Pfuhl, Angelo Pirrone, Simon Porcher, John Protzko, Constantin Prox, Shelly Qi, Rima-Maria Rahal, Md. Shahinoor Rahman, M. Reina, Satyanarayana Rentala, Zahid Riaz, Ivan Ropovik, Lukas Röseler, Robert M. Ross, Amanda Rotella, Leopold H. O. Roth, Thomas J. Roulet, Matthew M. Rubin, André Sammartino, Johann Sanchez, Adrian Saville, Michael Schaerer, Joyce Elena Schleu, Leo Schmallenbach, Landon Schnabel, Frederik Schulze Spüntrup, Birga M. Schumpe, Tony Senanayake, Raffaello Seri, Feng Sheng, Roary E. Snider, Di Song, Victoria Song, Sylwia E. Starnawska, Kai A. Stern, Samantha Stevens, Eirik Strømland, Wunhong Su, Hao Sun, Kevin P Sweeney, Reina Takamatsu, Maria Terskova, Kian Siong Tey, Warren Tierney, Mariya M. Todorova, Daniel Tolstoy, Lasse Torkkeli, Joshua M. Tybur, Francisco Valderrey, Ana María Vallina-Hernández, Ranjith P. Vasudevan, Gudivada Venkat Rao, Antoine Vernet, Tiia Vissak, Hinrich Voss, Thorsten Wahle, Jonathan Wai, Lauren E.T. Wakabayashi, Junnan Wang, Peng Wang, Yating Wang, Robert W. Warmenhoven, Karl Wennberg, Georg Wernicke, Jan K. Woike, Conny Wollbrant, Greg Woodin, Joshua D. Wright, Qiong Xia, Zhenzhen Xie, Sangsuk Yoon, Wenlong Yuan, Lin Yuan, Meltem Yucel, Zhao Zheng, Haibo Zhou, Cristina Zogmaister, Ro’i Zultan · 发表于：Proceedings of the National Academy of Sciences · 年份：2022 · DOI：10.1073/pnas.2120377119 · 被引用次数：57 · 研究领域：Data Quality and Management、Research Data Management Practices、Scientific Computing and Data Management

This initiative examined systematically the extent to which a large set of archival research findings generalizes across contexts. We repeated the key analyses for 29 original strategic management effects in the same context (direct reproduction) as well as in 52 novel time periods and geographies; 45% of the reproductions returned results matching the original reports together with 55% of tests in different spans of years and 40% of tests in novel geographies. Some original findings were associated with multiple new tests. Reproducibility was the best predictor of generalizability-for the findings that proved directly reproducible, 84% emerged in other available time periods and 57% emerged in other geographies. Overall, only limited empirical evidence emerged for context sensitivity. In a forecasting survey, independent scientists were able to anticipate which effects would find support in tests in new samples.