The preregistration revolution
- PMID: 29531091
- PMCID: PMC5856500
- DOI: 10.1073/pnas.1708274114
The preregistration revolution
Abstract
Progress in science relies in part on generating hypotheses with existing observations and testing hypotheses with new observations. This distinction between postdiction and prediction is appreciated conceptually but is not respected in practice. Mistaking generation of postdictions with testing of predictions reduces the credibility of research findings. However, ordinary biases in human reasoning, such as hindsight bias, make it hard to avoid this mistake. An effective solution is to define the research questions and analysis plan before observing the research outcomes-a process called preregistration. Preregistration distinguishes analyses and outcomes that result from predictions from those that result from postdictions. A variety of practical strategies are available to make the best possible use of preregistration in circumstances that fall short of the ideal application, such as when the data are preexisting. Services are now available for preregistration across all disciplines, facilitating a rapid increase in the practice. Widespread adoption of preregistration will increase distinctiveness between hypothesis generation and hypothesis testing and will improve the credibility of research findings.
Keywords: confirmatory analysis; exploratory analysis; methodology; open science; preregistration.
Conflict of interest statement
Conflict of interest statement: B.A.N., A.C.D., and D.T.M. are employed by the nonprofit Center for Open Science that has as its mission to increase openness, integrity, and reproducibility of research.
Comment in
-
Reply to Ledgerwood: Predictions without analysis plans are inert.Proc Natl Acad Sci U S A. 2018 Nov 6;115(45):E10518. doi: 10.1073/pnas.1816418115. Epub 2018 Oct 19. Proc Natl Acad Sci U S A. 2018. PMID: 30341224 Free PMC article. No abstract available.
-
The preregistration revolution needs to distinguish between predictions and analyses.Proc Natl Acad Sci U S A. 2018 Nov 6;115(45):E10516-E10517. doi: 10.1073/pnas.1812592115. Epub 2018 Oct 19. Proc Natl Acad Sci U S A. 2018. PMID: 30341225 Free PMC article. No abstract available.
Similar articles
-
Preregistering qualitative research.Account Res. 2019 Apr;26(3):229-244. doi: 10.1080/08989621.2019.1580147. Epub 2019 Mar 1. Account Res. 2019. PMID: 30741570
-
Scientific progress despite irreproducibility: A seeming paradox.Proc Natl Acad Sci U S A. 2018 Mar 13;115(11):2632-2639. doi: 10.1073/pnas.1711786114. Proc Natl Acad Sci U S A. 2018. PMID: 29531095 Free PMC article. Review.
-
Preregistration of Analyses of Preexisting Data.Psychol Belg. 2019 Aug 22;59(1):338-352. doi: 10.5334/pb.493. Psychol Belg. 2019. PMID: 31497308 Free PMC article. Review.
-
Facilitating open science practices for research syntheses: PreregRS guides preregistration.Res Synth Methods. 2022 Mar;13(2):284-289. doi: 10.1002/jrsm.1540. Epub 2022 Feb 2. Res Synth Methods. 2022. PMID: 34921744
-
Improving evidence-based practice through preregistration of applied research: Barriers and recommendations.Account Res. 2023 Feb;30(2):88-108. doi: 10.1080/08989621.2021.1969233. Epub 2021 Aug 31. Account Res. 2023. PMID: 34396837 Review.
Cited by
-
Evaluating variable selection methods for multivariable regression models: A simulation study protocol.PLoS One. 2024 Aug 9;19(8):e0308543. doi: 10.1371/journal.pone.0308543. eCollection 2024. PLoS One. 2024. PMID: 39121055 Free PMC article.
-
CASCADE: A Community-Engaged Action Model for Generating Rapid, Patient-Engaged Decisions in Clinical Research.Res Sq [Preprint]. 2024 Aug 27:rs.3.rs-4790564. doi: 10.21203/rs.3.rs-4790564/v1. Res Sq. 2024. PMID: 39257986 Free PMC article. Preprint.
-
Heterogeneity in effect size estimates.Proc Natl Acad Sci U S A. 2024 Aug 6;121(32):e2403490121. doi: 10.1073/pnas.2403490121. Epub 2024 Jul 30. Proc Natl Acad Sci U S A. 2024. PMID: 39078672 Free PMC article.
-
Recommendations in pre-registrations and internal review board proposals promote formal power analyses but do not increase sample size.PLoS One. 2020 Jul 31;15(7):e0236079. doi: 10.1371/journal.pone.0236079. eCollection 2020. PLoS One. 2020. PMID: 32735597 Free PMC article.
-
Simulation Tests of Methods in Evolution, Ecology, and Systematics: Pitfalls, Progress, and Principles.Annu Rev Ecol Evol Syst. 2022 Nov;53(1):113-136. doi: 10.1146/annurev-ecolsys-102320-093722. Epub 2022 Jul 29. Annu Rev Ecol Evol Syst. 2022. PMID: 38107485 Free PMC article.
References
-
- Box GEP. Science and statistics. J Am Stat Assoc. 1976;71:791–799.
-
- Box GEP. Robustness in the strategy of scientific model building. In: Launer RL, Wilkinson GN, editors. Robustness in Statistics. Academic; New York: 1979. pp. 201–236.
-
- de Groot AD. The meaning of “significance” for different types of research. Acta Psychol (Amst) 2014;148:188–194. - PubMed
-
- Hoyningen-Huene P. Context of discovery and context of justification. Stud Hist Philos Sci. 1987;18:501–515.
-
- Kuhn TS. Logic of discovery or psychology of research? In: Lakatos I, Musgrave A, editors. Criticism and the Growth of Knowledge. Cambridge Univ Press; Cambridge, UK: 1970. pp. 1–23.
Publication types
MeSH terms
Grants and funding
LinkOut - more resources
Full Text Sources
Other Literature Sources