Here is a short question. What would make you drop a belief you hold? Most people answer fast. They say “good evidence.” They say “real proof.” Neither answer names a single thing. A real answer names something that could actually happen to you.
That is a harder question than it sounds. It is also the only honest test of an open mind. Not “have you heard the other side.” This one.
Sources
Karl Popper, “Science: Conjectures and Refutations,” in Conjectures and Refutations: The Growth of Scientific Knowledge (1963). VERIFIED — PDF downloaded and all six quoted passages string-matched verbatim, including Popper's unhyphenated “nonscientific”. Also confirms the 1919 Adler anecdote, the seven conclusions, the Einstein/eclipse passage (“light must be attracted by heavy bodies”), and the non-scientific-is-not-meaningless caveat. http://www.dpi.inpe.br/gilberto/cursos/cst-311/popper_conjectures_refutations.pdf
Karl Popper's 1919 work with Adler: Popper cooperated in Adler's social work with children and young people in the working-class districts of Vienna, volunteering at a clinic — he did not run it. Corrected in the module accordingly. Stanford Encyclopedia of Philosophy, “Karl Popper.” https://plato.stanford.edu/entries/popper/
Sherman Kent, “Words of Estimative Probability,” Studies in Intelligence, Central Intelligence Agency, 1964. VERIFIED — PDF downloaded; the quoted sentence matches word for word. Confirms NIE 29-51 (March 1951, “Probability of an Invasion of Yugoslavia in 1951”), Kent's own 65-to-35, the 20-to-80 / 80-to-20 spread, and that the questioner was the State Department Policy Planning Staff's chairman. Also establishes that the estimate was drafted in the Office of National Estimates, worked over by ~20 experts, presided over by the Board of National Estimates and approved by the IAC — Kent was one Board member, not the author. https://www.cia.gov/resources/csi/static/Words-of-Estimative-Probability.pdf
Jeffrey A. Friedman, Joshua D. Baker, Barbara A. Mellers, Philip E. Tetlock & Richard Zeckhauser, “The Value of Precision in Probability Assessment: Evidence from a Large-Scale Geopolitical Forecasting Tournament,” International Studies Quarterly 62(2), 2018, 410–422. VERIFIED — 888,328 forecasts; coarsening numeric probabilities into qualitative expressions consistently sacrifices accuracy, and the finding does not depend on extreme estimates, short horizons, scoring rule or individual attributes. https://academic.oup.com/isq/article-abstract/62/2/410/4944059
Pavel Atanasov, Jens Witkowski, Lyle Ungar, Barbara Mellers & Philip Tetlock, “Small steps to accuracy: Incremental belief updaters are better forecasters,” Organizational Behavior and Human Decision Processes 160, 2020, 19–35. VERIFIED — over 400,000 forecasts on almost 500 geopolitical questions from a four-year tournament; the most accurate forecasters make frequent small updates, low-skill forecasters make infrequent large revisions or simply restate their prior forecast. https://www.sciencedirect.com/science/article/abs/pii/S0749597819300949
Anne M. Scheel, Mitchell R. M. J. Schijen & Daniël Lakens, “An Excess of Positive Results: Comparing the Standard Psychology Literature With Registered Reports,” Advances in Methods and Practices in Psychological Science 4(2), 2021. VERIFIED — 146 of 152 standard reports (96.05%) vs 31 of 71 Registered Reports (43.66%) positive on the first hypothesis. CAVEATS NOW CARRIED IN THE MODULE: the authors state “Our study was not an experiment—hypotheses, authors, and editors were not randomly assigned to each publication format—and thus precludes strong causal inferences,” and replications were 41/71 (57.75%) of RRs against 4/152 (2.63%) of SRs. https://journals.sagepub.com/doi/10.1177/25152459211007467
Robert M. Kaplan & Veronica L. Irvin, “Likelihood of Null Effects of Large NHLBI Clinical Trials Has Increased over Time,” PLOS ONE, 2015. VERIFIED — 55 large NHLBI cardiovascular trials 1970–2012; 17 of 30 before 2000 vs 2 of 25 after 2000 reported significant benefit. Authors' own caveat, used in sense in the module's info_pop: they cannot claim causation, the sample is small, and the findings are limited to large NHLBI-funded trials on cardiovascular outcomes in adults. https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0132382
Ben Goldacre et al., “COMPare: a prospective cohort study correcting and monitoring 58 misreported trials in real time,” Trials, 2019. VERIFIED — 67 trials assessed between 19 October and 30 November 2015 across five CONSORT-endorsing journals (NEJM, Lancet, JAMA, BMJ, Annals of Internal Medicine); 58 had discrepancies requiring a correction letter (87%); 365 undeclared novel outcomes, a mean of 5.4 per trial. https://pmc.ncbi.nlm.nih.gov/articles/PMC6375128/
Lee Ross, Mark R. Lepper & Michael Hubbard, “Perseverance in self-perception and social perception: Biased attributional processes in the debriefing paradigm,” Journal of Personality and Social Psychology 32, 1975, 880–892. VERIFIED — two experiments; participants received predetermined, randomly assigned false success/failure feedback and were then fully debriefed about its false and random nature; self-perceptions persevered. NOTE: the study tested debriefing, not conditions named in advance — the module's reveal was corrected to stop attributing a mechanism the paper never examined. https://www.scirp.org/reference/referencespapers?referenceid=2075083
Justin P. Friesen, Troy H. Campbell & Aaron C. Kay, “The psychological advantage of unfalsifiability: The appeal of untestable religious and political ideologies,” Journal of Personality and Social Psychology 108(3), 2015, 515–529. VERIFIED — Experiment 4: when facts threaten their worldview, partisans construe political issues as more unfalsifiable (“moral opinion”) rather than falsifiable (“a matter of facts”); unfalsifiability also lets adherents hold beliefs with more conviction and criticise opponents more extremely. https://pubmed.ncbi.nlm.nih.gov/25402678/
Donald A. Redelmeier & Amos Tversky, “On the belief that arthritis pain is related to the weather,” Proceedings of the National Academy of Sciences 93(7), 1996, 2895–2896. VERIFIED — 18 patients followed for more than a year, no statistically significant association between pain and the weather conditions each individual had implicated; separately, 97 students perceived correlations between uncorrelated random sequences. https://pmc.ncbi.nlm.nih.gov/articles/PMC39730/
William G. Dixon et al., “How the weather affects the pain of citizen scientists using a smartphone app,” npj Digital Medicine, 2019. VERIFIED — 13,207 downloaded the app, 2,658 analysed, 15 months (February 2016 – April 2017) across the UK; higher relative humidity, lower atmospheric pressure and higher wind speed were associated with pain, temperature and rainfall were not. The authors' own wording: the effect sizes, while statistically significant, were “modest.” https://pmc.ncbi.nlm.nih.gov/articles/PMC6811599/