Someone says a study proved it. Maybe a friend. Maybe a headline you scrolled past. You will not read that study. Almost nobody does.
Many papers sit behind a paywall. They are written for other scientists. Even the free ones are hard going.
So you need something else. You need better questions. Good questions work from outside the paper. That is what this module hands you.
Sources
Petrosino A, Turpin-Petrosino C, Hollis-Peel ME, Lavenberg JG. "'Scared Straight' and other juvenile awareness programs for preventing juvenile delinquency." Cochrane Database of Systematic Reviews, 2013. Nine randomised trials, United States, 1967-1992, 946 participants. Verified verbatim: "We conclude that programs such as 'Scared Straight' increase delinquency relative to doing nothing at all to similar youths." https://pubmed.ncbi.nlm.nih.gov/23862186/
Academy of Motion Picture Arts and Sciences, Awards Database — "Scared Straight!" (1978), Best Documentary Feature. Backs the Academy Award claim; the programme itself began at Rahway State Prison, New Jersey, in 1976. https://awardsdatabase.oscars.org/
Peto R, Doll R, Buckley JD, Sporn MB. "Can dietary beta-carotene materially reduce human cancer rates?" Nature, 1981. Four authors; called for intervention trials. https://www.nature.com/articles/290201a0
The Alpha-Tocopherol, Beta Carotene Cancer Prevention Study Group. "The effect of vitamin E and beta carotene on the incidence of lung cancer and other cancers in male smokers." New England Journal of Medicine, 1994. 29,133 male smokers, Finland, aged 50-69, followed 5-8 years; beta carotene group had 18% higher lung cancer incidence and 8% higher total mortality. https://pubmed.ncbi.nlm.nih.gov/8127329/
Omenn GS et al. "Effects of a combination of beta carotene and vitamin A on lung cancer and cardiovascular disease" (the CARET trial). New England Journal of Medicine, 1996. 18,314 smokers, former smokers and asbestos-exposed workers; stopped 21 months early; lung cancer RR 1.28, total mortality RR 1.17. https://pubmed.ncbi.nlm.nih.gov/8602180/
Open Science Collaboration. "Estimating the reproducibility of psychological science." Science, 2015. Verified verbatim: "Ninety-seven percent of original studies had statistically significant results" and "Thirty-six percent of replications had statistically significant results"; replication effects were half the magnitude of original effects. Sampled from three psychology journals. https://pubmed.ncbi.nlm.nih.gov/26315443/
Camerer CF et al. "Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015." Nature Human Behaviour, 2018. 21 experiments; significant effect in the same direction for 13 (62%); replication effect sizes on average about 50% of the originals. https://www.nature.com/articles/s41562-018-0399-z
Errington TM, Mathur M, Soderberg CK et al. "Investigating the replicability of preclinical cancer biology." eLife, 2021. Project ran roughly eight years; of the effects that could be compared, "92% of replication effect sizes were smaller than the original", with the median 85% smaller for positive effects. https://elifesciences.org/articles/71601
Sumner P et al. "The association between exaggeration in health related science news and academic press releases: retrospective observational study." BMJ, 2014. 462 press releases from 20 leading UK universities in 2011 plus 668 news stories; 33% of releases contained exaggerated causal claims; where the release exaggerated, 81% of news stories did, against 18% where it did not. https://pubmed.ncbi.nlm.nih.gov/25498121/
Rosenthal R. "The 'file drawer problem' and tolerance for null results." Psychological Bulletin, 1979. https://pages.ucsd.edu/~cmckenzie/Rosenthal1979PsychBulletin.pdf
Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. "Selective publication of antidepressant trials and its influence on apparent efficacy." New England Journal of Medicine, 2008. 74 FDA-registered trials; 38 judged positive; of the 36 not positive, 22 were never published, 11 were published in a way that conveyed a positive outcome, 3 were published as not positive. Verified verbatim: "According to the published literature, it appeared that 94% of the trials conducted were positive. By contrast, the FDA analysis showed that 51% were positive." https://pubmed.ncbi.nlm.nih.gov/18199864/
International Committee of Medical Journal Editors. "Clinical Trial Registration" (ICMJE Recommendations, current). Verified verbatim, current wording: registration "at or before the time of first patient consent for enrollment as a condition of consideration for publication". Note: the 2004 statement's wording was "at or before the onset of patient enrollment". https://www.icmje.org/recommendations/browse/publishing-and-editorial-issues/clinical-trial-registration.html
De Angelis C, Drazen JM, Frizelle FA et al. "Clinical trial registration: a statement from the International Committee of Medical Journal Editors." New England Journal of Medicine, 2004. Source for the 1 July 2005 enrolment cut-off. https://www.nejm.org/doi/full/10.1056/NEJMe048225
Kaplan RM, Irvin VL. "Likelihood of null effects of large NHLBI clinical trials has increased over time." PLOS ONE, 2015. Verified verbatim: "17 of 30 studies (57%) published prior to 2000 showed a significant benefit of intervention on the primary outcome in comparison to only 2 among the 25 (8%) trials published after 2000". The authors state they "cannot say that trend toward null trials to preregistration in ClinicalTrials.gov is causal". https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0132382
Scheel AM, Schijen MRMJ, Lakens D. "An excess of positive results: comparing the standard psychology literature with Registered Reports." Advances in Methods and Practices in Psychological Science, 2021. 96.05% of 152 standard reports had positive results against 43.66% of 71 Registered Reports; the authors note the two sets are not perfectly matched. https://journals.sagepub.com/doi/10.1177/25152459211007467
Goldacre B, Drysdale H, Dale A et al. "COMPare: a prospective cohort study correcting and monitoring 58 misreported trials in real time." Trials, 2019. 67 trials across five journals; a mean of 5.4 undeclared novel outcomes per trial; 58 trials required a correction letter and 23 of the 58 letters (40%) were published. https://pmc.ncbi.nlm.nih.gov/articles/PMC6375128/
Lundh A, Lexchin J, Mintzes B, Schroll JB, Bero L. "Industry sponsorship and research outcome." Cochrane Database of Systematic Reviews, 2017. 75 included studies of drug and device trials; industry sponsorship associated with more favourable efficacy results and conclusions; the authors conclude the industry bias "cannot be explained by standard 'Risk of bias' assessments". https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.MR000033.pub3/full
Bes-Rastrollo M, Schulze MB, Ruiz-Canela M, Martinez-Gonzalez MA. "Financial conflicts of interest and reporting bias regarding the association between sugar-sweetened beverages and weight gain: a systematic review of systematic reviews." PLOS Medicine, 2013. Verified verbatim: "Among those reviews without any reported conflict of interest, 83.3% of the conclusions (10/12) were that SSB consumption could be a potential risk factor for weight gain. In contrast, the same percentage of conclusions, 83.3% (5/6), of those SRs disclosing some financial conflict of interest with the food industry were that the scientific evidence was insufficient to support a positive association." https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1001578
Center for Open Science. "Registered Reports" (definition and adopting journals). https://www.cos.io/initiatives/registered-reports
U.S. National Library of Medicine. ClinicalTrials.gov — the public trial register named in the module; opened in 2000, which is the split point the NHLBI analysis uses. https://clinicaltrials.gov/
World Health Organization. International Clinical Trials Registry Platform (ICTRP) — searches across national registries. https://trialsearch.who.int/