Advanced critical appraisal and EBM for psychiatry

Meet the paper

In 2008, Erick Turner and colleagues did something almost nobody had done: they stopped reading journals and went to the regulator's files. They pulled the FDA reviews for 74 registered trials of 12 antidepressants in 12,564 patients, then went looking for each one in the published literature.^[15]

What they found is the number you carry into every viva. Read the journals and 94% of those trials looked positive. Read the FDA's own analysis of the same trials and 51% were positive. Thirty-one percent of the studies — 3,449 participants' worth — had never been published at all, and of the negative or questionable studies, 22 vanished and 11 were published in a way that conveyed a positive result.^[15]

The evidence base had not lied. It had been curated. Pooled effect sizes from the journal literature ran 32% higher overall than from the complete FDA dataset, ranging from 11% to 69% inflation across individual drugs. Every technique on this page exists because of that gap.^[15][7]

^{[1] [5] [7] [8]}

References20ShowHide

1. [1]Sterne JAC, Savović J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials BMJ, 2019.PMID 31462531
2. [2]Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions BMJ, 2016.PMID 27733354
3. [3]Whiting PF, Rutjes AWS, Westwood ME, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies Ann Intern Med, 2011.PMID 22007046
4. [4]Higgins JP, Thompson SG, Deeks JJ, Altman DG Measuring inconsistency in meta-analyses BMJ, 2003.PMID 12958120
5. [5]Riley RD, Higgins JPT, Deeks JJ Interpretation of random effects meta-analyses BMJ, 2011.PMID 21310794
6. [6]Egger M, Davey Smith G, Schneider M, Minder C Bias in meta-analysis detected by a simple, graphical test BMJ, 1997.PMID 9310563
7. [7]Sterne JAC, Sutton AJ, Ioannidis JPA, et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials BMJ, 2011.PMID 21784880
8. [8]Guyatt GH, Oxman AD, Vist GE, et al; GRADE Working Group GRADE: an emerging consensus on rating quality of evidence and strength of recommendations BMJ, 2008.PMID 18436948
9. [9]Balshem H, Helfand M, Schünemann HJ, et al. GRADE guidelines: 3. Rating the quality of evidence J Clin Epidemiol, 2011.PMID 21208779
10. [10]Guyatt GH, Oxman AD, Kunz R, et al. Going from evidence to recommendations BMJ, 2008.PMID 18467413
11. [11]Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews BMJ, 2021.PMID 33782057
12. [12]Piaggio G, Elbourne DR, Pocock SJ, Evans SJW, Altman DG; CONSORT Group Reporting of noninferiority and equivalence randomized trials: extension of the CONSORT 2010 statement JAMA, 2012.PMID 23268518
13. [13]Mills EJ, Ioannidis JPA, Thorlund K, Schünemann HJ, Puhan MA, Guyatt GH How to use an article reporting a multiple treatment comparison meta-analysis JAMA, 2012.PMID 23011714
14. [14]Chaimani A, Salanti G, Leucht S, Geddes JR, Cipriani A Common pitfalls and mistakes in the set-up, analysis and interpretation of results in network meta-analysis: what clinicians should look for in a published article Evid Based Ment Health, 2017.PMID 28739577
15. [15]Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R Selective publication of antidepressant trials and its influence on apparent efficacy N Engl J Med, 2008.PMID 18199864
16. [16]Cipriani A, Furukawa TA, Salanti G, et al. Comparative efficacy and acceptability of 21 antidepressant drugs for the acute treatment of adults with major depressive disorder: a systematic review and network meta-analysis Lancet, 2018.PMID 29477251
17. [17]Sun X, Ioannidis JPA, Agoritsas T, Alba AC, Guyatt G How to use a subgroup analysis: users' guide to the medical literature JAMA, 2014.PMID 24449319
18. [18]Rothwell PM External validity of randomised controlled trials: "to whom do the results of this trial apply?" Lancet, 2005.PMID 15639683
19. [19]Schulz KF, Chalmers I, Hayes RJ, Altman DG Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials JAMA, 1995.PMID 7823387
20. [20]Thompson SG, Higgins JPT How should meta-regression analyses be undertaken and interpreted? Stat Med, 2002.PMID 12111920