Phys · general-medicine
Evidence-Based Medicine and Critical Appraisal
Also known as evidence-based medicine · EBM · critical appraisal · CASP · CONSORT · STROBE · PRISMA · QUADAS-2 · GRADE · PICO · number needed to treat · NNT · number needed to harm · NNH · absolute risk reduction · ARR · relative risk · odds ratio · confidence interval · intention-to-treat · forest plot · I-squared · heterogeneity · publication bias · funnel plot · Type I error · Type II error · statistical power · hierarchy of evidence · external validity · internal validity
Consultant-physician-depth guide to evidence-based medicine and critical appraisal for FRACP DWE and DCE — the Sackett definition of EBM (integration of best research evidence, clinical expertise and patient values), the hierarchy of evidence (case reports through to systematic reviews and meta-analyses), the study designs (cross-sectional for prevalence, case-control for rare diseases reporting odds ratios and prone to recall bias, cohort for common diseases reporting relative risk and establishing temporality, RCT as the gold standard for therapy with randomisation, allocation concealment, blinding and intention-to-treat analysis), the critical-appraisal frameworks (CASP checklists, CONSORT for RCTs, STROBE for observational studies, PRISMA for systematic reviews, QUADAS-2 for diagnostic accuracy), the PICO framework, effect sizes (relative risk, odds ratio, absolute risk reduction, NNT, NNH), confidence intervals (95 per cent CI, crossing 1 for a ratio means non-significant), p-values and the distinction between statistical and clinical significance, Type I and Type II errors and power, intention-to-treat versus per-protocol analysis, subgroup analyses and the hazard of multiple comparisons, the bias types (selection, information or recall, publication detected by funnel plot, confounding, lead-time, length-time, spectrum), internal versus external validity, forest plots and meta-analysis with I-squared heterogeneity and fixed versus random effects, applying evidence to the individual patient, and the GRADE system for guideline development (high, moderate, low, very low quality; strong versus weak recommendations).
On this page
Study tools
Your progress
Saved on this device.
Practise this topic
Target exams
Red flags
- A 95 per cent confidence interval that crosses 1 for a ratio measure (RR, OR, HR) means the result is NOT statistically significant at the 5 per cent level — never claim benefit from the point estimate alone when the interval includes the null
- A relative risk reduction of 50 per cent in a rare event may be a clinically trivial absolute reduction — always compute the ARR and the NNT for the patient's baseline risk before claiming a meaningful benefit
- A p-value below 0.05 in a very large trial does NOT mean the effect is clinically important — statistical significance is a property of the sample size and the variance, not the size of the effect; clinical significance requires the effect to be large enough to matter to a patient
- An unplanned, post-hoc subgroup finding is hypothesis-generating only — with enough comparisons some subgroups will differ by chance; treat a subgroup result as definitive only if it was pre-specified, biologically plausible, supported by a significant interaction test and replicated
- A patient who would have been excluded from the pivotal trial (elderly, renally impaired, pregnant, multi-morbid) may not share the trial's relative effect — judge external validity explicitly and do not extrapolate uncritically
- A meta-analysis of biased studies is a biased pooled estimate — high I-squared (above 50 per cent) or funnel-plot asymmetry means the pooled result may be misleading; assess the quality of the included studies before accepting the pooled estimate
Meet the patient
A 78-year-old woman with heart failure and stage 3 chronic kidney disease is offered a new tablet. The registrar quotes a "50 per cent reduction in events" from a trial whose average patient was 62 with normal renal function. The ward-round question is the question at the heart of this topic: does that result apply to her, and what is her real benefit?[1]
Hold two questions and the whole topic slots into place. Does the trial answer the question I am asking (validity and PICO)? What is the size and precision of the effect, and does it apply to this patient (importance and applicability)?[1]
References8ShowHide
- [1]Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS Evidence based medicine: what it is and what it isn't BMJ, 1996.PMID 8555924
- [2]Guyatt GH, Oxman AD, Vist GE, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations BMJ, 2008.PMID 18436948
- [3]Schulz KF, Altman DG, Moher D CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials BMJ, 2010.PMID 20332509
- [4]Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement PLoS Med, 2009.PMID 19621072
- [5]Whiting PF, Rutjes AW, Westwood ME, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies Ann Intern Med, 2011.PMID 22007046
- [6]von Elm E, Altman DG, Egger M, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies Lancet, 2007.PMID 18064739
- [7]Higgins JP, Thompson SG, Deeks JJ, Altman DG Measuring inconsistency in meta-analyses BMJ, 2003.PMID 12958120
- [8]Fergusson D, Aaron SD, Guyatt G, et al. Post-randomisation exclusions: the intention to treat principle and excluding patients from analysis BMJ, 2002.PMID 12242181