Paediatric study design and bias

Overview & Definition

A parent brings you a headline claiming a new therapy dramatically improves a childhood condition, and you must decide whether to believe it. The headline rarely names the study design, yet the design is the single largest determinant of whether the claim can be trusted. A dramatic result from a single small case series means something very different from the same result reproduced in a large blinded randomised trial, because the designs offer different levels of protection against being misled. [1] [2]

A study design is the architecture of a research project. It decides who is studied, when the exposure and the outcome are measured relative to one another, and how the comparison between groups is protected from extraneous influences. The design is chosen to suit the question: a therapy question asks whether an intervention changes an outcome, a harm question asks whether an exposure causes harm, a prognosis question asks what happens to a defined group over time, and a descriptive question asks how common something is. Each question has a design that answers it best, and using the wrong design does not just weaken the answer, it can produce a wrong one. [2] [8]

Bias is the reason design matters. A bias is any systematic error that distorts the comparison between groups, as distinct from random error, which chance produces and which larger samples reduce. The three biases that threaten every study are selection bias, information bias, and confounding, and each has a design manoeuvre that resists it. The hierarchy of evidence is really a hierarchy of protection against these biases, climbing from designs that control none of them to designs that control all three. This page owns the choice and appraisal of study designs and the recognition and prevention of bias; the computation of effect estimates belongs to the clinical epidemiology leaf, and the appraisal workflow belongs to the evidence-based medicine leaf. [6] [11]

References15Show ledgerHide ledger

1. [1]Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS Evidence based medicine: what it is and what it isn't BMJ, 1996.PMID 8555924
2. [2]Grimes DA, Schulz KF An overview of clinical research: the lay of the land Lancet, 2002.PMID 11809203
3. [3]Grimes DA, Schulz KF Descriptive studies: what they can and cannot do Lancet, 2002.PMID 11809274
4. [4]Grimes DA, Schulz KF Cohort studies: marching towards outcomes Lancet, 2002.PMID 11830217
5. [5]Schulz KF, Grimes DA Case-control studies: research in reverse Lancet, 2002.PMID 11844534
6. [6]Grimes DA, Schulz KF Bias and causal associations in observational research Lancet, 2002.PMID 11812579
7. [7]Schulz KF, Chalmers I, Hayes RJ, Altman DG Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials JAMA, 1995.PMID 7823387
8. [8]Concato J, Shah N, Horwitz RI Randomized, controlled trials, observational studies, and the hierarchy of research designs N Engl J Med, 2000.PMID 10861325
9. [9]Ioannidis JP Contradicted and initially stronger effects in highly cited clinical research JAMA, 2005.PMID 16014596
10. [10]Rothwell PM External validity of randomised controlled trials: to whom do the results of this trial apply? Lancet, 2005.PMID 15639683
11. [11]Vandenbroucke JP Observational research, randomised trials, and two views of medical science PLoS Med, 2008.PMID 18336067
12. [12]Moher D, Hopewell S, Schulz KF, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials BMJ, 2010.PMID 20332511
13. [13]Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions BMJ, 2016.PMID 27733354
14. [14]Sterne JAC, Savović J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials BMJ, 2019.PMID 31462531
15. [15]Sedgwick P Understanding the ecological fallacy BMJ, 2015.PMID 26391012