Anaes · Measurement & monitoring physics

Hypothesis testing, p-values, confidence intervals and error

Also known as Hypothesis testing · p-values · Confidence intervals · Type I and Type II error · Statistical power · Sample size · Multiple testing · Effect size

Every anaesthetic trial asks whether a difference between two groups is real or the product of chance, and the framework that answers that question is hypothesis testing. The model rests on seven exam-critical ideas. First, the NULL HYPOTHESIS (H0) assumes no difference between groups and is presumed true until the evidence is strong enough to reject it; the ALTERNATIVE HYPOTHESIS (H1) asserts that a difference exists. Second, a TYPE I ERROR (alpha) is a FALSE POSITIVE — rejecting H0 when it is true — conventionally set at 0.05, and the p-value is the probability of the observed data or more extreme data IF H0 is true. Third, a TYPE II ERROR (beta) is a FALSE NEGATIVE — failing to reject H0 when it is false — conventionally set at 0.20. Fourth, POWER is 1 minus beta, the probability of correctly detecting a true difference, conventionally 0.80, and it rises with sample size, effect size and alpha and falls with variability; an underpowered study wastes resources and manufactures false negatives. Fifth, the p-value is widely misused — it is NOT the probability that H0 is true, a p less than 0.05 does not guarantee clinical importance, and multiple comparisons inflate the Type I error rate, corrected by the BONFERRONI method (divide alpha by the number of tests). Sixth, a CONFIDENCE INTERVAL is a range that contains the true population parameter with a specified probability (typically 95 percent), conveying both the effect size and its precision; if the CI crosses 1.0 for a ratio or 0 for a difference the result is not statistically significant. Seventh, the EFFECT SIZE (number needed to treat equals 1 divided by the absolute risk reduction; odds ratio and relative risk as measures of association) tells you HOW BIG a difference is, complementing the p-value which tells you only IF a difference exists; parametric tests (t-test, ANOVA) suit normal data, non-parametric tests (Mann-Whitney U, Kruskal-Wallis) suit non-normal data, and chi-squared suits categorical data; survival analysis uses Kaplan-Meier curves, the log-rank test and the hazard ratio for time-to-event outcomes such as time to recovery or time to first analgesic. Built on the Greenland et al. guide to misinterpretations of tests, p-values, confidence intervals and power, the Altman-Gardner confidence-interval paper, the Bender-Lange multiple-testing paper, the Cook-Sackett number-needed-to-treat paper and the Bland-Altman survival papers (Kaplan-Meier, logrank, time-to-event), with applied examples from the emergence-hypertension cohort (Jiao 2026), the hip-arthroplasty cohort (Ali 2026) and the neurological-deterioration prediction review (Chen 2026).