Critical appraisal of a trial abstract — PICO, validity, effect size and applicability

(a) PICO and internal validity (5 marks).

State the PICO explicitly before appraising.^[2][1]

• Population — adult emergency department patients with unexplained syncope (1200 patients, 14 centres).
• Intervention — management strategy guided by the discharge-risk score.
• Comparator — standard management.
• Outcome — 30-day serious adverse event (a patient-centred, clinically important endpoint, not a surrogate).

Strengths of internal validity. Computer-generated randomisation with concealed allocation distributes known and unknown confounders equally between groups and prevents selection bias — allocation concealment is the single most important protectant of trial validity.^[5] Blinded outcome adjudication removes ascertainment bias for the subjective composite endpoint. Follow-up is near-complete at 99 per cent, so loss-to-follow-up bias is negligible. The analysis is by intention to treat, preserving the randomisation and avoiding the bias of analysing only those who complied.

Threats to internal validity. Clinicians and patients were unblinded (an open-label trial), so performance bias is possible — clinicians may manage score-guided patients more intensively regardless of the score's own contribution, inflating the apparent benefit. The trial is funded by the score's manufacturer, an industry-funding conflict of interest that biases protocol design, analysis and reporting toward a favourable result; the candidate must check for trial registration, a published protocol and a pre-specified primary outcome. Apply the Cochrane RoB 2 tool across its five domains — randomisation, deviations from intended interventions, missing outcome data, measurement of the outcome, and selection of the reported result — to reach an overall low, some-concerns, or high risk of bias.^[5]

(b) Measures of effect, confidence interval and p value (5 marks).^[3]

• Absolute risk reduction (ARR) = control event rate minus experimental event rate = 0.10 minus 0.06 = 0.04, or 4 per cent.
• Relative risk reduction (RRR) = ARR divided by CER = 0.04 / 0.10 = 0.40, or 40 per cent. Note the inflation: a 40 per cent relative reduction sounds transformative, but it is a 4 per cent absolute reduction.
• Number needed to treat (NNT) = 1 / ARR = 1 / 0.04 = 25. Twenty-five patients must be managed with the score-guided strategy for one additional 30-day serious adverse event to be prevented.

Confidence interval. The 95 per cent confidence interval for the relative risk is 0.41 to 0.89. It does not cross unity (1), so the result is statistically significant and consistent with a real effect; the plausible benefit ranges from an 11 per cent to a 59 per cent relative reduction. The interval is moderately narrow, reflecting adequate precision from 1200 patients.^[3]

p value. The p value of 0.009 is the probability of observing data this extreme if the null hypothesis of no effect were true — it is not the probability that the null is true, and it does not measure the size or importance of the effect. Because it is below 0.05 the result is statistically significant, but clinical significance rests on the absolute risk reduction and the NNT, not on the p value alone.^[1][3]

(c) Applicability — the three questions (3 marks).

Applicability is the step most often skipped; complete it before adopting the score.^[1]

1. Does my patient resemble those in the trial? Compare the 78-year-old patient to the trial population — age range, comorbidities, the definition of unexplained syncope used, the baseline SAE rate. If the elderly patient was under-represented or the trial excluded heart failure and pacing, the result may not transfer.
2. Is the outcome one the patient cares about? A 30-day serious adverse event is patient-centred (death, arrhythmia, haemorrhage, return for a serious diagnosis), not a surrogate, so this criterion is satisfied.
3. Are the benefits worth the harms, costs and burden, and does the patient agree? Weigh the NNT of 25 against any cost, the risk of over-investigation generated by the score, the bed-day implications of additional admissions, and the patient's values. A shared decision completes the transfer.

(d) Reporting standard, risk-of-bias tool and GRADE rating (2 marks).

The trial is a parallel-group randomised controlled trial and should be reported under CONSORT 2010.^[4] Apply the Cochrane RoB 2 tool to assess its internal validity domain by domain.^[5] Under GRADE, a body of randomised-trial evidence begins as high quality and is downgraded here for the open-label design (risk of bias from lack of clinician blinding) and for the manufacturer funding and probable publication-bias risk; the likely rating is moderate, supporting a conditional (weak) recommendation to adopt the score pending independent replication and a cost-effectiveness analysis.^[6]