When are randomised trials unnecessary? Picking signal from noise
A BMJ analysis paper arguing that some treatment effects are so large that bias can be ruled out without randomisation, and proposing a way to tell which. The method is a signal to noise ratio: the effect of treatment set against the expected course of the condition without it, expressed as a rate ratio. The worked examples run from the mother's kiss technique for a bead lodged in a child's nostril, where the ratio is on the order of a thousand, to laser treatment of portwine stains at around eighty and laparoscopic fundoplication for reflux at around twenty. The proposed rule of thumb is a rate ratio beyond ten together with a conventional probability below 0.01, on the grounds that simulation and empirical work suggest confounding rarely manufactures associations that large. The paper is careful about its own limits, and those limits matter here. It classifies prognosis from most to least predictable, as stable, progressive, spontaneously remitting, fluctuating, episodic and probabilistic, and argues that dramatic effects are demonstrable at the predictable end while randomisation is usually still required at the other. It offers hormone replacement therapy and beta carotene as warnings, cases where non-randomised evidence misled precisely because the signal to noise ratio was modest. This is the paper behind the dramatic effect argument in the Parachutes and Evidence article, and it supports the two halves of that argument unequally. Falling from altitude without a parachute is a stable, rapid and near-certain outcome, which is the quadrant the criterion was built for. Organisational psychological safety sits closer to the probabilistic end, a possible future event with a fluctuating course, which is where Glasziou and colleagues judge the approach weakest. The honest reading is that the catastrophe record is a mechanistic and forensic inference rather than a rate ratio: the paper licenses scepticism about the demand for a trial, without licensing the claim that the organisational evidence is already dramatic in their technical sense.
Neighbours (8)
- Complexity and Postmodernism: Understanding Complex Systems
- Methodological Fit in Management Field Research
- Thinking About Process Safety Indicators
- Parachutes & Evidence
- Qualitative Measurement of PS
- Studying Complexity in Health Services Research: Desperately Seeking an Overdue Paradigm Shift
- PS & Science
- Social Science vs Epidemiology