Statistically significant differences versus convincing evidence of real treatment effects: an analysis of the false positive risk for single-centre trials in anaesthesia
Authors' conclusion
Our findings suggest that a substantial proportion of single-centre trials in anaesthesia reporting statistically significant differences provide limited evidence of real treatment effects, or, alternatively, required an implausibly high prior belief in a real treatment effect.
Study context
- Population
- Single-centre RCTs in 10 anaesthesia journals over 6 yr with a statistically significant difference in a primary binary outcome
- Comparison
- False positive risk (FPR) calculated by the Gunel vs Kass Bayes factor methods
- Outcome
- False positive risk (FPR) for equal pretest probabilities of 0.5
Methodological detailssystematic review of single-centre randomised trials, Bayesian analysis, false positive risk
- Systematic review of single-centre randomised trials
- Bayesian analysis
- False positive risk
Pooled result
As reported in the abstract
FPR, equal pretest probabilities of 0.5
Further pooled results
Median
| Outcome | Gunel | Kass |
|---|---|---|
| FPR in trials reporting P-values 0.01-0.05 (subgroup), equal pretest probabilities of 0.5 | 25% (IQR 16–30%) | 20% (IQR 16–25%) |
- FPR in trials reporting P-values 0.01-0.05 (subgroup), equal pretest probabilities of 0.5 — Trials reporting P-values 0.01-0.05 requiring a pretest probability >0.5 to achieve an FPR of 5%
- More than 90%
Difference in FPR between the two methods
- Median (IQR) difference in FPR calculated by the two methods
- 0% (0-2%)
Original publication
Statistically significant differences versus convincing evidence of real treatment effects: an analysis of the false positive risk for single-centre trials in anaesthesia. Br J Anaesth. Published 28 November 2023. doi:10.
Composed by AI from the published abstract and checked against it. How Briefs are made →