Print page Print page
Switch language
The Capital Region of Denmark - a part of Copenhagen University Hospital

Systematic reviews of anesthesiologic interventions reported as statistically significant: problems with power, precision, and type 1 error protection

Research output: Contribution to journalJournal articleResearchpeer-review

  1. Acute Kidney Injury and Risk of Death After Elective Surgery: Prospective Analysis of Data From an International Cohort Study

    Research output: Contribution to journalJournal articleResearchpeer-review

  2. Ketamine as a Rapid Sequence Induction Agent in the Trauma Population: A Systematic Review

    Research output: Contribution to journalReviewResearchpeer-review

  3. Anesthesia in Enhanced Recovery Pathways for Hip and Knee Arthroplasty: Where Is the Evidence?

    Research output: Contribution to journalComment/debateResearchpeer-review

  4. Enhanced Recovery Pathways: Looking Into the Future

    Research output: Contribution to journalEditorialResearchpeer-review

View graph of relations

BACKGROUND: The GRADE Working Group assessment of the quality of evidence is being used increasingly to inform clinical decisions and guidelines. The assessment involves explicit consideration of all sources of uncertainty. One of these sources is imprecision or random error. Many published meta-analyses are underpowered and likely to be updated in the future. When data are sparse and there are repeated updates, the risk of random error is increased. Trial Sequential Analysis (TSA) is one of several methodologies that estimates this increased risk (and decreased precision) in meta-analyses. With nominally statistically significant meta-analyses of anesthesiologic interventions, we used TSA to estimate power and imprecision in the context of sparse data and repeated updates.

METHODS: We conducted a search to identify all systematic reviews with meta-analyses that investigated an intervention that may be implemented by an anesthesiologist during the perioperative period. We randomly selected 50 meta-analyses that reported a statistically significant dichotomous outcome in their abstract. We applied TSA to these meta-analyses by using 2 main TSA approaches: relative risk reduction 20% and relative risk reduction consistent with the conventional 95% confidence limit closest to null. We calculated the power achieved by each included meta-analysis, by using each TSA approach, and we calculated the proportion that maintained statistical significance when allowing for sparse data and repeated updates.

RESULTS: From 11,870 titles, we found 682 systematic reviews that investigated anesthesiologic interventions. In the 50 sampled meta-analyses, the median number of trials included was 8 (interquartile range [IQR], 5-14), the median number of participants was 964 (IQR, 523-1736), and the median number of participants with the outcome was 202 (IQR, 96-443). By using both of our main TSA approaches, only 12% (95% CI, 5%-25%) of the meta-analyses had power ≥80%, and only 32% (95% CI, 20%-47%) of the meta-analyses preserved the risk of type 1 error <5%.

CONCLUSIONS: Most nominally statistically significant meta-analyses of anesthesiologic interventions are underpowered, and many do not maintain their risk of type 1 error <5% if TSA monitoring boundaries are applied. Consideration of the effect of sparse data and repeated updates is needed when assessing the imprecision of meta-analyses of anesthesiologic interventions.

Original languageEnglish
JournalAnesthesia and Analgesia
Issue number6
Pages (from-to)1611-22
Number of pages12
Publication statusPublished - Dec 2015

ID: 45806662