Forskning
Udskriv Udskriv
Switch language
Rigshospitalet - en del af Københavns Universitetshospital
Udgivet

Apparently conclusive meta-analyses on interventions in critical care may be inconclusive - a meta-epidemiological study

Publikation: Bidrag til tidsskriftReviewForskningpeer review

Harvard

APA

CBE

MLA

Vancouver

Author

Bibtex

@article{8973e17f4f5c499ca4d7cf8b03ebd9eb,
title = "Apparently conclusive meta-analyses on interventions in critical care may be inconclusive - a meta-epidemiological study",
abstract = "INTRODUCTION: Risks of random type I and II errors are associated with false positive and negative findings. In conventional meta-analyses, the risks of random errors are insufficiently evaluated. Many meta-analyses which appear conclusive might in fact be inconclusive due to risks of random errors. We hypothesize that for interventions in critical care, false positive and negative findings frequently become apparent when accounting for the risks of random error. We aim to investigate to which extent apparently conclusive conventional meta-analyses remain conclusive when adjusting statistical significance levels and confidence intervals considering sparse data and repeated testing through Trial Sequential Analysis (TSA).METHODS: We searched The Cochrane Library, MEDLINE, and EMBASE for reviews of interventions in critical care. We used TSA with the relative risk reduction from the estimated meta-analysed intervention effects adjusted for heterogeneity based on the observed diversity. We report proportions of meta-analyses and potential inconclusive findings of positive, neutral, and negative conclusions based on conventional na{\"i}ve meta-analyses which use an alpha of 5{\%} and 95{\%} confidence intervals. In TSA-controlled meta-analyses showing a beneficial or harmful intervention effect, we assessed the risk of bias by six Cochrane domains.RESULTS: A total of 464 reviews containing 1080 meta-analyses of (co-)primary outcomes were analysed. From the 266 conventional meta-analyses suggesting a beneficial effect, 133 (50{\%}) were true positive and 133 (50{\%}) potentially false positive according to TSA. From the 755 conventional meta-analyses suggesting a neutral effect, there were 214 (28{\%}) true neutral and 541 (72{\%}) potentially false neutral according to TSA. From the 59 conventional meta-analyses suggesting a harmful effect, 17 (29{\%}) were true negative and 42 (71{\%}) were potentially false negative according to TSA. When the true beneficial and true harmful meta-analyses according to TSA were evaluated for risk of bias, new TSAs conducted on only trials with overall low risk of bias showed only firm evidence of a beneficial effect on one outcome and a harmful effect on one outcome.CONCLUSIONS: Out of all meta-analyses in critical care, a large proportion may reach false conclusions due to unknown risks of random type I or type II errors. Future critical care meta-analyses, should aim for establishing an effect of interventions accounting for risks of bias and random errors.",
author = "Koster, {T M} and J Wetterslev and C Gluud and Jakobsen, {J C} and T Kaufmann and Eck, {R J} and G Koster and B Hiemstra and {van der Horst}, {I C C} and F Keus",
note = "Copyright {\circledC} 2019 Elsevier Inc. All rights reserved.",
year = "2019",
month = "6",
day = "11",
doi = "10.1016/j.jclinepi.2019.05.011",
language = "English",
volume = "114",
pages = "1--10",
journal = "Journal of Clinical Epidemiology",
issn = "0895-4356",
publisher = "Elsevier Inc",

}

RIS

TY - JOUR

T1 - Apparently conclusive meta-analyses on interventions in critical care may be inconclusive - a meta-epidemiological study

AU - Koster, T M

AU - Wetterslev, J

AU - Gluud, C

AU - Jakobsen, J C

AU - Kaufmann, T

AU - Eck, R J

AU - Koster, G

AU - Hiemstra, B

AU - van der Horst, I C C

AU - Keus, F

N1 - Copyright © 2019 Elsevier Inc. All rights reserved.

PY - 2019/6/11

Y1 - 2019/6/11

N2 - INTRODUCTION: Risks of random type I and II errors are associated with false positive and negative findings. In conventional meta-analyses, the risks of random errors are insufficiently evaluated. Many meta-analyses which appear conclusive might in fact be inconclusive due to risks of random errors. We hypothesize that for interventions in critical care, false positive and negative findings frequently become apparent when accounting for the risks of random error. We aim to investigate to which extent apparently conclusive conventional meta-analyses remain conclusive when adjusting statistical significance levels and confidence intervals considering sparse data and repeated testing through Trial Sequential Analysis (TSA).METHODS: We searched The Cochrane Library, MEDLINE, and EMBASE for reviews of interventions in critical care. We used TSA with the relative risk reduction from the estimated meta-analysed intervention effects adjusted for heterogeneity based on the observed diversity. We report proportions of meta-analyses and potential inconclusive findings of positive, neutral, and negative conclusions based on conventional naïve meta-analyses which use an alpha of 5% and 95% confidence intervals. In TSA-controlled meta-analyses showing a beneficial or harmful intervention effect, we assessed the risk of bias by six Cochrane domains.RESULTS: A total of 464 reviews containing 1080 meta-analyses of (co-)primary outcomes were analysed. From the 266 conventional meta-analyses suggesting a beneficial effect, 133 (50%) were true positive and 133 (50%) potentially false positive according to TSA. From the 755 conventional meta-analyses suggesting a neutral effect, there were 214 (28%) true neutral and 541 (72%) potentially false neutral according to TSA. From the 59 conventional meta-analyses suggesting a harmful effect, 17 (29%) were true negative and 42 (71%) were potentially false negative according to TSA. When the true beneficial and true harmful meta-analyses according to TSA were evaluated for risk of bias, new TSAs conducted on only trials with overall low risk of bias showed only firm evidence of a beneficial effect on one outcome and a harmful effect on one outcome.CONCLUSIONS: Out of all meta-analyses in critical care, a large proportion may reach false conclusions due to unknown risks of random type I or type II errors. Future critical care meta-analyses, should aim for establishing an effect of interventions accounting for risks of bias and random errors.

AB - INTRODUCTION: Risks of random type I and II errors are associated with false positive and negative findings. In conventional meta-analyses, the risks of random errors are insufficiently evaluated. Many meta-analyses which appear conclusive might in fact be inconclusive due to risks of random errors. We hypothesize that for interventions in critical care, false positive and negative findings frequently become apparent when accounting for the risks of random error. We aim to investigate to which extent apparently conclusive conventional meta-analyses remain conclusive when adjusting statistical significance levels and confidence intervals considering sparse data and repeated testing through Trial Sequential Analysis (TSA).METHODS: We searched The Cochrane Library, MEDLINE, and EMBASE for reviews of interventions in critical care. We used TSA with the relative risk reduction from the estimated meta-analysed intervention effects adjusted for heterogeneity based on the observed diversity. We report proportions of meta-analyses and potential inconclusive findings of positive, neutral, and negative conclusions based on conventional naïve meta-analyses which use an alpha of 5% and 95% confidence intervals. In TSA-controlled meta-analyses showing a beneficial or harmful intervention effect, we assessed the risk of bias by six Cochrane domains.RESULTS: A total of 464 reviews containing 1080 meta-analyses of (co-)primary outcomes were analysed. From the 266 conventional meta-analyses suggesting a beneficial effect, 133 (50%) were true positive and 133 (50%) potentially false positive according to TSA. From the 755 conventional meta-analyses suggesting a neutral effect, there were 214 (28%) true neutral and 541 (72%) potentially false neutral according to TSA. From the 59 conventional meta-analyses suggesting a harmful effect, 17 (29%) were true negative and 42 (71%) were potentially false negative according to TSA. When the true beneficial and true harmful meta-analyses according to TSA were evaluated for risk of bias, new TSAs conducted on only trials with overall low risk of bias showed only firm evidence of a beneficial effect on one outcome and a harmful effect on one outcome.CONCLUSIONS: Out of all meta-analyses in critical care, a large proportion may reach false conclusions due to unknown risks of random type I or type II errors. Future critical care meta-analyses, should aim for establishing an effect of interventions accounting for risks of bias and random errors.

U2 - 10.1016/j.jclinepi.2019.05.011

DO - 10.1016/j.jclinepi.2019.05.011

M3 - Review

VL - 114

SP - 1

EP - 10

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

SN - 0895-4356

ER -

ID: 57396966