Ensuring basic competency in chest tube insertion using a simulated scenario: an international validation study

Introduction: Chest tube insertion can be associated with serious complications. A structured training programme is needed to minimise complications and enhance patient safety. Novices should pass a reliable test with solid evidence of validity before performing the procedure supervised on patients. The aim of this study was to establish a credible pass/fail standard.

Methods: We used an established assessment tool the Chest Tube Insertion Competency Test (TUBE-iCOMPT). Validity evidence was explored according to Messick's five sources of validity. Two methods were used to establish a credible pass/fail standard. Contrasting groups' method: 34 doctors (23 novices and 11 experienced surgeons) performed the procedure twice and all procedures were video recorded, edited, blinded and rated by two independent, international raters. Modified Angoff method: seven thoracic surgeons individually determined the scores that defined the pass/fail criteria. The data was gathered in Copenhagen, Denmark and Riyadh, Saudi Arabia.

Results: Internal consistency reliability was calculated as Cronbach's alpha to 0.94. The generalisability coefficient with two raters and two procedures was 0.91. Mean scores were 50.7 (SD±13.2) and 74.7 (SD±4.8) for novices and experienced surgeons, respectively (p<0.001). The pass/fail score of 62 points resulted in zero false negatives and only three false positives.

Discussion: We have gathered valuable additional validity evidence for the assessment tool TUBE-iCOMPT including establishment of a credible pass/fail score. The TUBE-iCOMPT can now be integrated in mastery learning programmes to ensure competency before independent practice.