Assessing VATS competence based on simulated lobectomies of all five lung lobes

Tamim Ahmad Haidari*, Flemming Bjerrum, Thomas Decker Christensen, Henrik Vad, Lars Borgbjerg Møller, Henrik Jessen Hansen, Lars Konge, René Horsleben Petersen

*Corresponding author for this work
1 Citation (Scopus)

Abstract

OBJECTIVES: To determine the number of procedures and expert raters necessary to provide a reliable assessment of competence in Video-Assisted Thoracoscopic Surgery (VATS) lobectomy.

METHODS: Three randomly selected VATS lobectomies were performed on a virtual reality simulator by participants with varying experience in VATS. Video recordings of the procedures were independently rated by three blinded VATS experts using a modified VATS lobectomy assessment tool (VATSAT). The unitary framework of validity was used to describe validity evidence, and generalizability theory was used to explore the reliability of different assessment options.

RESULTS: Forty-one participants (22 novices, 10 intermediates, and 9 experienced) performed a total of 123 lobectomies. Internal consistency reliability, inter-rater reliability, and test-retest reliability were 0.94, 0.85, and 0.90, respectively. Generalizability theory found that a minimum of two procedures and four raters or three procedures and three raters were needed to ensure the overall reliability of 0.8. ANOVA showed significant differences in test scores between the three groups (P < 0.001). A pass/fail level of 19 out of 25 points was established using the contrasting groups' standard setting method, leaving one false positive (one novice passed) and zero false negatives (all experienced passed).

CONCLUSION: We demonstrated validity evidence for a VR simulator test with different lung lobes, and a credible pass/fail level was identified. Our results can be used to implement a standardized mastery learning training program for trainees in VATS lobectomies that ensures that everyone reaches basic competency before performing supervised operations on patients.

Original languageEnglish
JournalSurgical Endoscopy
Volume36
Issue number11
Pages (from-to)8067-8075
Number of pages9
ISSN0930-2794
DOIs
Publication statusPublished - Nov 2022

Keywords

  • Assessment
  • Competency
  • Educating VATS surgeons
  • Mastery learning
  • VATS lobectomy
  • Virtual reality simulation
  • Clinical Competence
  • Reproducibility of Results
  • Pneumonectomy/methods
  • Humans
  • Lung
  • Thoracic Surgery, Video-Assisted/methods

Fingerprint

Dive into the research topics of 'Assessing VATS competence based on simulated lobectomies of all five lung lobes'. Together they form a unique fingerprint.

Cite this