The left upper lobe challenge in Video-Assisted Thoracoscopic Surgery - use of a composite score to improve the assessment of simulated lobectomy

AIM: The aim of this study is to develop a reliable composite score based on simulator metrics to assess competency in virtual reality video-assisted thoracoscopic surgery lobectomy and explore the benefits of combining it with expert rater assessments.

METHODS: Standardized objective assessments (time, bleeding, economy of movement) and subjective expert rater assessments from 2 previous studies were combined. A linear mixed model including experience level, lobe and the number of previous simulated procedures was applied for the repeated measurements. Reliability for each of the 4 assessments was calculated using Cronbach's alpha. The Nelder-Mead numerical optimization algorithm was used for optimal weighting of scores. A pass-fail standard for the composite score was determined using the contrasting groups' method.

RESULTS: In total, 123 virtual reality video-assisted thoracoscopic surgery lobectomies were included. Across the 4 different assessments, there were significant effects (P < 0.01) of experience, lobe, and simulator experience, but not for simulator attempts on bleeding (P = 0.98). The left upper lobe was significantly more difficult compared to other lobes (P = 0.02). A maximum reliability of 0.92 could be achieved by combining the standardized simulator metrics with standardized expert rater scores. The pass/fail level for the composite score when including 1 expert rater was 0.33.

CONCLUSIONS: Combining simulator metrics with 1 or 2 raters increases reliability and can serve as a more objective method for assessing surgical trainees. The composite score may be used to implement a standardized and feasible simulation-based mastery training program in video-assisted thoracoscopic surgery lobectomy.