Competency assessment for the Versius surgical robot: a validity investigation study of a virtual reality simulator-based test

BACKGROUND: When introducing new equipment like robotic surgical systems, it is essential to ensure that surgeons have the basic skills before operating on patients. The objective was to investigate the validity evidence for a competency-based test for basic robotic surgical skills using the Versius® trainer.

METHODS: We recruited medical students, residents, and surgeons which were classified based on data on clinical experience with the Versius system as either novices (0 min), intermediates (1-1000 min), or experienced (> 1000 min). All participants completed three rounds of eight basic exercises on the Versius trainer, where the first was used for familiarization and the final two for data analysis. The simulator automatically recorded data. Validity evidence was summarized using Messick's framework, and the contrasting groups' standard-setting method was used to define pass/fail levels.

RESULTS: 40 participants completed the three rounds of exercises. The discriminatory abilities of all parameters were tested, and five exercises including relevant parameters were selected to be part of the final test. 26 of 30 parameters could differentiate between novices and experienced surgeons but none of the parameters could discriminate between the intermediate and experienced surgeons. Test-retest reliability analysis using Pearson's r or Spearman's rho showed only 13 of 30 parameters had moderate or higher reliability. Non-compensatory pass/fail levels were defined for each exercise and showed that all novices failed all the exercises and that most experienced surgeons either passed or nearly passed all five exercises.

CONCLUSION: We identified relevant parameters for five exercises that could be used to assess basic robotic skills for the Versius robotic system and defined a credible pass/fail level. This is the first step in developing a proficiency-based training program for the Versius system.