Research
Print page Print page
Switch language
Rigshospitalet - a part of Copenhagen University Hospital
Published

Expert sampling of VR simulator metrics for automated assessment of mastoidectomy performance

Research output: Contribution to journalJournal articleResearchpeer-review

DOI

  1. Association of SLC26A4 mutations, morphology, and hearing in pendred syndrome and NSEVA

    Research output: Contribution to journalJournal articleResearchpeer-review

  2. Laryngeal precursor lesions: Interrater and intrarater reliability of histopathological assessment

    Research output: Contribution to journalJournal articleResearchpeer-review

  3. Competency-based assessment in surgeon-performed head and neck ultrasonography: A validity study

    Research output: Contribution to journalJournal articleResearchpeer-review

  4. The human endolymphatic sac expresses natriuretic peptides

    Research output: Contribution to journalJournal articleResearchpeer-review

  5. Mapping the plateau of novices in virtual reality simulation training of mastoidectomy

    Research output: Contribution to journalJournal articleResearchpeer-review

  1. Development and Validation of an Assessment Tool for Technical Skills in Handheld Otoscopy

    Research output: Contribution to journalJournal articleResearchpeer-review

  2. The effect of structured self-assessment in virtual reality simulation training of mastoidectomy

    Research output: Contribution to journalJournal articleResearchpeer-review

  3. Decentralized virtual reality mastoidectomy simulation training: a prospective, mixed-methods study

    Research output: Contribution to journalJournal articleResearchpeer-review

View graph of relations

OBJECTIVE: Often the assessment of mastoidectomy performance requires time-consuming manual rating. Virtual reality (VR) simulators offer potentially useful automated assessment and feedback but should be supported by validity evidence. We aimed to investigate simulator metrics for automated assessment based on the expert performance approach, comparison with an established assessment tool, and the consequences of standard setting.

METHODS: The performances of 11 experienced otosurgeons and 37 otorhinolaryngology residents. Participants performed three mastoidectomies in the Visible Ear Simulator. Nine residents contributed additional data on repeated practice in the simulator. One hundred and twenty-nine different performance metrics were collected by the simulator and final-product files were saved. These final products were analyzed using a modified Welling Scale by two blinded raters.

RESULTS: Seventeen metrics could discriminate between resident and experienced surgeons' performances. These metrics mainly expressed various aspects of efficiency: Experts demonstrated more goal-directed behavior and less hesitancy, used less time, and selected large and sharp burrs more often. The combined metrics-based score (MBS) demonstrated significant discriminative ability between experienced surgeons and residents with a mean difference of 16.4% (95% confidence interval [12.6-20.2], P < 0.001). A pass/fail score of 83.6% was established. The MBS correlated poorly with the final-product score but excellently with the final-product score per time.

CONCLUSION: The MBS mainly reflected efficiency components of the mastoidectomy procedure, and although it could have some uses in self-directed training, it fails to measure and encourage safe routines. Supplemental approaches and feedback are therefore required in VR simulation training of mastoidectomy.

LEVEL OF EVIDENCE: 2b Laryngoscope, 129:2170-2177, 2019.

Original languageEnglish
JournalThe Laryngoscope
Volume129
Issue number9
Pages (from-to)2170-2177
Number of pages8
ISSN0023-852X
DOIs
Publication statusPublished - Sep 2019

    Research areas

  • Adult, Aged, Clinical Competence, Education, Medical, Continuing, Education, Medical, Graduate, Educational Measurement, Female, Humans, Male, Mastoidectomy, Middle Aged, Otolaryngology/education, Simulation Training/methods, User-Computer Interface, Virtual Reality

ID: 59291729