TY - JOUR
T1 - Performance metrics in mastoidectomy training
T2 - a systematic review
AU - Al-Shahrestani, Fahd
AU - Sørensen, Mads Sølvsten
AU - Andersen, Steven Arild Wuyts
PY - 2019/3
Y1 - 2019/3
N2 - OBJECTIVE: To investigate validity evidence, and strengths and limitations of performance metrics in mastoidectomy training.METHODS: A systematic review following the PRISMA guidelines. Studies reporting performance metrics in mastoidectomy/temporal bone surgery were included. Data on design, outcomes, and results were extracted by two reviewers. Validity evidence according to Messick's framework and level of evidence were assessed.RESULTS: The search yielded a total of 1085 studies from the years 1947-2018 and 35 studies were included for full data extraction after abstract and full-text screening. 33 different metrics on mastoidectomy performance were identified and ranked according to the number of reports. Most of the 33 metrics identified had some amount of validity evidence. The metrics with most validity evidence were related to drilling time, volume drilled per time, force applied near vital structures, and volume removed.CONCLUSIONS: This review provides an overview of current metrics of mastoidectomy performance, their validity, strengths and limitations, and identifies the gap in validity evidence of some metrics. Evidence-based metrics can be used for performance assessment in temporal bone surgery and for providing integrated and automated feedback in virtual reality simulation training. The use of such metrics in simulation-based mastoidectomy training can potentially address some of the limitations in current temporal bone skill assessment and ease assessment in repeated practice. However, at present, an automated feedback based on metrics in VR simulation does not have sufficient empirical basis and has not been generally accepted for use in training and certification.LEVEL OF EVIDENCE: 2a.
AB - OBJECTIVE: To investigate validity evidence, and strengths and limitations of performance metrics in mastoidectomy training.METHODS: A systematic review following the PRISMA guidelines. Studies reporting performance metrics in mastoidectomy/temporal bone surgery were included. Data on design, outcomes, and results were extracted by two reviewers. Validity evidence according to Messick's framework and level of evidence were assessed.RESULTS: The search yielded a total of 1085 studies from the years 1947-2018 and 35 studies were included for full data extraction after abstract and full-text screening. 33 different metrics on mastoidectomy performance were identified and ranked according to the number of reports. Most of the 33 metrics identified had some amount of validity evidence. The metrics with most validity evidence were related to drilling time, volume drilled per time, force applied near vital structures, and volume removed.CONCLUSIONS: This review provides an overview of current metrics of mastoidectomy performance, their validity, strengths and limitations, and identifies the gap in validity evidence of some metrics. Evidence-based metrics can be used for performance assessment in temporal bone surgery and for providing integrated and automated feedback in virtual reality simulation training. The use of such metrics in simulation-based mastoidectomy training can potentially address some of the limitations in current temporal bone skill assessment and ease assessment in repeated practice. However, at present, an automated feedback based on metrics in VR simulation does not have sufficient empirical basis and has not been generally accepted for use in training and certification.LEVEL OF EVIDENCE: 2a.
KW - Clinical Competence
KW - Humans
KW - Mastoid/surgery
KW - Mastoidectomy/education
KW - Otologic Surgical Procedures
KW - Simulation Training
KW - Temporal Bone/surgery
U2 - 10.1007/s00405-018-05265-9
DO - 10.1007/s00405-018-05265-9
M3 - Journal article
C2 - 30604063
SN - 0003-9195
VL - 276
SP - 657
EP - 664
JO - European archives of oto-rhino-laryngology : official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS) : affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery
JF - European archives of oto-rhino-laryngology : official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS) : affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery
IS - 3
ER -