TY - JOUR
T1 - 3-D-Printed Models for Temporal Bone Training
T2 - A Validity Study
AU - Frithioff, Andreas
AU - Frendø, Martin
AU - Weiss, Kenneth
AU - Foghsgaard, Søren
AU - Mikkelsen, Peter Trier
AU - Frederiksen, Thomas Winther
AU - Pedersen, David Bue
AU - Sørensen, Mads Sølvsten
AU - Andersen, Steven Arild Wuyts
N1 - Copyright © 2023, Otology & Neurotology, Inc.
PY - 2023/8/1
Y1 - 2023/8/1
N2 - OBJECTIVE: 3-D printing offers convenient and low-cost mastoidectomy training; nonetheless, training benefits using 3-D-printed temporal bones remain largely unexplored. In this study, we have collected validity evidence for a low-cost, 3-D-printed temporal bone for mastoidectomy training and established a credible pass/fail score for performance on the model.STUDY DESIGN: A prospective educational study gathering validity evidence using Messick's validity framework.SETTING: Seven Danish otorhinolaryngology training institutions.PARTICIPANTS: Eighteen otorhinolaryngology residents (novices) and 11 experienced otosurgeons (experts).INTERVENTION: Residents and experienced otosurgeons each performed two to three anatomical mastoidectomies on a low-cost, 3-D-printed temporal bone model produced in-house. After drilling, mastoidectomy performances were rated by three blinded experts using a 25-item modified Welling scale (WS).MAIN OUTCOME MEASURE: Validity evidence using Messick's framework including reliability assessment applying both classical test theory and Generalizability theory.RESULTS: Novices achieved a mean score of 13.9 points; experienced otosurgeons achieved 23.2 points. Using the contrasting groups method, we established a 21/25-point pass/fail level. The Generalizability coefficient was 0.91, and 75% of the score variance was attributable to participant performance, indicating a high level of assessment reliability. Subsequent D studies revealed that two raters rating one performance or one rater rating two performances were sufficiently reliable for high-stakes assessment.CONCLUSION: Validity evidence supports using a low-cost, 3-D-printed model for mastoidectomy training. The model can be printed in-house using consumer-grade 3-D printers and serves as an additional training tool in the temporal bone curriculum. For competency-based training, we established a cut-off score of 21 of 25 WS points using the contrasting groups method.
AB - OBJECTIVE: 3-D printing offers convenient and low-cost mastoidectomy training; nonetheless, training benefits using 3-D-printed temporal bones remain largely unexplored. In this study, we have collected validity evidence for a low-cost, 3-D-printed temporal bone for mastoidectomy training and established a credible pass/fail score for performance on the model.STUDY DESIGN: A prospective educational study gathering validity evidence using Messick's validity framework.SETTING: Seven Danish otorhinolaryngology training institutions.PARTICIPANTS: Eighteen otorhinolaryngology residents (novices) and 11 experienced otosurgeons (experts).INTERVENTION: Residents and experienced otosurgeons each performed two to three anatomical mastoidectomies on a low-cost, 3-D-printed temporal bone model produced in-house. After drilling, mastoidectomy performances were rated by three blinded experts using a 25-item modified Welling scale (WS).MAIN OUTCOME MEASURE: Validity evidence using Messick's framework including reliability assessment applying both classical test theory and Generalizability theory.RESULTS: Novices achieved a mean score of 13.9 points; experienced otosurgeons achieved 23.2 points. Using the contrasting groups method, we established a 21/25-point pass/fail level. The Generalizability coefficient was 0.91, and 75% of the score variance was attributable to participant performance, indicating a high level of assessment reliability. Subsequent D studies revealed that two raters rating one performance or one rater rating two performances were sufficiently reliable for high-stakes assessment.CONCLUSION: Validity evidence supports using a low-cost, 3-D-printed model for mastoidectomy training. The model can be printed in-house using consumer-grade 3-D printers and serves as an additional training tool in the temporal bone curriculum. For competency-based training, we established a cut-off score of 21 of 25 WS points using the contrasting groups method.
KW - Humans
KW - Prospective Studies
KW - Reproducibility of Results
KW - Temporal Bone/surgery
KW - Mastoidectomy/methods
KW - Otolaryngology/education
KW - Simulation Training/methods
KW - Clinical Competence
UR - http://www.scopus.com/inward/record.url?scp=85164843203&partnerID=8YFLogxK
U2 - 10.1097/MAO.0000000000003936
DO - 10.1097/MAO.0000000000003936
M3 - Journal article
C2 - 37442608
SN - 1531-7129
VL - 44
SP - e497-e503
JO - Otology & neurotology : official publication of the American Otological Society, American Neurotology Society [and] European Academy of Otology and Neurotology
JF - Otology & neurotology : official publication of the American Otological Society, American Neurotology Society [and] European Academy of Otology and Neurotology
IS - 7
ER -