Forskning
Udskriv Udskriv
Switch language
Region Hovedstaden - en del af Københavns Universitetshospital
E-pub ahead of print

Reliable Assessment of Surgical Technical Skills Is Dependent on Context: An Exploration of Different Variables Using Generalizability Theory

Publikation: Bidrag til tidsskriftTidsskriftartikelForskningpeer review

  1. The Positivism Paradigm of Research

    Publikation: Bidrag til tidsskriftTidsskriftartikelForskningpeer review

  2. Embracing Multiple Stakeholder Perspectives in Defining Trainee Competence

    Publikation: Bidrag til tidsskriftTidsskriftartikelForskningpeer review

  3. A Night on Trauma

    Publikation: Bidrag til tidsskriftTidsskriftartikelForskningpeer review

  4. In reply to walsh

    Publikation: Bidrag til tidsskriftKommentar/debatForskningpeer review

  1. Development and Validation of an Assessment Tool for Technical Skills in Handheld Otoscopy

    Publikation: Bidrag til tidsskriftTidsskriftartikelForskningpeer review

  2. Maintaining Competence in Airway Management

    Publikation: Bidrag til tidsskriftTidsskriftartikelForskningpeer review

  3. Developing a national e-learning course in otorhinolaryngology: the Danish experience

    Publikation: Bidrag til tidsskriftTidsskriftartikelForskningpeer review

  4. The Effect of Simulator-Integrated Tutoring for Guidance in Virtual Reality Simulation Training

    Publikation: Bidrag til tidsskriftTidsskriftartikelForskningpeer review

  5. Using computerized assessment in simulated colonoscopy: a validation study

    Publikation: Bidrag til tidsskriftTidsskriftartikelForskningpeer review

Vis graf over relationer

PURPOSE: Reliable assessment of surgical skills is vital for competency-based medical training. Several factors influence not only the reliability of judgements but also the number of observations needed for making judgments of competency that are both consistent and reproducible. The aim of this study was to explore the role of various conditions-through the analysis of data from large-scale, simulation-based assessments of surgical technical skills-by examining the effects of those conditions on reliability using Generalizability theory.

METHOD: Assessment data from large-scale, simulation-based temporal bone surgical training research studies in 2012-2018 were pooled, yielding collectively 3,574 assessments of 1,723 performances. The authors conducted generalizability analyses using an unbalanced random-effects design, and they performed decision studies to explore the effect of the different variables on projections of reliability.

RESULTS: Overall, five observations were needed to achieve a Generalizability coefficient > 0.8. Several variables modified the projections of reliability: increased learner experience necessitated more observations (5 for medical students, 7 for residents, and 8 for experienced surgeons); the more complex cadaveric dissection required fewer observations than virtual reality simulation (2 vs. 5 observations); and increased fidelity simulation graphics reduced the number of observations needed from 7 to 4. The training structure (either massed or distributed practice) and simulator-integrated tutoring had little effect on reliability. Finally, more observations were needed during initial training when the learning curve was steepest (6 observations) compared with the plateau phase (4 observations).

CONCLUSIONS: Reliability in surgical skills assessment seems less stable than it is often reported to be. Training context and conditions influence reliability. The findings from this study highlight that medical educators should exercise caution when using a specific simulation-based assessment in other contexts.

OriginalsprogEngelsk
TidsskriftAcademic medicine : journal of the Association of American Medical Colleges
ISSN1040-2446
DOI
StatusE-pub ahead of print - 23 jun. 2020

ID: 60286350