Forskning
Udskriv Udskriv
Switch language
Region Hovedstaden - en del af Københavns Universitetshospital
Udgivet

Reliable Assessment of Surgical Technical Skills Is Dependent on Context: An Exploration of Different Variables Using Generalizability Theory

Publikation: Bidrag til tidsskriftTidsskriftartikelForskningpeer review

  1. The Positivism Paradigm of Research

    Publikation: Bidrag til tidsskriftTidsskriftartikelForskningpeer review

  2. Embracing Multiple Stakeholder Perspectives in Defining Trainee Competence

    Publikation: Bidrag til tidsskriftTidsskriftartikelForskningpeer review

  3. A Night on Trauma

    Publikation: Bidrag til tidsskriftTidsskriftartikelForskningpeer review

  4. In reply to walsh

    Publikation: Bidrag til tidsskriftKommentar/debatForskningpeer review

  1. Reliable test of clinicians' mastery in skin cancer diagnostics

    Publikation: Bidrag til tidsskriftTidsskriftartikelForskningpeer review

  2. Educational needs in the COVID-19 pandemic: a Delphi study among doctors and nurses in Wuhan, China

    Publikation: Bidrag til tidsskriftTidsskriftartikelForskningpeer review

  3. European Respiratory Society Statement on Thoracic Ultrasound

    Publikation: Bidrag til bog/antologi/rapportBidrag til bog/antologiForskning

  4. Simulation Based Training and Assessment in Open Vascular Surgery: A Systematic Review

    Publikation: Bidrag til tidsskriftReviewpeer review

Vis graf over relationer

PURPOSE: Reliable assessment of surgical skills is vital for competency-based medical training. Several factors influence not only the reliability of judgments but also the number of observations needed for making judgments of competency that are both consistent and reproducible. The aim of this study was to explore the role of various conditions-through the analysis of data from large-scale, simulation-based assessments of surgical technical skills-by examining the effects of those conditions on reliability using generalizability theory.

METHOD: Assessment data from large-scale, simulation-based temporal bone surgical training research studies in 2012-2018 were pooled, yielding collectively 3,574 assessments of 1,723 performances. The authors conducted generalizability analyses using an unbalanced random-effects design, and they performed decision studies to explore the effect of the different variables on projections of reliability.

RESULTS: Overall, 5 observations were needed to achieve a generalizability coefficient > 0.8. Several variables modified the projections of reliability: increased learner experience necessitated more observations (5 for medical students, 7 for residents, and 8 for experienced surgeons), the more complex cadaveric dissection required fewer observations than virtual reality simulation (2 vs 5 observations), and increased fidelity simulation graphics reduced the number of observations needed from 7 to 4. The training structure (either massed or distributed practice) and simulator-integrated tutoring had little effect on reliability. Finally, more observations were needed during initial training when the learning curve was steepest (6 observations) compared with the plateau phase (4 observations).

CONCLUSIONS: Reliability in surgical skills assessment seems less stable than it is often reported to be. Training context and conditions influence reliability. The findings from this study highlight that medical educators should exercise caution when using a specific simulation-based assessment in other contexts.

OriginalsprogEngelsk
TidsskriftAcademic medicine : journal of the Association of American Medical Colleges
Vol/bind95
Udgave nummer12
Sider (fra-til)1929-1936
Antal sider8
ISSN1040-2446
DOI
StatusUdgivet - dec. 2020

ID: 60286350