Udskriv Udskriv
Switch language
Region Hovedstaden - en del af Københavns Universitetshospital

Ensuring competence in ultrasound-guided procedures-a validity study of a newly developed assessment tool

Publikation: Bidrag til tidsskriftTidsskriftartikelpeer review


  1. Using Self-Rated Examinations to Ensure Competence in Laparoscopy

    Publikation: Bidrag til tidsskriftTidsskriftartikelpeer review

  2. Effects of shared mental models in teams performing video-assisted thoracoscopic surgery lobectomy

    Publikation: Bidrag til tidsskriftTidsskriftartikelpeer review

  3. Pretraining of basic skills on a virtual reality vitreoretinal simulator: A waste of time

    Publikation: Bidrag til tidsskriftTidsskriftartikelpeer review

Vis graf over relationer

Objectives: To investigate the validity of the Interventional Ultrasound Skills Evaluation (IUSE) tool for assessment of procedural competence in ultrasound-guided procedures in a clinical environment, including a pass/fail score. Methods: Novices and experienced radiologists were recruited from four hospitals and were observed and assessed while performing ultrasound-guided procedures. Performances were assessed using the IUSE tool by two independent raters. Validity evidence was gathered in accordance with Messick’s framework: response process was ensured by standardisation of written rater instructions. Internal structure was explored using Cronbach’s alpha for internal consistency reliability; inter-rater reliability was calculated as Pearson’s r independently across all ratings, and test-retest reliability was reported using Cronbach’s alpha. Relationship to other variables was investigated by comparing performances of the participants in each group. Consequences evidence was explored by calculating a pass/fail standard using the contrasting groups method. Results: Six novices and twelve experienced radiologists were enrolled. The IUSE tool had high internal consistency (Cronbach’s alpha = 0.96, high inter-rater reliability (Pearson’s r = 0.95), and high test-retest reliability (Cronbach’s alpha = 0.98), and the mean score was 33.28 for novices and 59.25 for experienced with a highly significant difference (p value < 0.001). The pass/fail score was set at 55 resulting in no false positives or false negatives. Conclusions: Validity evidence from multiple sources supports the use of the IUSE tool for assessment of competence in ultrasound-guided procedures in a clinical environment and its use in high-stakes assessment such as certification. A credible pass/fail criterion was established to inform decision-making. Key Points: • A multi-site validity investigation established that the Interventional Ultrasound Skills Evaluation (IUSE) tool can be used to assess procedural competence in ultrasound-guided procedures. • Validity evidence was gathered according to Messick’s framework validity from the following sources: response process, internal structure, relationship to other variables, and consequences evidence. • The IUSE tool can be used for both formative and summative assessment, and a credible pass/fail score was established to help inform decision-making such as certification.

TidsskriftEuropean Radiology
Udgave nummer7
Sider (fra-til)4954-4966
Antal sider13
StatusUdgivet - jul. 2022

Bibliografisk note

© 2022. The Author(s), under exclusive licence to European Society of Radiology.

ID: 74976496