Print page Print page
Switch language
The Capital Region of Denmark - a part of Copenhagen University Hospital

Ensuring competence in ultrasound-guided procedures-a validity study of a newly developed assessment tool

Research output: Contribution to journalJournal articleResearchpeer-review

  1. Assessing VATS competence based on simulated lobectomies of all five lung lobes

    Research output: Contribution to journalJournal articleResearchpeer-review

  2. Lower urinary tract obstruction caused by fibroepithelial polyp in a newborn boy

    Research output: Contribution to journalJournal articleResearchpeer-review

  3. Training and assessment of musculoskeletal ultrasound and injection skills-a systematic review

    Research output: Contribution to journalReviewResearchpeer-review

View graph of relations

Objectives: To investigate the validity of the Interventional Ultrasound Skills Evaluation (IUSE) tool for assessment of procedural competence in ultrasound-guided procedures in a clinical environment, including a pass/fail score. Methods: Novices and experienced radiologists were recruited from four hospitals and were observed and assessed while performing ultrasound-guided procedures. Performances were assessed using the IUSE tool by two independent raters. Validity evidence was gathered in accordance with Messick’s framework: response process was ensured by standardisation of written rater instructions. Internal structure was explored using Cronbach’s alpha for internal consistency reliability; inter-rater reliability was calculated as Pearson’s r independently across all ratings, and test-retest reliability was reported using Cronbach’s alpha. Relationship to other variables was investigated by comparing performances of the participants in each group. Consequences evidence was explored by calculating a pass/fail standard using the contrasting groups method. Results: Six novices and twelve experienced radiologists were enrolled. The IUSE tool had high internal consistency (Cronbach’s alpha = 0.96, high inter-rater reliability (Pearson’s r = 0.95), and high test-retest reliability (Cronbach’s alpha = 0.98), and the mean score was 33.28 for novices and 59.25 for experienced with a highly significant difference (p value < 0.001). The pass/fail score was set at 55 resulting in no false positives or false negatives. Conclusions: Validity evidence from multiple sources supports the use of the IUSE tool for assessment of competence in ultrasound-guided procedures in a clinical environment and its use in high-stakes assessment such as certification. A credible pass/fail criterion was established to inform decision-making. Key Points: • A multi-site validity investigation established that the Interventional Ultrasound Skills Evaluation (IUSE) tool can be used to assess procedural competence in ultrasound-guided procedures. • Validity evidence was gathered according to Messick’s framework validity from the following sources: response process, internal structure, relationship to other variables, and consequences evidence. • The IUSE tool can be used for both formative and summative assessment, and a credible pass/fail score was established to help inform decision-making such as certification.

Original languageEnglish
JournalEuropean Radiology
Issue number7
Pages (from-to)4954-4966
Number of pages13
Publication statusPublished - Jul 2022

Bibliographical note

© 2022. The Author(s), under exclusive licence to European Society of Radiology.

    Research areas

  • Certification, Education, medical, Educational measurement, Reproducibility of results, Ultrasonography, interventional

ID: 74976496