Assessment of competence in local anaesthetic thoracoscopy: development and validity investigation of a new assessment tool

Leizl Joy Nayahangan, Morten Bo Søndergaard Svendsen, Uffe Bodtger, Najib Rahman, Nick Maskell, Jatinder Singh Sidhu, Jonathan Lawaetz, Paul Frost Clementsen, Lars Konge

3 Citationer (Scopus)

Abstract

Background: The aims of the study were to develop an assessment tool in local anaesthetic thoracoscopy (LAT), investigate validity evidence, and establish a pass/fail standard.

Methods: Validity evidence for the assessment tool was gathered using the unified Messick framework. The tool was developed by five experts in respiratory medicine and medical education. Doctors with varying experience performed two consecutive procedures in a standardized, simulation-based setting using a newly developed thorax/lung silicone model. Performances were video-recorded and assessed by four expert raters using the new tool. Contrasting groups' method was used to set a pass/fail standard.

Results: Nine novices and 8 experienced participants were included, generating 34 recorded performances and 136 expert assessments. The tool had a high internal consistency (Cronbach's alpha =0.94) and high inter-rater reliability (Cronbach's alpha =0.91). The total item score significantly correlated with the global score (rs=0.86, P<0.001). Participants' first performance correlated to second performance (test-retest reliability) with a Pearson's r of 0.93, P<0.001. Generalisability (G) study showed a G-coefficient of 0.92 and decision (D) study estimated that one performance assessed by two raters or four performances assessed by one rater are needed to reach an acceptable reliability, i.e., G-coefficient >0.80. The tool was able to discriminate between the two groups in both performances: experienced mean score =30.8±4.2; novice mean score =15.8±2.3, P<0.001. Pass/fail standard was set at 22 points.

Conclusions: The newly developed assessment tool showed solid evidence of validity and can be used to ensure competence in LAT.

OriginalsprogEngelsk
TidsskriftJournal of Thoracic Disease
Vol/bind13
Udgave nummer7
Sider (fra-til)3998-4007
Antal sider10
ISSN2072-1439
DOI
StatusUdgivet - jul. 2021

Fingeraftryk

Dyk ned i forskningsemnerne om 'Assessment of competence in local anaesthetic thoracoscopy: development and validity investigation of a new assessment tool'. Sammen danner de et unikt fingeraftryk.

Citationsformater