Research
Print page Print page
Switch language
The Capital Region of Denmark - a part of Copenhagen University Hospital
Published

Assessment of competence in local anaesthetic thoracoscopy: development and validity investigation of a new assessment tool

Research output: Contribution to journalJournal articleResearchpeer-review

Harvard

APA

CBE

MLA

Vancouver

Author

Bibtex

@article{b00fa5d22a244a09854eed5d0692c19c,
title = "Assessment of competence in local anaesthetic thoracoscopy: development and validity investigation of a new assessment tool",
abstract = "Background: The aims of the study were to develop an assessment tool in local anaesthetic thoracoscopy (LAT), investigate validity evidence, and establish a pass/fail standard.Methods: Validity evidence for the assessment tool was gathered using the unified Messick framework. The tool was developed by five experts in respiratory medicine and medical education. Doctors with varying experience performed two consecutive procedures in a standardized, simulation-based setting using a newly developed thorax/lung silicone model. Performances were video-recorded and assessed by four expert raters using the new tool. Contrasting groups' method was used to set a pass/fail standard.Results: Nine novices and 8 experienced participants were included, generating 34 recorded performances and 136 expert assessments. The tool had a high internal consistency (Cronbach's alpha =0.94) and high inter-rater reliability (Cronbach's alpha =0.91). The total item score significantly correlated with the global score (rs=0.86, P<0.001). Participants' first performance correlated to second performance (test-retest reliability) with a Pearson's r of 0.93, P<0.001. Generalisability (G) study showed a G-coefficient of 0.92 and decision (D) study estimated that one performance assessed by two raters or four performances assessed by one rater are needed to reach an acceptable reliability, i.e., G-coefficient >0.80. The tool was able to discriminate between the two groups in both performances: experienced mean score =30.8±4.2; novice mean score =15.8±2.3, P<0.001. Pass/fail standard was set at 22 points.Conclusions: The newly developed assessment tool showed solid evidence of validity and can be used to ensure competence in LAT.",
keywords = "Assessment, Pleuroscopy, Simulation training, Thoracoscopy, Training, Validation study",
author = "Nayahangan, {Leizl Joy} and Svendsen, {Morten Bo S{\o}ndergaard} and Uffe Bodtger and Najib Rahman and Nick Maskell and Sidhu, {Jatinder Singh} and Jonathan Lawaetz and Clementsen, {Paul Frost} and Lars Konge",
note = "2021 Journal of Thoracic Disease. All rights reserved.",
year = "2021",
month = jul,
doi = "10.21037/jtd-20-3560",
language = "English",
volume = "13",
pages = "3998--4007",
journal = "Journal of Thoracic Disease",
issn = "2072-1439",
publisher = "Pioneer Bioscience Publishing Company (PBPC)",
number = "7",

}

RIS

TY - JOUR

T1 - Assessment of competence in local anaesthetic thoracoscopy

T2 - development and validity investigation of a new assessment tool

AU - Nayahangan, Leizl Joy

AU - Svendsen, Morten Bo Søndergaard

AU - Bodtger, Uffe

AU - Rahman, Najib

AU - Maskell, Nick

AU - Sidhu, Jatinder Singh

AU - Lawaetz, Jonathan

AU - Clementsen, Paul Frost

AU - Konge, Lars

N1 - 2021 Journal of Thoracic Disease. All rights reserved.

PY - 2021/7

Y1 - 2021/7

N2 - Background: The aims of the study were to develop an assessment tool in local anaesthetic thoracoscopy (LAT), investigate validity evidence, and establish a pass/fail standard.Methods: Validity evidence for the assessment tool was gathered using the unified Messick framework. The tool was developed by five experts in respiratory medicine and medical education. Doctors with varying experience performed two consecutive procedures in a standardized, simulation-based setting using a newly developed thorax/lung silicone model. Performances were video-recorded and assessed by four expert raters using the new tool. Contrasting groups' method was used to set a pass/fail standard.Results: Nine novices and 8 experienced participants were included, generating 34 recorded performances and 136 expert assessments. The tool had a high internal consistency (Cronbach's alpha =0.94) and high inter-rater reliability (Cronbach's alpha =0.91). The total item score significantly correlated with the global score (rs=0.86, P<0.001). Participants' first performance correlated to second performance (test-retest reliability) with a Pearson's r of 0.93, P<0.001. Generalisability (G) study showed a G-coefficient of 0.92 and decision (D) study estimated that one performance assessed by two raters or four performances assessed by one rater are needed to reach an acceptable reliability, i.e., G-coefficient >0.80. The tool was able to discriminate between the two groups in both performances: experienced mean score =30.8±4.2; novice mean score =15.8±2.3, P<0.001. Pass/fail standard was set at 22 points.Conclusions: The newly developed assessment tool showed solid evidence of validity and can be used to ensure competence in LAT.

AB - Background: The aims of the study were to develop an assessment tool in local anaesthetic thoracoscopy (LAT), investigate validity evidence, and establish a pass/fail standard.Methods: Validity evidence for the assessment tool was gathered using the unified Messick framework. The tool was developed by five experts in respiratory medicine and medical education. Doctors with varying experience performed two consecutive procedures in a standardized, simulation-based setting using a newly developed thorax/lung silicone model. Performances were video-recorded and assessed by four expert raters using the new tool. Contrasting groups' method was used to set a pass/fail standard.Results: Nine novices and 8 experienced participants were included, generating 34 recorded performances and 136 expert assessments. The tool had a high internal consistency (Cronbach's alpha =0.94) and high inter-rater reliability (Cronbach's alpha =0.91). The total item score significantly correlated with the global score (rs=0.86, P<0.001). Participants' first performance correlated to second performance (test-retest reliability) with a Pearson's r of 0.93, P<0.001. Generalisability (G) study showed a G-coefficient of 0.92 and decision (D) study estimated that one performance assessed by two raters or four performances assessed by one rater are needed to reach an acceptable reliability, i.e., G-coefficient >0.80. The tool was able to discriminate between the two groups in both performances: experienced mean score =30.8±4.2; novice mean score =15.8±2.3, P<0.001. Pass/fail standard was set at 22 points.Conclusions: The newly developed assessment tool showed solid evidence of validity and can be used to ensure competence in LAT.

KW - Assessment

KW - Pleuroscopy

KW - Simulation training

KW - Thoracoscopy

KW - Training

KW - Validation study

UR - http://www.scopus.com/inward/record.url?scp=85111504550&partnerID=8YFLogxK

U2 - 10.21037/jtd-20-3560

DO - 10.21037/jtd-20-3560

M3 - Journal article

C2 - 34422330

VL - 13

SP - 3998

EP - 4007

JO - Journal of Thoracic Disease

JF - Journal of Thoracic Disease

SN - 2072-1439

IS - 7

ER -

ID: 67245020