Inter-grader reliability in the Danish screening programme for diabetic retinopathy

Anne Suhr Thykjaer; Jens Andresen; Nis Andersen; Toke Bek; Steffen Heegaard; Javad Hajari; Caroline Schmidt Laugesen; Sören Möller; Frederik Nørregaard Pedersen; Ryo Kawasaki; Kurt Højlund; Katrine Hass Rubin; Lonny Stokholm; Tunde Peto; Jakob Grauslund

doi:10.1111/aos.15667

Inter-grader reliability in the Danish screening programme for diabetic retinopathy

Anne Suhr Thykjaer^*, Jens Andresen, Nis Andersen, Toke Bek, Steffen Heegaard, Javad Hajari, Caroline Schmidt Laugesen, Sören Möller, Frederik Nørregaard Pedersen, Ryo Kawasaki, Kurt Højlund, Katrine Hass Rubin, Lonny Stokholm, Tunde Peto, Jakob Grauslund

^*Corresponding author for this work

4 Citations (Scopus)

Abstract

PURPOSE: The Danish Registry of Diabetic Retinopathy includes information from >200 000 patients who attends diabetic retinopathy (DR) screening in Denmark. Screening of patients with uncomplicated type 2 diabetes is often performed by practicing ophthalmologists, while patients with type 1 and complicated type 2 diabetes attends screening at hospitals. We performed a clinical reliability study of retinal images from Danish screening facilities to explore the inter-grader agreement between the primary screening ophthalmologist and a blinded, certified grader.

METHODS: Invitations to participate were sent to screening facilities across Denmark. The primary grader uploaded fundus photographs with information on estimated level of DR (International Clinical Diabetic Retinopathy scale as 0 [no DR], 1-3 [mild, moderate or severe nonproliferative DR {NPDR}], or 4 [proliferative DR {PDR}]), region of screening, image style, and screening facility. Images were then regraded by a blinded, certified, secondary grader. Weighted kappa analysis was performed to evaluate agreement.

RESULTS: Fundus photographs from 230 patients (458 eyes) were received from practicing ophthalmologists (52.6%) and hospital-based grading centres (47.4%) from all Danish regions. Reported levels of DR by the primary graders were 66.8%, 12.2%, 13.1%, 1.3% and 5.5% for DR levels 0-4. The overall agreement between primary and secondary graders was 93% (κ = 0.83). Based on screening facility agreement was 96% (κ = 0.89) and 90% (κ = 0.76) for practicing ophthalmologists and hospital-based graders.

CONCLUSION: In this nationwide study, we observed a high overall inter-grader agreement and based on this, it is reasonable to assume that reported DR gradings in the screening programme in Denmark, accurately reflect the truth.

Original language	English
Journal	Acta Ophthalmologica
Volume	101
Issue number	7
Pages (from-to)	783-788
Number of pages	6
ISSN	1755-375X
DOIs	https://doi.org/10.1111/aos.15667
Publication status	Published - Nov 2023

Keywords

Denmark/epidemiology
Diabetes Mellitus, Type 2/complications
Diabetic Retinopathy/diagnosis
Humans
Mass Screening/methods
Photography/methods
Reproducibility of Results
screening
reliability
diabetic retinopathy
interrater agreement

Access to Document

10.1111/aos.15667

Cite this

Thykjaer, A. S., Andresen, J., Andersen, N., Bek, T., Heegaard, S., Hajari, J., Schmidt Laugesen, C., Möller, S., Pedersen, F. N., Kawasaki, R., Højlund, K., Rubin, K. H., Stokholm, L., Peto, T., & Grauslund, J. (2023). Inter-grader reliability in the Danish screening programme for diabetic retinopathy. Acta Ophthalmologica, 101(7), 783-788. https://doi.org/10.1111/aos.15667

Thykjaer, AS, Andresen, J, Andersen, N, Bek, T, Heegaard, S , Hajari, J, Schmidt Laugesen, C, Möller, S, Pedersen, FN, Kawasaki, R, Højlund, K, Rubin, KH, Stokholm, L, Peto, T & Grauslund, J 2023, 'Inter-grader reliability in the Danish screening programme for diabetic retinopathy', Acta Ophthalmologica, vol. 101, no. 7, pp. 783-788. https://doi.org/10.1111/aos.15667

@article{cb28a13d430b43abb5ce64b683048f01,

title = "Inter-grader reliability in the Danish screening programme for diabetic retinopathy",

abstract = "PURPOSE: The Danish Registry of Diabetic Retinopathy includes information from >200 000 patients who attends diabetic retinopathy (DR) screening in Denmark. Screening of patients with uncomplicated type 2 diabetes is often performed by practicing ophthalmologists, while patients with type 1 and complicated type 2 diabetes attends screening at hospitals. We performed a clinical reliability study of retinal images from Danish screening facilities to explore the inter-grader agreement between the primary screening ophthalmologist and a blinded, certified grader.METHODS: Invitations to participate were sent to screening facilities across Denmark. The primary grader uploaded fundus photographs with information on estimated level of DR (International Clinical Diabetic Retinopathy scale as 0 [no DR], 1-3 [mild, moderate or severe nonproliferative DR {NPDR}], or 4 [proliferative DR {PDR}]), region of screening, image style, and screening facility. Images were then regraded by a blinded, certified, secondary grader. Weighted kappa analysis was performed to evaluate agreement.RESULTS: Fundus photographs from 230 patients (458 eyes) were received from practicing ophthalmologists (52.6%) and hospital-based grading centres (47.4%) from all Danish regions. Reported levels of DR by the primary graders were 66.8%, 12.2%, 13.1%, 1.3% and 5.5% for DR levels 0-4. The overall agreement between primary and secondary graders was 93% (κ = 0.83). Based on screening facility agreement was 96% (κ = 0.89) and 90% (κ = 0.76) for practicing ophthalmologists and hospital-based graders.CONCLUSION: In this nationwide study, we observed a high overall inter-grader agreement and based on this, it is reasonable to assume that reported DR gradings in the screening programme in Denmark, accurately reflect the truth.",

keywords = "Denmark/epidemiology, Diabetes Mellitus, Type 2/complications, Diabetic Retinopathy/diagnosis, Humans, Mass Screening/methods, Photography/methods, Reproducibility of Results, screening, reliability, diabetic retinopathy, interrater agreement",

author = "Thykjaer, {Anne Suhr} and Jens Andresen and Nis Andersen and Toke Bek and Steffen Heegaard and Javad Hajari and {Schmidt Laugesen}, Caroline and S{\"o}ren M{\"o}ller and Pedersen, {Frederik N{\o}rregaard} and Ryo Kawasaki and Kurt H{\o}jlund and Rubin, {Katrine Hass} and Lonny Stokholm and Tunde Peto and Jakob Grauslund",

note = "{\textcopyright} 2023 The Authors. Acta Ophthalmologica published by John Wiley & Sons Ltd on behalf of Acta Ophthalmologica Scandinavica Foundation.",

year = "2023",

month = nov,

doi = "10.1111/aos.15667",

language = "English",

volume = "101",

pages = "783--788",

journal = "Acta Ophthalmologica",

issn = "1755-375X",

publisher = "Wiley-Blackwell Munksgaard",

number = "7",

}

TY - JOUR

T1 - Inter-grader reliability in the Danish screening programme for diabetic retinopathy

AU - Thykjaer, Anne Suhr

AU - Andresen, Jens

AU - Andersen, Nis

AU - Bek, Toke

AU - Heegaard, Steffen

AU - Hajari, Javad

AU - Schmidt Laugesen, Caroline

AU - Möller, Sören

AU - Pedersen, Frederik Nørregaard

AU - Kawasaki, Ryo

AU - Højlund, Kurt

AU - Rubin, Katrine Hass

AU - Stokholm, Lonny

AU - Peto, Tunde

AU - Grauslund, Jakob

PY - 2023/11

Y1 - 2023/11

N2 - PURPOSE: The Danish Registry of Diabetic Retinopathy includes information from >200 000 patients who attends diabetic retinopathy (DR) screening in Denmark. Screening of patients with uncomplicated type 2 diabetes is often performed by practicing ophthalmologists, while patients with type 1 and complicated type 2 diabetes attends screening at hospitals. We performed a clinical reliability study of retinal images from Danish screening facilities to explore the inter-grader agreement between the primary screening ophthalmologist and a blinded, certified grader.METHODS: Invitations to participate were sent to screening facilities across Denmark. The primary grader uploaded fundus photographs with information on estimated level of DR (International Clinical Diabetic Retinopathy scale as 0 [no DR], 1-3 [mild, moderate or severe nonproliferative DR {NPDR}], or 4 [proliferative DR {PDR}]), region of screening, image style, and screening facility. Images were then regraded by a blinded, certified, secondary grader. Weighted kappa analysis was performed to evaluate agreement.RESULTS: Fundus photographs from 230 patients (458 eyes) were received from practicing ophthalmologists (52.6%) and hospital-based grading centres (47.4%) from all Danish regions. Reported levels of DR by the primary graders were 66.8%, 12.2%, 13.1%, 1.3% and 5.5% for DR levels 0-4. The overall agreement between primary and secondary graders was 93% (κ = 0.83). Based on screening facility agreement was 96% (κ = 0.89) and 90% (κ = 0.76) for practicing ophthalmologists and hospital-based graders.CONCLUSION: In this nationwide study, we observed a high overall inter-grader agreement and based on this, it is reasonable to assume that reported DR gradings in the screening programme in Denmark, accurately reflect the truth.

AB - PURPOSE: The Danish Registry of Diabetic Retinopathy includes information from >200 000 patients who attends diabetic retinopathy (DR) screening in Denmark. Screening of patients with uncomplicated type 2 diabetes is often performed by practicing ophthalmologists, while patients with type 1 and complicated type 2 diabetes attends screening at hospitals. We performed a clinical reliability study of retinal images from Danish screening facilities to explore the inter-grader agreement between the primary screening ophthalmologist and a blinded, certified grader.METHODS: Invitations to participate were sent to screening facilities across Denmark. The primary grader uploaded fundus photographs with information on estimated level of DR (International Clinical Diabetic Retinopathy scale as 0 [no DR], 1-3 [mild, moderate or severe nonproliferative DR {NPDR}], or 4 [proliferative DR {PDR}]), region of screening, image style, and screening facility. Images were then regraded by a blinded, certified, secondary grader. Weighted kappa analysis was performed to evaluate agreement.RESULTS: Fundus photographs from 230 patients (458 eyes) were received from practicing ophthalmologists (52.6%) and hospital-based grading centres (47.4%) from all Danish regions. Reported levels of DR by the primary graders were 66.8%, 12.2%, 13.1%, 1.3% and 5.5% for DR levels 0-4. The overall agreement between primary and secondary graders was 93% (κ = 0.83). Based on screening facility agreement was 96% (κ = 0.89) and 90% (κ = 0.76) for practicing ophthalmologists and hospital-based graders.CONCLUSION: In this nationwide study, we observed a high overall inter-grader agreement and based on this, it is reasonable to assume that reported DR gradings in the screening programme in Denmark, accurately reflect the truth.

KW - Denmark/epidemiology

KW - Diabetes Mellitus, Type 2/complications

KW - Diabetic Retinopathy/diagnosis

KW - Humans

KW - Mass Screening/methods

KW - Photography/methods

KW - Reproducibility of Results

KW - screening

KW - reliability

KW - diabetic retinopathy

KW - interrater agreement

UR - http://www.scopus.com/inward/record.url?scp=85153227524&partnerID=8YFLogxK

U2 - 10.1111/aos.15667

DO - 10.1111/aos.15667

M3 - Journal article

C2 - 37066883

SN - 1755-375X

VL - 101

SP - 783

EP - 788

JO - Acta Ophthalmologica

JF - Acta Ophthalmologica

IS - 7

ER -

Inter-grader reliability in the Danish screening programme for diabetic retinopathy

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this