External validation of an AI-Based fracture detection tool for hip and pelvic radiographs in a multicenter retrospective cohort

OBJECTIVE: Artificial intelligence tools show promise in fracture detection but may be impaired by hidden stratification. We aim to evaluate the diagnostic accuracy of a commercially available AI tool for detecting hip and pelvic fractures, with emphasis on clinically relevant subgroups and AO/OTA fracture classifications.

MATERIALS AND METHODS: This retrospective multicenter diagnostic test accuracy study included consecutive trauma patients who underwent hip or pelvic radiography. The reference standard was based on post-conference clinical radiology reports, incorporating MRI, CT, and radiography in hierarchical order. Fractures were classified according to the AO/OTA system. Studied subgroups included surgical metal, degenerative disease, old fractures, radiographically occult fractures and fracture classification. Sensitivity and specificity were calculated with 95% confidence intervals.

RESULTS: Among 642 patients (median age 82 years), 262 (42%) had fractures. Overall sensitivity was 87% [83-91%] and specificity 86% [82-89%]. Specificity was reduced in cases with old fractures (29% [13-51%]). Sensitivity was high for femoral neck (95% [88-98%]) and trochanteric fractures (92% [82-97%]) and moderate for pelvic (82% [70-91%]) and acetabular fractures (58% [28-85%]). Within each segment the classifications with most missed fractures were unilateral anterior pelvic arch (61A2.2), subcapital femur neck (31B1), and simple trochanteric fractures (31A1).

CONCLUSION: In summary, while specificity was notably reduced in cases with old fractures and sensitivity was moderate for pelvic and acetabular fractures, the AI tool demonstrated high diagnostic accuracy for hip-region trauma radiographs, with most missed cases occurring among the subtle AO/OTA fracture groups.