Assessing the utility of natural language processing for detecting postoperative complications from free medical text

Emilie Even Dencker, Alexander Bonde, Anders Troelsen, Martin Sillesen*

*Corresponding author af dette arbejde

Abstract

BACKGROUND: Postoperative complication rates are often assessed through administrative data, although this method has proven to be imprecise. Recently, new developments in natural language processing have shown promise in detecting specific phenotypes from free medical text. Using the clinical challenge of extracting four specific and frequently undercoded postoperative complications (pneumonia, urinary tract infection, sepsis, and septic shock), it was hypothesized that natural language processing would capture postoperative complications on a par with human-level curation from electronic health record free medical text.

METHODS: Electronic health record data were extracted for surgical cases (across 11 surgical sub-specialties) from 18 hospitals in the Capital and Zealand regions of Denmark that were performed between May 2016 and November 2021. The data set was split into training/validation/test sets (30.0%/48.0%/22.0%). Model performance was compared with administrative data and manual extraction of the test data set.

RESULTS: Data were obtained for 17 486 surgical cases. Natural language processing achieved a receiver operating characteristic area under the curve of 0.989 for urinary tract infection, 0.993 for pneumonia, 0.992 for sepsis, and 0.998 for septic shock, whereas administrative data achieved a receiver operating characteristic area under the curve of 0.595 for urinary tract infection, 0.624 for pneumonia, 0.571 for sepsis, and 0.625 for septic shock.

CONCLUSION: The natural language processing approach was able to capture complications with acceptable performance, which was superior to administrative data. In addition, the model performance approached that of manual curation and thereby offers a potential pathway for complete real-time coverage of postoperative complications across surgical procedures based on natural language processing assessment of electronic health record free medical text.

OriginalsprogEngelsk
Artikelnummerzrae020
TidsskriftBJS open
Vol/bind8
Udgave nummer2
ISSN2474-9842
DOI
StatusUdgivet - 1 mar. 2024

Fingeraftryk

Dyk ned i forskningsemnerne om 'Assessing the utility of natural language processing for detecting postoperative complications from free medical text'. Sammen danner de et unikt fingeraftryk.

Citationsformater