Development and validation of a neural network survival prediction model for ischemic heart disease

BACKGROUND: Current risk prediction models for ischemic heart disease in clinical use are relatively simple and use a limited collection of well-known risk factors. Using machine learning to integrate a broader panel of features from electronic health records (EHRs) may improve post-angiography prognostication.

METHODS: This retrospective model development and validation study was based on Danish EHR data. Icelandic EHR data were used for external test. Patients with a coronary angiography-confirmed diagnosis of coronary atherosclerosis between 2006 and 2016 were included for model development (n = 39,746). Time to all-cause mortality, the prediction target, was tracked until 2019, or up to 5 years, whichever came first. To model time-to-event data and deal with censoring, neural network-based discrete-time survival models were used. The model, PMHnet, uses 584 different features including clinical characteristics, laboratory tests, and diagnosis and procedure codes. Model performance was evaluated using time-dependent AUC (tdAUC) and the Brier score. PMHnet was benchmarked against the updated GRACE2.0 risk score and less feature-rich neural network models. Models were evaluated using hold-out data (n = 5000) and external validation data from Iceland. Feature importance and model explainability were assessed using SHAP analysis.

RESULTS: On the test set (n = 5000), the tdAUC of PMHnet was 0.88 [ 0.86-0.90] (case count = 196) at six months, 0.88 [0.86-0.90] (cc = 261) at one year, 0.84 [0.82-0.86] (cc = 395) at three years, and 0.82 [0.80-0.84] (cc = 763) at five years. PMHnet showed similar performance in the Icelandic data. Compared to the GRACE2.0 score and intermediate models limited to GRACE2.0 features or single data modalities, PMHnet had significantly better model discrimination across all evaluated prediction timepoints.

CONCLUSIONS: More complex and feature-rich machine learning models can better predict all-cause mortality in ischemic heart disease and may be used by clinicians and patients to inform and guide treatment and management.