Artificial Intelligence-Based ChatGPT Responses for Patient Questions on Optic Disc Drusen

INTRODUCTION: Optic disc drusen (ODD) are acellular deposits in the optic nerve head, which are most often benign and asymptomatic. Patients may develop visual field defects and be at increased risk of ischemic co-morbidities. As ODD can be difficult to distinguish from papilledema, patients are at risk of unnecessary clinical workups. Patient information is a key aspect of ODD management. In this study, we explored the accuracy of ChatGPT responses for typical patient questions on ODD.

METHODS: Two content experts reached consensus on 20 typical patient questions. We retrieved five separate responses for each question from ChatGPT, totaling 100 responses. Each response was evaluated on a 5-point Likert-scale on accuracy by each content expert in an individual fashion.

RESULTS: The two experts were in fair/substantial agreement in the evaluation of responses (Cronbach's alpha: 0.64). Of the 100 responses, 17 were relevant and without any inaccuracies, 78 were relevant and with inaccuracies not being harmful, and five were relevant and with inaccuracies potentially harmful. The lowest accuracy scores were obtained for questions dealing with treatment and prognosis.

CONCLUSIONS: ChatGPT often provides relevant answers for patient questions on ODD, but inaccuracies become potentially harmful when questions deal with treatment and prognosis.