Simulation Studies for the evaluation of health information technologies: Experiences and results

Elske Ammenwerth, Werner O Hackl, Kristine Binzer, Tue E H Christoffersen, Sanne Jensen, Kitta Lawton, Peter Skjoet, Christian Nohr

    34 Citations (Scopus)


    It is essential for new health information technologies (IT) to undergo rigorous evaluations to ensure they are effective and safe for use in real-world situations. However, evaluation of new health IT is challenging, as field studies are often not feasible when the technology being evaluated is not sufficiently mature. Laboratory-based evaluations have also been shown to have insufficient external validity. Simulation studies seem to be a way to bridge this gap. The aim of this study was to evaluate, using a simulation methodology, the impact of a new prototype of an electronic medication management system on the appropriateness of prescriptions and drugrelated activities, including laboratory test ordering or medication changes. This article presents the results of a controlled simulation study with 50 simulation runs, including ten doctors and fi ve simulation patients, and discusses experiences and lessons learnt while conducting the study. Although the new electronic medication management system showed tendencies to improve medication safety when compared with the standard system, this tendency was not significant. Altogether, fi ve distinct situations were identifi ed where the new medication management system did help to improve medication safety. This simulation study provided a good compromise between internal validity and external validity. However, several challenges need to be addressed when undertaking simulation evaluations including: preparation of adequate test cases; training of participants before using unfamiliar applications; consideration of time, effort and costs of conducting the simulation; technical maturity of the evaluated system; and allowing adequate preparation of simulation scenarios and simulation setting. Simulation studies are an interesting but time-consuming approach, which can be used to evaluate newly developed health IT systems, particularly those systems that are not yet suffi ciently mature to undergo field evaluation studies.
    Original languageEnglish
    JournalHealth Information Management Journal
    Issue number2
    Pages (from-to)14-21
    Number of pages8
    Publication statusPublished - 2012


    Dive into the research topics of 'Simulation Studies for the evaluation of health information technologies: Experiences and results'. Together they form a unique fingerprint.

    Cite this