Why most responder analyses are misleading

OBJECTIVE: So-called responder analyses are commonly used in randomized controlled trials (RCT) for osteoarthritis and are typically based on observed change from baseline in self-reported pain. However, it is well known in the methodological literature that such responder analyses are misleading. We aimed to illustrate the size of the problem using simulation.

DESIGN: We generated individual pain trajectories based on real-life assumptions: normal distribution, mean pain 45 on visual analogue scale (VAS, range 0-100), within person standard deviation 12, between person standard deviation 25. Further, we generated plausible data from RCTs with true treatment effect on pain varying from 0 to 15 points on VAS and true proportion of responders 0% or 100%. We applied typical responder analysis to these generated trials.

RESULTS: With natural fluctuations of pain, the observed change in pain from baseline does not equal response to treatment. Even if a treatment is highly effective in reducing pain in all patients (100%) by 15 mm VAS, and no patient (0%) is responder to placebo, a typical responder analysis would suggest that 80% in the active treatment arm compared to 50% of persons in a placebo arm are responders, underestimating both the absolute and relative efficacy/effectiveness of the treatment and falsely implying heterogeneity in treatment effects.

CONCLUSIONS: Responder analysis based on change from baseline in VAS pain should be abandoned in analysis of parallel-group RCTs. Responder criteria based on change from baseline in other fluctuating outcomes, e.g. patients' self-reported symptoms, function and global assessment, should be scrutinized, as they likely share similar limitations."