On feature relevance in image-based prediction models: An empirical study

Determining disease-related variations of the anatomy and function is an important step in better understanding diseases and developing early diagnostic systems. In particular, image-based multivariate prediction models and the "relevant features" they produce are attracting attention from the community. In this article, we present an empirical study on the relevant features produced by two recently developed discriminative learning algorithms: neighborhood approximation forests (NAF) and the relevance voxel machine (RVoxM). Specifically, we examine whether the sets of features these methods produce are exhaustive; that is whether the features that are not marked as relevant carry disease-related information. We perform experiments on three different problems: image-based regression on a synthetic dataset for which the set of relevant features is known, regression of subject age as well as binary classification of Alzheimer's Disease (AD) from brain Magnetic Resonance Imaging (MRI) data. Our experiments demonstrate that aging-related and AD-related variations are widespread and the initial sets of relevant features discovered by the methods are not exhaustive. Our findings show that by knocking-out features and re-training models, a much larger set of disease-related features can be identified.