A sensitivity analysis of preprocessing pipelines: Toward a solution for multiverse analyses

Being able to aggregate results from many acceptable data analysis pipelines (multiverse analyses) is a desirable feature in almost all aspects of imaging neuroscience. This is because multiple noise sources may contaminate the acquired imaging data, and different pipelines will attenuate or remove those noise source effects differentially. Here, we used multiple preprocessing pipelines that are known to impact the final results and conclusions of Positron Emission Tomography (PET) neuroimaging studies significantly. We developed conceptual and practical tools for statistical analyses that aggregate pipeline results and a new sensitivity analysis testing for hypotheses across pipelines, such as "no effect across all pipelines" or "at least one pipeline with no effect". The proposed framework is generic and can be applied to any multiverse scenario. Code to reproduce all analyses and figures is openly available, including a step-by-step tutorial, so other researchers can carry out their own multiverse analysis.