GRADE guidance 36: updates to GRADE's approach to addressing inconsistency

Gordon Guyatt, Yunli Zhao, Martin Mayer, Matthias Briel, Reem Mustafa, Ariel Izcovich, Monica Hultcrantz, Alfonso Iorio, Ana Carolina Alba, Farid Foroutan, Xin Sun, Holger Schunemann, Hans DeBeer, Elie A Akl, Robin Christensen, Stefan Schandelmaier


OBJECTIVES: To update previous Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidance by addressing inconsistencies and interpreting subgroup analyses.

STUDY DESIGN AND SETTING: Using an iterative process, we consulted with members of the GRADE working group through multiple rounds of written feedback and discussions at GRADE working group meetings.

RESULTS: The guidance complements previous guidance with clarification in two areas: (1) assessing inconsistency and (2) assessing the credibility of possible effect modifiers that might explain inconsistency. Specifically, the guidance clarifies that inconsistency refers to variability in results, not in study characteristics; that inconsistency assessment for binary outcomes requires consideration of both relative and absolute effects; how to decide between narrower and broader questions in systematic reviews and guidelines; that, with the same evidence, ratings of inconsistency may differ depending on the target of certainty rating; and how GRADE inconsistency ratings relate to a statistical measure of inconsistency I2 depending on the context in which one views results. The second part of the guidance illustrates, based on a worked example, the use of the instrument to assess the credibility of effect modification analyses. The guidance explains the stepwise process of moving from a subgroup analysis to assessing the credibility of effect modification and, if found credible, to subgroup-specific effect estimates and GRADE certainty ratings.

CONCLUSION: This updated guidance addresses specific conceptual and practical issues that systematic review authors frequently face when considering the degree of inconsistency in estimates of treatment effects across studies.

Original languageEnglish
JournalJournal of Clinical Epidemiology
Pages (from-to)70-83
Number of pages14
Publication statusPublished - Jun 2023


  • Humans
  • GRADE Approach
  • Group Processes
  • Systematic Reviews as Topic


Dive into the research topics of 'GRADE guidance 36: updates to GRADE's approach to addressing inconsistency'. Together they form a unique fingerprint.

Cite this