Christopher R. Carpenter, Lucas Oliveira J. e Silva, Suneel Upadhye, Joshua Broder ¡ 5 authors
Emergency medicine is often a specialty defined by diagnostic uncertainty when worried patients present with constellations of symptoms seeking explanation and relief. Abdominal pain is a common chief complaint among adult emergency department (ED) patients, with recurrent symptoms in the subsequent days, weeks, months, and even years, sometimes prompting repeat evaluations.1 The differential diagnosis is broad and diverse, including multiple organs and systems and extraabdominal causes. Ideally, clinical practice guidelines (CPGs) synthesize the entirety of evidence for questions relevant to an explicitly defined patient population and outcomes, but until now no CPG existed for the scenario of recurrent abdominal pain. Consequently, significant practice variation exists in the diagnostic and therapeutic approach to this clinical condition.2 The Society for Academic Emergency Medicine (SAEM) second "Guidelines for Reasonable and Appropriate Care in the Emergency Department 2 (GRACE-2)" article provides that CPG with adherence to Grading of Recommendations, Assessment, Development and Evaluations (GRADE) methodology including incorporation of patient priorities and external stakeholders.3 Through adherence to the GRADE methodology we aimed to create rigorous and trustworthy guidelines. The GRACE-2 writing team deliberated to select topics and questions and to explicitly define a clinically meaningful population, ultimately settling on definitions of recurrence within 30 days and adults with "low-risk" abdominal pain. Identifying no well-accepted or validated definition of "low risk" (as opposed to a risk model like the HEART score for chest pain4), the GRACE-2 writing team devised a definition of "low risk" that resonated with our clinical intuition and excluded populations that emergency physicians would routinely identify as moderate or high risk. Subsequently, the GRACE-2 writing team worked with medical librarians to focus searches based on the patient-intervention-control-outcome-time (PICOT) template5 and completed systematic reviews for each question before developing the recommendations.6, 7 The PICOT-oriented literature search revealed no studies that aligned with our definition of low risk and few that defined recurrence within our predefined time frame. However, that does not mean that our search rendered zero published evidence around which GRACE-2 could contemplate actionable recommendations via the GRADE Evidence to Decision (EtD) framework.8, 9 We had to make a decision about how to classify and incorporate the published research that we did identify. GRADE provides a framework for evidence synthesis and development of clinical guidelines and recommends inclusion of both direct and indirect evidence into CPGs, while providing guidance around the distinction between the two.8-12 Therefore, we decided to classify evidence as "direct" if each element of the PICOT question matched the study's inclusion criteria and outcomes assessed. "Indirect" evidence was defined by deviation from any component of the PICOT question (Table 1). These definitions are necessary for guideline developers who need to evaluate the domain of indirectness when rating the certainty of evidence.10, 12 Since the overall value of CPGs rests upon the rigor and transparency of the evidentiary search, quality assessment, and synthesis in conjunction with an explicit and representative assessment of anticipated benefits or potential harms of the subsequent recommendations, weighing the pros and cons of including indirect evidence is merited while considering if and how to incorporate the GRACE-2 recommendations into ED practice. Not emergency department Not adult Not recurrent Not undifferentiated abdominal pain Not low risk Initial CT identified explanatory pathology like kidney stone Repeat CT >12 months after initial CT GRADE recognizes indirectness as a key domain in the assessment of certainty of evidence.10, 12 Whenever systematic review authors or guideline developers identify important indirectness issues from a body of evidence that deviates from the original PICOT question, the certainty of evidence must be downgraded by one or two levels.10, 12 Despite such guidance from GRADE, conceptualization of what type of indirect evidence could be synthesized and used by CPGs remains debatable, especially when there is insufficient direct evidence.13 Also, a second layer of complexity is added when guideline developers need to evaluate the directness of research evidence for all criteria of the GRADE EtD framework including values, resources, cost-effectiveness, equity, acceptability, and feasibility. This cognitive framework can quickly diffuse into uncertainties. Nonetheless, indirect evidence is not fundamentally or inevitably flawed. For example, a study might be rated as indirect because the age range of enrolled subjects does not perfectly match the intended population or because the time frame of follow-up slightly exceeds the desired target. Such evidence may provide a very reasonable estimate of the range of expected outcomes in the intended population. In addition, many randomized controlled trials employ so many exclusion criteria and carefully controlled experimental conditions that the results may not predict outcomes in a broader ED population. GRADE attempts to distill value from the broad range of available evidence, rather than taking a nihilistic attitude that rejects evidence on the basis of any imperfection. Nihilism suggests "we know nothing" and have no basis for any decisionâbut emergency medicine requires that physicians make the best decision possible based on the available, if imperfect, evidence in a more pragmatic approach. GRADE recommends that if a large body of indirect evidence that can be convincingly linked to the PICOT support the management strategy, recommendations should be made.14 Specifically, GRADE states that "clinicians will rarely explore the evidence as thoroughly as a guideline panel, nor devote as much thought to the trade-offs, or the possible underlying values and preferences in the population. We therefore encourage panels to deal with their discomfort and to make recommendations even when confidence in effect estimate is low and/or desirable and undesirable consequences are closely balanced."15 The ideal strategy to dealing with absent direct evidence for a certain question during CPG development is not fully established among guideline developers, and each panel along with their methodologists need to individualize such decisions according to resource availability (e.g., methodological expertise, funding) and feasibility. Murad et al.,13 for example, suggest five strategies for supplementing systematic review findings when evidence on benefits or harms is found to be insufficient, including: (1) reconsider eligible study designs, (2) summarize indirect evidence, (3) summarize contextual and implementation evidence, (4) consider modeling, and (5) incorporate unpublished health system data in the evidence synthesis. In GRACE-2, summary of indirect evidence was chosen as the main approach and different "bodies of indirect evidence" were systematically synthesized.6, 7 As three out of four questions in GRACE-2 were related to diagnostic tests, we also faced the reality that direct evidence evaluating the impact of different testing strategies on patient-important outcomes (i.e., diagnostic randomized trials) seldom exists, which ultimately leads to the use of different types of indirect evidence such as diagnostic accuracy.16-18 Even within the adaptive framework of GRADE, indirect evidence may leave the PICOT question unanswered, and subsequent recommendations are often necessarily weak or nonexistent. Strong and definitive recommendations necessitate multiple studies evaluating the identical patient population and diagnostic approach quantifying the same outcomes in similar time frames with minimal concerns about health inequities, resource consumption, imprecision, or feasibility. Indirect evidence alone is unlikely to justify strong recommendations, but does provide substantive proof of the scope of the problem relative to the paucity of evidence. If the indirect evidence was excluded, CPG stakeholders would not be cognizant that this research was identified and reviewed in developing the recommendations. Clinicians, educators, and researchers would remain unaware of how little empiric evidence exists around which to shape decision making or how future investigators could more directly address the knowledge void. GRACE-2 is not alone in identifying a painfully surprising gap between what emergency medicine thinks is common knowledge and high-quality research to justify those beliefs. One reflection of this is that the majority of American College of Emergency Physician Clinical Policy recommendations are not level A.19 Ultimately, concerns about the directness of evidence for CPGs represent an existential crisis for emergency medicine. As society's safety net for potentially life-threatening medical, surgical, or psychiatric illness, emergency medicine's breadth of knowledge must remain broad and open-ended as clinical science continues to expand the horizons of possibility. The hierarchy of evidence-based medicine places CPGs at the top of the information pyramid, yet guidelines for common syndromic presentations like acute abdominal pain do not exist in emergency medicine. Certainly, our specialty could await others to create CPGs for these conditions, but those organizations are most likely to view patient encounters through the lens of an established or highly suspected diagnosis. In contrast, emergency physicians confront undifferentiated patients with constellations of signs and symptoms in a chaotic environment. We navigate the challenges of accurate and timely diagnosis, often with imperfect data. Paradoxically, we face expectations of constraint with testingâoften without evidence-based guidance for when to safely limit workupsâwhile avoiding critical misses. Experience with other organization's CPGs has shown that ED clinicians are often noncompliant with their recommendations, which commonly do not account for the real-life conditions of emergency care. The consequence is an unfair judgment that emergency physicians engage in inferior, nonâevidence-based, and excessively costly health care by external stakeholders.20-23 The Association of Academic Chairs of Emergency Medicine's 2030 research goals focus on increasing the proportion of NIH R01-funded emergency medicine investigators, who will someday close the knowledge gap for prevalent conditions that currently remain underinvestigated.24 The absence of direct evidence and the imperfection of indirect evidence for high-priority emergency medicine CPGs illustrates the importance of forming a National Institute of Emergency Care. In addition to supporting the next generation of independent health-outcomes researchers, a central prioritizing body could ensure that the most pertinent questions affecting patient care on a daily basis are the focus of funding opportunities.19, 25 Until that day, GRACE-2 provides a synthesis of evidence and corresponding recommendations derived using GRADE methodology upon which to guide imaging and therapy decisions for adults with low-risk and recurrent abdominal pain. Medicine is equal parts science and art with clinical judgment based on hermeneutic thinking to generate a holistic understanding of an individual patient's current condition.26 Our vision is for the inclusion of direct and indirect evidence in this CPG to ground that decision making in a realistic understanding of our current state of knowledge, while catalyzing more pertinent research in the near future.