Developing and Evaluating Clinical Practice Guidelines: A Systematic Approach   1  

Rona F. Levin and Susan Kaplan Jacobs

OVERVIEW

Clinical decision making that is grounded in the best available evidence is essential to promoting patient safety and quality health care outcomes. With the knowledge base for geriatric nursing rapidly expanding, assessing geriatric clinical practice guidelines (CPGs) for their validity and incorporation of the best available evidence is critical to the safety and outcomes of care. In the second edition of this book, Lucas and Fulmer challenged geriatric nurses to take the lead in the assessment of CPGs, recognizing that, in the absence of best evidence, guidelines and protocols have little value for clinical decision making (Lucas & Fulmer, 2003). In the third edition of this book Levin, Singleton, and Jacobs (2008) proposed a method for ensuring that the protocols included here were based on a systematic review of the literature and synthesis of best evidence.

The purpose of this chapter is to describe the process that was used to create the fourth and current fifth edition of Evidence-Based Geriatric Nursing Protocols for Best Practice. Before the third edition of this book, there was no standard process or specific criteria for protocol development, nor was there any indication of the “level of evidence” of each source cited in a chapter (i.e., the evidence base for the protocol). In the third and fourth editions of this book, the process previously used to develop the geriatric nursing protocols was enhanced and described in detail. That process differed from the procedures followed in the current edition. This chapter is a guide to understanding how the protocols contained in this book were developed, and it details how to use a systematic, efficient, and evidence-based approach to discovering and evaluating evidence, which is the process needed to guide the assessment, development, and updating of practice protocols in any area of nursing practice.

DEFINITION OF TERMS

Evidence-based practice (EBP) is a framework for clinical decision making that uses (a) the best available evidence, (b) the clinician’s expertise, and (c) a patient’s values and circumstances to guide judgments about a patient’s personal health condition (Keefer & Levin, 2013; Melnyk & Fineout-Overholt, 2011; Straus, Glasziou, Richardson, & Haynes, 2010). Health care professionals often use the terms recommendations, guidelines, and protocols interchangeably but they are not synonymous.

A recommendation is a suggestion for practice, not necessarily sanctioned by a formal, expert group. A CPG is an “official recommendation” or suggested approach to diagnose and manage a broad health condition or problem (e.g., heart failure, smoking cessation, or pain management). A protocol is a more detailed guide for approaching a clinical problem or health condition and is tailored to a specific practice situation. For example, guidelines for falls prevention recommend developing a protocol for toileting elderly, sedated, or confused patients (National Guideline Clearinghouse, 2013). The specific practices or protocols that each health care organization implements, however, are agency specific. The validity of any of these practice guides can vary depending on the type and the level of evidence on which they are based. Using standard criteria to develop or refine CPGs or protocols assures reliability of their content. Standardization gives both nurses, who use the guideline/protocol, and patients, who receive care based on the guideline/protocol, assurance that the geriatric content and practice recommendations are based on the best evidence.

In contrast to these practice guides, “standards of practice” are not specific or necessarily evidence based; rather, they are a generally accepted, formal, and published framework for practice. As an example, the American Nurses Association document, Nursing: Scope and Standards of Practice, contains a standard regarding nurses’ accountability for making an assessment of a patient’s health status (American Nurses Association, 2010). The standard is a general statement. A protocol, on the other hand, may specify the measurement tool(s) to use in that assessment—for example, STRATIFY, an instrument used to measure the risk of falls (Smith, Forster, & Young, 2006).

The AGREE (Appraisal of Guidelines for Research and Evaluation) and AGREE II Instruments

The AGREE instrument, originally created and evaluated by a team of international guideline developers and researchers for use by the National Health Service (AGREE Enterprise, 2003), has been revised and updated and remains a generic tool designed primarily to help guideline developers and users assess the methodological quality of guidelines (Brouwers et al., 2010). This appraisal includes evaluation of the methods used to develop the CPG, assessment of the validity of the recommendations made in the guideline, and consideration of factors related to the use of the CPG in practice. Although the AGREE instrument was created to critically appraise CPGs, the process and criteria can also be applied to the development of clinical practice protocols. Thus, the AGREE instrument has been expanded for that purpose to standardize the creation and revision of the geriatric nursing practice protocols in this book.

The initial AGREE instrument and the one used for clinical guideline/protocol development in the third edition of this book has six quality domains: scope and purpose, stakeholder involvement, rigor of development, clarity and presentation, application, and editorial independence. A total of 23 items divided among the domains were rated on a four-point Likert-type scale from “strongly disagree” to “strongly agree.” Appraisers evaluate how well the guideline they are assessing meets the criteria (i.e., items) of the six quality domains. For example, when evaluating the rigor of development, appraisers rated seven items. The reliability of the AGREE instrument is increased when each guideline is appraised by more than one appraiser. Each of the six domains receives an individual domain score and, based on these scores, the appraiser subjectively assesses the overall quality of a guideline.

Important to note, however, is that the original AGREE instrument was revised in 2009 (AGREE Next Steps Consortium, 2013), is now called AGREE II, and is the version that we used for the fourth and fifth editions of this book. The revision added one new item to the rigor of development domain. This is the current item 9, which underscores the importance of evaluating the evidence that is applied to practice. Item 9 reads: “The strengths and limitations of the body of evidence are clearly described” (Table 1.1). The remainder of the changes included a revision of the Likert-type scale used to evaluate each item in the AGREE II, a reordering of the number assigned to each item based on the addition of the new item 9, and minor editing of items for clarity. No other substantive changes were made. Table 1.1 includes the items that are in the rigor of development domain and were used for evaluation of evidence in the current edition of this book. A 2013 update of the AGREE II instrument includes a history of the project, information about language translations, and enhanced online training tools freely available to support guideline developers (AGREE Enterprise, 2014).

The rigor of development section of the AGREE instrument provides standards for literature searching and documenting the databases and terms searched. Adhering to these criteria to find and use the best available evidence on a clinical question is critical to the validity of geriatric nursing protocols and ultimately to patient safety and outcomes of care.

Published guidelines can be appraised using the AGREE II instrument, as discussed previously. In the process of guideline development, however, the clinician is faced with the added responsibility of appraising all available evidence for its quality and relevance. In other words, how well does the available evidence support recommended clinical practices? The clinician needs to be able to support or defend the inclusion of each recommendation in the protocol based on its level and quality of evidence. To do so, the guideline must reflect a systematic, structured approach to find and assess the available evidence.

TABLE 1.1

Sample Domain and Items From the AGREE II Instrument for Critical Appraisal of Clinical Practice Guidelines

AGREE, Appraisal of Guidelines for Research and Evaluation.

Reprinted from the AGREE II Instrument by permission of Melissa Brouwers. www.agreetrust.org

Searching for the Best Evidence

Models of EBP describe the evidence-based process in five steps (Melnyk & Fineout-Overholt, 2011; Titler, 2010):

Although the evidence-based process encompasses these five steps, for the purposes of this volume of protocols and their development, this chapter focuses on the first three steps in more detail.

Step 1: Develop an Answerable Question

Developing an answerable question is critical before one can choose relevant sources to search. The information needed may be in the form of a specific “foreground” question (one that is focused on a particular clinical issue) or it may be a broad question (one that asks for overview information about a disease, condition, or aspect of health care) (Melnyk & Fineout-Overholt, 2011; Straus, Glasziou, Richardson, & Haynes, 2010) to gain some background of the practice problem and interventions, and gain insight into its significance. Background information includes both internal data from a specific agency and external data to place the health condition or problem in a broader societal context. Internal data usually include quality metrics from the health care agency in conjunction with health care providers’ observations. External data might require a search for local and/or national benchmarking data and prevalence statistics as well as general literature describing the local problem as one that goes beyond a specific health care setting, population, or intervention.

An example of a background query might be one that seeks data: What is the prevalence of falls in elderly residents in a long-term care facility? Should these data demonstrate an unacceptable fall rate compared to national benchmark and safety target statistics, then the local problem can be shown to have significance beyond the specific clinical agency. A broad research query (an example of an overarching background question) related to a larger category of disease or health problem and encompassing multiple interventions might be: What is the best evidence for fall prevention in hospitalized older adults? A first place to search for evidence would be the National Guideline Clearinghouse (http://guidelines.gov) as described in Table 1.2).

A related question—What is the best evidence for falls prevention for the elderly in hospitals and long-term care facilities?—is addressed in a systematic review from the Cochrane Library, cited in Table 1.2. The Cochrane Library of Systematic Reviews contains rigorous and comprehensive narrative and statistical (meta-analyses) reviews that synthesize multiple studies of interventions (Cameron et al., 2012). The information contained in this review synthesizes multifactorial interventions and may help to further focus the inquiry into a question about the effectiveness of a specific intervention.

A similar example in Table 1.2 cites a Joanna Briggs Institute evidence summary, which answers a general background or overview question: What is the evidence regarding specific interventions to prevent falls in older adults? (Slade, 2013).

Once the overall evidence regarding a background question is uncovered, the question can be narrowed into a specific “PICO” format to specify the intervention or assessment tool being examined (Straus et al., 2010, p. 15). PICO stands for:

Stay updated, free articles. Join our Telegram channel

Join