25714

The Cochrane Collaboration uses the following definition for SRs: “SRs are concise summaries of the best available evidence that seek to collate evidence that fits pre-specified eligibility criteria in order to answer a specific research question, and minimize bias, with systematic methods that are documented in advance” (Chandler et al., 2019, n.p.). SRs are further described as a “review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant research and to collect and analyze data from the studies that are included in the review; statistical methods (meta-analysis) may or may not be used to analyze and summarize the results of the included studies” (Higgins et al., 2019, n.p.).

In short, the SR is a type of evidence summary that uses a rigorous scientific approach to combine results from a body of primary research studies into a clinically meaningful whole (Stevens, 2015). It is essential to evaluate the methodological quality of the SR prior to moving the new evidence into clinical decision-making; reporting guidelines for such appraisals are discussed in this chapter.

As a scientific investigation, the SR focuses on a specific type of research question and uses explicit, transparent methods through each step of identifying, selecting, assessing, and summarizing individual research studies (Haynes et al., 2006; West et al., 2002). Essential aspects of each of these steps in the SR methodology are discussed in this chapter. It is crucial that SR investigative methods be preplanned, transparent, and replicable, as is true in other research designs. Reviews may or may not include a quantitative analysis of the results of selected studies to develop inferences (conclusions) about the population of interest (Institute of Medicine [IOM], 2008, 2011).

■ HIGHLIGHTS OF THE EVOLUTION OF SYSTEMATIC REVIEWS

Because of the relative nascence of SRs as a research design, the health science field uses multiple terms to refer to similar, sometimes overlapping, sometimes less rigorous approaches to summarizing the science on a given topic. These terms include review (used in the medical literature), state-of-the-science review (used in the nursing literature), and review of literature (traditionally used in research methods textbooks). There are, however, important distinctions to be made. Today, the SR is considered to be the most reliable information from research on the benefits and harms of specific interventions, actions, or strategies. All review types other than the SR are considered to be weaker forms of evidence and likely to provide biased conclusions about what 259works in clinical care. Even the traditional review of literature, performed to demonstrate a gap in knowledge, and therefore a need for a research study, has come under scrutiny for lack of rigor (IOM, 2011). Indeed, experts agree that two forms of knowledge point to “what works in healthcare,” that is, systematic reviews and credible clinical practice guidelines (IOM, 2008, 2011).

A high level of scientific rigor in research synthesis is not yet reflected in all published SRs. In seminal research, Mulrow (1987) created a strong case for moving from the then loosely performed “review” in medicine to the more scientifically performed SR. Mulrow’s assessment of 50 “reviews” published in the medical literature showed that the rigor of the reviews was woefully lacking and that, therefore, the conclusions were not trustworthy.

The distinction between SRs, traditional literature reviews, and other types of reviews discussed in this chapter is the strict scientific design that is employed in SRs. As in other research designs, if strict methods are not employed, then the conclusion is called into question for bias and accuracy. That is, the conclusion could be wrong about the efficacy of the intervention, either missing the impact or overestimating the impact on the health outcome. Because clinicians rely on SRs to summarize what is known about a clinical intervention, it is crucial that the results of reviews be highly reliable (IOM, 2011) before application in clinical decision-making. It is equally important to understand what is known before investing additional resources to conduct additional primary research—perhaps over questions for which the answers are already known.

Since Mulrow (1987), several other studies have appraised the quality of reviews in the medical literature. Kelly et al. (2001) conducted a study in which they assessed the quality of SRs in the emergency medicine literature. Likewise, Choi et al. (2001) conducted a critical appraisal of SRs in the anesthesia literature. Dixon et al. (2005) completed a critical appraisal study in which they evaluated meta-analyses in the surgical literature. In each case, the rigor of the published SRs was found to be lacking.

Stevens (2006) demonstrated that SRs published in the nursing literature also reflected a lack of rigor. In a study similar to that of Mulrow’s (1987), SRs were located in nursing journals. Randomly selected articles classified in the Cumulative Index to Nursing and Allied Health Literature (CINAHL) of the publication type “systematic review” were evaluated using the Overview Quality Assessment Questionnaire (OQAQ; Oxman & Guyatt, 1991), a widely used critical appraisal instrument. This study showed that SRs are overclassified in CINAHL, with classification as SR occurring when the article did not specify that the SR methods were used. In addition, 90% of the SRs fell short of the expected level of rigor.

The poignant caution of an early EBP leader drives home the point of the need for rigorous SRs:

More than a decade has passed since it was first shown that patients have been harmed by failure to prepare scientifically defensible reviews of existing research evidence. There are now many examples of the dangers of this continuing scientific sloppiness. Organizations and individuals concerned about improving the effectiveness and safety of healthcare now look to systematic reviews of research—not individual studies—to inform their judgments. (Chalmers, 2005)

Recognizing the poor state of rigor of SRs and the significance of “getting the evidence straight,” the IOM (2008) assessed what is needed to move the synthesis of science forward. Their report acknowledged the great strides made in the new science of SRs. However, it called for more methodological research to produce better SRs. The report suggested that investing in the science of research synthesis will increase the quality and value of evidence in SRs. The IOM committee recommended establishment of EBPs for SRs (IOM, 2008, 2011).

260A primary mover in the field of SRs, the Cochrane Collaboration (Clarke & Chalmers, 2018; Higgins et al., 2019) methodology workgroup continues to evolve methods for conducting SRs. Likewise, the IOM strongly urges continued development of methodological foundations and rigorous standards for SRs.

■ THE NEED FOR SYSTEMATIC REVIEWS

SRs are critical in assisting clinicians, patients, and policy makers to be able to keep up with the hundreds of thousands of new and often conflicting studies published every year. SRs offer a number of advantages to practice and in planning the next primary study. An SR distills a volume of data into a manageable form, clearly identifies cause-and-effect relationships, increases generalizability across settings and populations, reduces bias, resolves complexity and incongruence across single studies, increases rate of adoption of research into care, and offers a basis for ease of update as new evidence emerges (Mulrow, 1994). With such enduring advantages, the need for rigorous execution of SRs is clear.

SRs are a type of secondary research that follows highly rigorous and prescribed methods to produce an unbiased summary of what is known on a particular topic. In science, there is general agreement about a hierarchy of knowledge produced through various methods. In this hierarchy, the SR is considered the most robust, producing the most accurate view of objective truth. That is, SRs are deemed the most reliable form of research that provides conclusions about “what works” in healthcare to produce intended patient outcomes (IOM, 2008, 2011).

Moreover, given their value in determining the state of the science and the rigorous scientific standards now supporting the conduct of SRs, these reviews are considered a research design worthy of specific funding and support. This point was demonstrated in a study of the relative citation impact of study designs in the health sciences (Patsopoulos et al., 2005). The investigators compared the frequency of citation across a variety of research designs (SR, true experiment, cohort, case–control, case report, nonsystematic review, and decision analysis). Meta-analyses were cited significantly more often than all other designs after adjusting for year of publication, journal impact factor, and country of origin. When limited to studies that addressed treatment effects, meta-analyses received more citations than randomized trials (Patsopoulos et al., 2005).

The purpose of an SR is twofold: (a) to indicate what we know about the clinical effectiveness of a particular healthcare process; and (b) to identify gaps in what is known, pointing to a need for further research. So valuable is the SR in setting the stage for further research that leaders have recommended denial of funding of proposals that are not preceded by a SR on the topic (Chalmers, 2005).

Three of the most important reasons for conducting SRs are (a) to reduce the volume of literature that must guide clinical decisions, (b) to reduce bias arising from several sources, and (c) to provide a resolution among single primary studies that draw conflicting conclusions about whether an intervention is effective.

Reducing the Volume of Literature

An oft-cited benefit of a SR is that it reduces a number of single research studies into one, harmonious statement reflecting the state of the science on a given topic. Literally thousands of new health research studies are published weekly. In 2019, MEDLINE alone had 956,390 new citations (National Institutes of Health [NIH] National Library of Medicine [NLM], 2020); this represents 261over 2,600 new articles per day. Others suggest the doubling of global scientific output roughly every 9 years (Van Noorden, 2014). Individual readers are daunted by the challenge of reading and staying abreast of the published literature. The SR offers a solution in that it reduces the world’s scientific literature to a readable summary of synthesized knowledge, ready for use in clinical care.

For example, to demonstrate the value of SRs in informing care, suppose a nursing practice council sets out to improve care with the goal of preventing falls in the elderly. The council chooses to develop an evidence-based approach and searches the literature. A CINAHL search on “falls prevention” yields 5,639 articles. Limiting the search to “research publications” reduces the list to 2,092 articles. Narrowing the search further to “systematic reviews” yields 168 articles and adding the population “elderly” produces one SR. The SRs range in rigor; however, one article on the subject (Gillespie et al., 2009) was published in the Cochrane Database of Systematic Reviews, ensuring that the synthesis was conducted in a highly systematic (scientific) way. This SR report notes that, after searching multiple bibliography databases and screening studies for relevance and quality, the authors included 62 trials involving 21,668 people in the SR (Gillespie et al., 2009). Upon synthesizing effects using meta-analysis, the researchers drew conclusions about interventions that are likely to be beneficial in reducing falls and interventions for which the effectiveness is unknown. Results are expressed in terms of relative risk and confidence intervals. An example of a beneficial intervention is expressed as follows. A significant pooled relative risk (0.86 with a 95% confidence interval of 0.76–0.98) from five studies representing 1,176 participants, suggests the clinical effectiveness of a multidisciplinary multifactorial risk screening and an intervention program for elders with a history of falling or those at high risk to reduce falls (Gillespie et al., 2009).

SRs consolidate research results from multiple studies on a given topic to increase the power of what we know about cause and effect, making an excellent foundation for clinical decision-making.

Avoiding Bias

The term bias refers to a deviation in accuracy of the conclusion of the summarized studies (Higgins et al., 2019). A SR reduces bias and provides a true representation of the scientifically produced knowledge. Common sources of bias in SRs are (a) an incomplete literature search, (b) biased selection of literature, and (c) exclusion of nonpublished literature. Conducting SRs according to a structured scientific approach ensures that a true representation of knowledge is presented.

Resolving Conflicting Results

Rapid growth in the number of healthcare studies has sharpened the need for SRs to assist clinicians, patients, and policy makers in sorting through the confusing and sometimes conflicting array of available evidence. Although one study may conclude that an intervention is effective, a second study may conclude that the intervention offers no advantage over the comparison. Nurses are cautioned that the simplistic approach of comparing the number of studies that favor an intervention to the number that does not will most likely lead to an erroneous conclusion about intervention efficacy. Instead, rigorous methods, such as SRs, are a superior approach to combining results across all studies to come to this conclusion of efficacy. Some studies have larger sample sizes or higher quality methodologies and therefore carry more weight. Some studies of poor quality may be excluded by using preset criteria.

262The growth and maturation of methods and expertise for conducting and using SRs have increased the reliability of evidence for use in making healthcare decisions. It is crucial that nurses apply the accepted principles to conduct rigorous SRs and critically appraise those that are presented in the literature.

■ FUNDAMENTALS OF SYSTEMATIC REVIEWS

Whether they are serving as the lead investigator or as a member of an interprofessional team, nurses should have knowledge and skills related to SRs. Essential competencies for nurses include locating, critically appraising, and conducting SRs (Stevens, 2009).

Two of the primary organizations that have established guidelines for conducting SRs, include the Agency for Healthcare Research and Quality (AHRQ) and the Cochrane Collaboration. The process has been adapted and renamed by other organizations; however, there are commonly accepted principles for conducting a SR.

A rigorous SR includes a detailed description of the approach and parameters used to ensure completeness in identifying the available data, the rationale for study selection, the method of critical appraisal of the primary studies (evidence), and the method of analysis and interpretation. Documentation of each step is requisite and provides the necessary transparency so that the SR may be replicated. It is strongly suggested that persons well versed in SR methods be part of the research team for all SR studies.

The five basic steps listed here should be followed, and the key decisions that constitute each step of the review should be clearly documented (IOM, 2011):

Step 1: Formulate the research question.

Step 2: Construct an analytic (or logic) framework.

Step 3: Conduct a comprehensive search for evidence.

Step 4: Critically appraise the evidence.

Step 5: Synthesize the body of evidence.

Methodologically sound SRs are understood to include a clear basic and clinical hypothesis, a predefined protocol, designation of search strategies and bibliographic resources, a thorough search (regardless of publication status), transparent selection criteria, qualification of studies selected, synthesis of study data and information, relevant summary, and conclusion.

Step 1: Formulate the Research Question

Like other research designs, SRs use specific methods of inquiry to yield new and valid knowledge. The aim of an SR is to create a summary of scientific evidence related to the effects (outcome) produced by a specific action (intervention). Therefore, the research question used in a SR is designed in a very specific way. A well-formulated, clearly defined question lays the foundation for a rigorous SR. The question guides the analytic framework; the overall research design, including the search for evidence; decisions about types of evidence to be included; and critical appraisal of the relevant evidence from single research studies.

The SR research question must define a precise, unambiguous, answerable research question. The mnemonic PICO was devised (Richardson et al., 1995) to reflect the four key elements of the SR question:

1. 263Patient population
   
2. Intervention
   
3. Comparison
   
4. Outcome(s) of interest

The following is an example of a well-stated SR question used to guide a SR on the topic of “Interventions for Substance Use Disorders in Adolescents” (Steele et al., 2020):

What are the effects of behavioral, pharmacologic, and combined interventions (I) compared with placebo (C) or no active treatment (C) for substance use disorders and problematic substance use in adolescents (P) to achieve abstinence, reduce quantity and frequency of use, improve functional outcomes, and reduce substance-related harms (O)? (Steele et al., 2020)

A second question on promoting smoking cessation during pregnancy, in which the comparison condition is implied, is as follows:

What are the effects of smoking cessation programs (I) implemented during pregnancy on the health (O) of the fetus, infant, mother, and family (P)? (Lumley et al., 2004)

Note that, in this example, the implied comparison is the absence of smoking-cessation programs.

The population characteristics, such as age, gender, and comorbidities, usually vary across studies and are likely to be factors in the effect of an intervention. In addition, a given intervention may produce a number of outcomes of interest. The SR question is formulated so that it includes beneficial and adverse outcomes. For example, although prostate cancer treatment reduces mortality, the SR should also examine harmful effects of treatment such as urinary incontinence (IOM, 2008, 2011).

Depending on the specific SR question, different types of primary studies will be of interest. For example, questions about effectiveness of prescription drugs will generate searches for randomized controlled trials (RCTs). On the other hand, a question about the effects of illicit drug use will find no trials that assign one group to such drug use; in this case, the question will generate a search for observational studies that compare the health of otherwise similar groups of users and nonusers.

The SR question is typically formulated during initial literature searches and evolves as the SR team examines background literature. In addition, a broader group of stakeholders is often involved in question formulation. These may include policy makers, managers, health professionals, and consumers (AHRQ, 2005).

Step 2: Construct an Analytic (or Logic) Framework

After stating the SR question, the research team then constructs a framework. This framework maps the relations between the intervention and the outcomes of interest. In the case of the relations between screening and various outcomes as depicted in Figure 14.1, the analytic framework was developed by the U.S. Preventive Services Task Force (USPSTF) to depict causal pathways (USPSTF, 2020).

The analytic framework (Figure 14.1) demonstrates which factors are intermediate to the outcomes of interest and guides the construction of the search.

Stay updated, free articles. Join our Telegram channel

Join