“Research and Evidence-Based Practice

These selection criteria include critical elements for EBP, especially financial support for and outcomes related to research activities. Important research-related outcomes to be documented by agencies for Magnet status include nursing studies conducted and professional publications and presentations by nurses. For each study, the title of the study, principal investigator or investigators, role of nurses in the study, and study status need to be documented (Horstman & Fanning, 2010).

The Quality and Safety Education for Nurses (QSEN, 2013) project was implemented to improve prelicensure nurses’ “knowledge, skills, and attitudes (KSAs) that are necessary to continuously improve the quality and safety of the healthcare systems within which they work.” QSEN competencies were developed in six areas essential for students and registered nurses’ (RNs) practice—patient-centered care, teamwork and collaboration, EBP, quality improvement (QI), safety, and informatics. QSEN competencies were introduced in Chapter 1 and are linked to study findings in all chapters in this text. You can view the competencies on the QSEN Institute website (http://qsen.org/competencies/pre-licensure-ksas). EBP is an important area in your prelicensure education, and educators and students need to work toward achieving the following EBP competencies:

Students and RNs demonstrating these EBP competencies are able to provide quality, safe care to patients and to accomplish the goals outlined for Magnet status. Sherwood and Barnsteiner (2012) provide details on the QSEN competencies and educational experiences to promote the KSAs of prelicensure and graduate nurses. In working toward EBP, students and practicing nurses are encouraged to embrace the benefits of EBP; critically appraise current research evidence; refine agency protocols, algorithms or clinical decision trees, and policies based on current research; use the evidence-based guidelines available; and collect data as needed for research projects.

Did the title indicate if a systematic review or meta-analysis was conducted?

Choi and Hector (2012, p. 188.e21) titled their research synthesis as “Effectiveness of Intervention Programs in Preventing Falls: A Systematic Review of Recent 10 Years and Meta-Analysis.” The authors clearly indicated the types of research syntheses (systematic review and meta-analysis) included in their publication.

Did the abstract include a structured summary of the research synthesis?

Choi and Hector (2012) provided a clear, concise abstract that identified the purpose of the systematic review, data sources, study eligibility criteria, studies selected for review (17 RCTs), types of fall prevention programs, primary outcomes of number of falls and fall rates, critical appraisal of studies, results, conclusions, and implications for practice.

Was a significant, clear clinical question developed to direct the research synthesis?

A systemic review or meta-analysis is best directed by a relevant clinical question that focuses the review process and promotes the development of a quality synthesis of research evidence. One of the most common formats used to develop a relevant clinical question to guide a systematic review is the PICO or PICOS format described in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins & Green, 2008). The PICOS format includes the following elements:

Choi and Hector (2012) noted that falls were the leading cause of injury and deaths among older adults, with the direct medical costs estimated to be over $19.2 billion in 2000. Research syntheses had been conducted on fall prevention programs from the 1990s to 2000, but not for the last decade. What was not known was the effectiveness of the fall prevention interventions studied from 2000 to 2009. The population was older adults, and the intervention was fall prevention programs. The studies reviewed included different types of interventions, with most of the fall prevention programs including multiple approaches, such as individualized exercises for strength, coordination and balance, occupational therapy, home environmental and behavioral assessments, cognition assessment, medical examination, gait stability, and medication review. The intervention group was compared with groups receiving standard care, no treatment, or a variation of the treatment. The primary outcomes measured were number of falls and fall rate. The study design included synthesis of only RCTs using guidelines from the Cochrane Collaboration handbook (Higgins & Green, 2008; see Chapter 8 for a description of RCTs). The study design (RCTs) clearly focused the literature review but might have eliminated some important studies that could have expanded the knowledge related to the intervention, fall prevention programs.

Were the purpose and objectives or aims of the review expressed?

Systematic reviews of research include a purpose and sometimes specific aims or objects to guide the synthesis process (Bettany-Saltikov, 2010a; Rew, 2011). The purpose identifies the major goal or focus of the review. Choi and Hector (2012, p. 188.e13) identified their purpose as follows: “To examine the reported effectiveness of fall-prevention programs for older adults by reviewing randomized controlled trials from 2000 to 2009.” This clearly focused purpose was used to direct their systematic review and meta-analysis, and no additional objectives were identified.

Was the literature search criteria clearly identified?

Research reports of systematic reviews or meta-analyses need to identify the inclusion and exclusion criteria used to direct the literature search (see Table 13-2). The PICOS format might be used to develop the search criteria with more detail being developed for each of the elements. These search criteria might focus on the following: (1) type of research methods, such as quantitative, qualitative, or outcomes research; (2) population or type of study participants; (3) study designs, such as description, correlational, quasi-experimental, experimental, qualitative, or mixed methods; (4) sampling processes, such as probability or nonprobability sampling methods; (5) intervention and comparison interventions; and (6) specific outcomes to be measured. The search criteria also need to indicate the years for the review, language, and publication status (Bettany-Saltikov, 2010b; Higgins & Green, 2008; Rew, 2011).

Often searches have been limited to published sources in common databases, which excludes the grey literature from the research synthesis. Grey literature refers to studies that have limited distributions, such as theses and dissertations, unpublished research reports, articles in obscure journals, articles in some online journals, conference papers and abstracts, conference proceedings, research reports to funding agencies, and technical reports (Benzies, Premji, Hayden, & Serrett, 2006; Conn, Valentine, Cooper, & Rantz, 2003). Most grey literature is difficult to access through database searches and is often not peer-reviewed, with limited referencing information. These are some of the main reasons why grey literature is not included in systematic reviews and meta-analyses. However, excluding grey literature might result in misleading, biased results. Studies with significant findings are more likely to be published than studies with nonsignificant findings and are usually published in more high-impact, widely distributed journals that are indexed in computerized databases (Conn et al., 2003). Studies with significant findings are more likely to have duplicate publications, which should not be included in the systematic review or meta-analysis. More details on identifying publication bias are provided in the next section on critically appraising meta-analyses.

Choi and Hector (2012) designed their literature search strategies and their protocol for conducting their systematic review using sources such as the Cochrane Collaboration handbook (Higgins & Green, 2008) and the PRISMA statement (Liberati et al., 2009; Moher, Liberati, Tetzlaff, Altman, & PRISMA Group, 2009). PICOS format was also implemented with the literature search being directed by population of older adults, intervention of fall prevention programs, comparison of intervention groups with standard care and control groups, outcomes of number of falls and fall rates, and study designs limited to RCTs. The date restriction for the search was 2000 to 2009 based on the lack of research syntheses conducted in the last decade. The search was limited to published studies reported in English, which could result in important studies being omitted from the systematic review.

Was a comprehensive, systematic search of the research literature conducted?

The key search terms, different databases searched, and search results need to be recorded in the systematic review and meta-analysis publications. Sometimes authors provide a table that identifies the search terms and criteria. The PRISMA statement recommends presenting the full electronic search strategy used for at least one major database, such as CINAHL or MEDLINE (Liberati et al., 2009). The search strategies used to identify grey literature and other unpublished studies need to be identified.

Choi and Hector (2012, p. 188.e14) described their search of the literature in the following:

Was publication bias addressed?

Choi and Hector (2012) recognized the publication bias that occurs with using only published sources and not including grey literature. They also limited their search to the English language, resulting in a language bias. They noted that 33 studies were duplicated across the databases, and these were excluded, thus reducing the potential for duplication bias.

Was the process for selecting the studies for review detailed?

The selection of studies for inclusion in a systematic review or meta-analysis is a complex process that initially involves the review and removal of duplicate sources. The abstracts of the remaining studies are reviewed by two or more authors and sometimes by an external reviewer to ensure that they meet the criteria identified in step 5 (see Table 13-2). The abstracts might be excluded based on the study participants, interventions, outcomes, or design not meeting the search criteria (Bettany-Saltikov, 2010b; Higgins & Green, 2008; Liberati et al., 2009). After the abstracts meeting the designated criteria are identified, the next step is to retrieve the full-text citation for each study. If studies do not meet criteria, they need to be removed, with a rationale provided. The study selection process is best demonstrated by a flow diagram that was developed by the PRISMA Group (Liberati et al., 2009). Figure 13-1 shows this diagram, which has four phases: (1) identification of the sources; (2) screening of the sources based on set criteria; (3) determining if the sources meet eligibility requirements; and (4) identifying the studies included in the review.

Choi and Hector (2012) provided a description of their selection of sources and a flow diagram that documented the final results of the 17 RCTs included in their systematic review, with a meta-analysis. The following excerpt identifies the steps that they took to select their studies for inclusion in their research synthesis:

Were key elements of the studies presented?

Critical appraisals of studies for systematic reviews and meta-analyses are best done by constructing a table describing the characteristics of the included studies, such as the purposes of the studies, populations, sampling processes, interventions, outcomes, and results (Bettany-Saltikov, 2010b; Higgins & Green, 2008; Liberati et al., 2009). Choi and Hector (2012) detailed the key elements of the 17 studies in a table. The table included authors of the study, year, sample size of control and intervention groups, intervention and follow-up period, intervention model, setting, mean age of subjects, and brief description of the intervention programs. The table would have been stronger if the sampling methods, outcomes measured, and key results for the 17 studies had been included in a table format.

Were the studies critically appraised?

Two or more experts need to review the studies independently and make judgments about their quality. The critical appraisal of the studies is often difficult because of the differences in types of participants, designs, sampling methods, intervention protocols, outcome variables and measurement methods, and presentation of results. The studies are often rank-ordered based on their quality and contribution to the development of the review (Bettany-Saltikov 2010b; Liberati et al., 2009). Choi and Hector’s (2012) critical appraisal of the 17 studies involved examining the sample characteristics in the studies, types of interventions, settings of interventions, and intensity of the intervention. The final results of the studies were pooled using a meta-analysis.

Was a meta-analysis conducted as part of the systematic review?

Some authors conduct a meta-analysis in the synthesis of sources for their systematic review (Liberati et al., 2009). Because a meta-analysis involves the use of statistics to summarize results of different studies, it usually provides strong, objective information about the effectiveness of an intervention or solid knowledge about a clinical problem. The authors of the review need to provide a rationale for conducting the meta-analysis and detail the process that they used to conduct this analysis. For example, a meta-analysis might be conducted on a small group of similar studies to determine the effect of an intervention. The systematic review conducted by Choi and Hector (2012) included a meta-analysis of the 17 studies and provided a detailed discussion of the rationale for the meta-analysis and results for this procedure. The next section provides more details on conducting a meta-analysis.

Were the results of the review clearly presented?

The results of a systematic review and meta-analysis should include a description of the study participants, types of interventions implemented in the studies, outcomes measured, and measurement methods. The results of the different types of intervention might be best summarized in a table that includes the following: (1) study source; (2) structure of the intervention (stand-alone or multifaceted); (3) specific type of intervention (e.g., physiological treatment, education, counseling, or behavioral therapy); (4) delivery method (e.g., demonstration and return demonstration, verbal, video, or self-administered); (5) length of time the intervention is implemented; and (6) statistical difference between the intervention and control, standard care, placebo, or alternative intervention groups (Liberati et al., 2009).

The specific outcomes, including primary and secondary outcomes, of the studies might also be best summarized in a table. This table might include (1) the study source, (2) outcome variable(s), with an indication as to whether it was a primary or secondary outcome in the study, (3) measurement method used for each study outcome variable, and (4) quality of the measurement methods, such as the reliability and validity of a scale or the precision and accuracy of a physiological measure.

Choi and Hector (2012) included a description of the 17 fall prevention interventions in a table indicating that most of the interventions were multifactorial rather than an individual action. The interventions and their follow-up period had to be at least 5 months. The authors indicated that the primary outcomes of number of falls and fall rates were measured in different ways in the studies, which limited the quality of the results and findings of the systematic review and meta-analysis. The following excerpt presents the key results from the systematic review and meta-analysis:

Did the report conclude with a clear discussion section?

In a systematic review or meta-analysis, the discussion of the findings includes an overall evaluation of the types of interventions implemented and the outcomes measured. You can also expect the methodological issues or limitations of the review to be addressed. A quality discussion section explicitly connects the findings to the study’s framework to identify the theoretical implications of the findings. Finally, the discussion section needs to provide recommendations for further research, practice, and policy development (Bettany-Saltikov, 2010b; Higgins & Green, 2008; Liberati et al., 2009).

Choi and Hector (2012) provided a discussion of their findings, limitations, and recommendations for research and practice. The report would have been strengthened by including a framework and linking the findings back to the framework to indicate current knowledge regarding the effect of fall prevention programs on number of falls and fall rates. The following excerpt summarizes the key aspects of the discussion section of their report:

The limitations included studies with small sample sizes, varying types of fall prevention interventions, and measurements of outcomes of number of falls and fall rate in various ways. Several of the 17 studies concluded that the intervention made no significant difference. They also noted that the search of the literature might have been more rigorous in identifying additional studies.

Implications for practice

Systematic reviews and meta-analyses provide important, evidence-based knowledge for use in practice. Choi and Hector’s (2012) recommendations provide clear evidence-based interventions that students and RNs might use to reduce the falls in their agencies and institutions. The QSEN (2013) implication is that critically appraising systematic reviews and using relevant evidence in practice is essential for achieving EBP.

Was a clear concise report developed for publication?

The systematic review or meta-analysis report needs to include the content discussed in the previous 13 steps. You can use Table 13-2 when critically appraising a systematic review, indicate if the step is present, and comment about its quality with, supporting rationale. In summary, Choi and Hector (2012) developed a strong systematic review and meta-analysis for publication. The title clearly indicates the types of syntheses conducted. The clinical question addressed followed the PICOS format, and the purpose of the synthesis is clearly focused. The search of the literature might have been more rigorous and included additional studies, especially grey literature. The selection of studies for the synthesis was clearly presented in a flow chart and documented with rationale. The studies selected for the systematic review and meta-analysis were critically appraised, and the results from these syntheses were clearly presented in tables and narrative. The publication concluded with relevant findings, limitations, and recommendations for research and practice.