Step 1: Formulate a Relevant Clinical Question to Direct the Review

A systemic review or meta-analysis is best directed by a relevant clinical question that focuses the review process and promotes the development of a quality synthesis of research evidence. Formulating the question involves identifying a relevant topic, developing a question of interest that is worth investigating, deciding if the question will generate significant information for practice, and determining if the question will clearly direct the review process and synthesis of findings. A well-stated question will define the nature and scope of the literature search, identify keywords for the search, determine the best search strategy, provide guidance in selecting articles for the review, and guide the synthesis of results (Bettany-Saltikov, 2010a, 2010b; Higgins & Green, 2008; Liberati et al., 2009).

The question developed might focus on an intervention or therapy, health promotion action, illness prevention strategy, diagnostic process, prognosis, causation, or experiences (Bettany-Saltikov, 2010a). One of the most common formats used to develop a relevant clinical question to guide a systematic review is the PICO or PICOS format described in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins & Green, 2008). PICOS format includes the following elements:

Goulding et al. (2010) noted that interventions to change maladaptive illness beliefs were beneficial to people with CHD because positive illness representations may lead to improved lifestyle behaviors of exercise, smoking cessation, and balanced diet. What was not known was “[w]hich types of intervention to change illness cognitions (e.g. counseling, education, or cognitive behavioural) are most effective” (Goulding et al., 2010, p. 948) for people with CHD. The population was people with CHD, and the intervention was focused on changing maladaptive illness beliefs of these individuals. The different types of this intervention, including counseling, education, and cognitive behavioral therapy, were compared. The intervention group was compared with groups receiving standard care, no treatment, or a variation of the treatment. The primary outcome measured was the change in beliefs about CHD at follow-up. The study design included synthesis of only RCTs using guidelines from the Cochrane Collaboration handbook (Higgins & Green, 2008) (see Chapters 11 and 14 about conducting RCTs). The study design (RCTs) clearly focused the literature review but might have eliminated some important studies that could have expanded the knowledge related to the intervention of changing illness beliefs.

Step 2: State the Purpose and Objectives or Aims of the Review

Most systematic reviews of research include a purpose and specific aims or objects to guide the synthesis process (Bettany-Saltikov, 2010a; Rew, 2011). The purpose identifies the major goal or focus of the review. Goulding et al. (2010, p. 948) identified their purpose as follows: “This systematic review was therefore necessary to collate and present evidence of the effectiveness of maladaptive belief change interventions for people with CHD.” The specific aims direct the remaining steps of the research synthesis and become the focus of the discussion of the findings. Goulding et al. (2010) identified the following.

Step 3: Identify the Literature Search Criteria and Strategies

Researchers conducting a systematic review or meta-analysis need to identify the inclusion and exclusion criteria to be used to direct their literature search. The PICOS format might be used to develop the search criteria with more detail being developed for each of the elements. These search criteria might focus on the following: (1) type of research methods, such as quantitative, qualitative, or outcomes research; (2) the population or type of study participants; (3) study designs, such as description, correlational, quasi-experimental, experimental, qualitative, or mixed methods; (4) sampling processes, such as probability or nonprobability sampling methods; (5) intervention and comparison interventions; and (6) specific outcomes to be measured. The PICOS format is effective in identifying the key terms to be included in the search process. The search criteria also need to indicate the years for the review, language, and publication status. The focus of the review might be narrowed by limiting the years reviewed, the language to English, and only studies in print (Bettany-Saltikov, 2010b; Higgins & Green, 2008; Rew, 2011).

Often searches have been limited to published sources in common databases, which excludes the grey literature from the research synthesis. Grey literature refers to studies that have limited distributions, such as theses and dissertations, unpublished research reports, articles in obscure journals, articles in some online journals, conference papers and abstracts, conference proceedings, research reports to funding agencies, and technical reports (Benzies, Premji, Hayden, & Serrett, 2006; Conn, Valentine, Cooper, & Rantz, 2003). Most grey literature is difficult to access through database searches and is often not peer-reviewed with limited referencing information. These are some of the main reasons for not including grey literature in searches for systematic reviews and meta-analyses. However, excluding grey literature from these searches might result in misleading, biased results. Studies with significant findings are more likely to be published than studies with nonsignificant findings and are usually published in more high-impact, widely distributed journals that are indexed in computerized databases (Conn et al., 2003). Studies with significant findings are more likely to have duplicate publications that need to be excluded when selecting studies to include in a research synthesis. Benzies et al. made the following recommendations related to including grey literature in a systematic review or meta-analysis.

Authors of systematic reviews also need to identify the search strategies that they will use. Many sources are identified through searches of electronic databases using the criteria previously discussed. However, publication bias might best be reduced with more rigorous searches of the following areas for grey literature and other unpublished studies:

Often it is best to construct a table that includes the search criteria so that they can be applied consistently throughout the search process (Liberati et al., 2009) (see Chapter 6). Goulding et al. (2010) used the PICOS format to determine inclusion criteria for the studies in the search. The participants had to be adults with at least one of the following: angina, CHD, myocardial infarction, or eligible for or recently received revascularization by percutaneous coronary intervention or coronary artery bypass graft surgery. The studies needed to focus on an intervention to change knowledge, attitudes, perceptions, and misconceptions about CHD. The interventions to change maladaptive beliefs were compared with different interventions, usual care, or no intervention. The primary outcome was change in beliefs about CDH, and the secondary outcomes were focused on quality of life, behavior change, anxiety level, depression, psychological well-being, and modifiable risk factors. The study design was limited to only RCTs that included a comparison of the intervention group with a control group or another intervention.

Goulding et al. (2010) designed their literature search strategies and their protocol for conducting their systematic review using sources such as the Cochrane Collaboration handbook (Higgins & Green, 2008) and the Quality of Reporting of Meta-analyses (QUOROM) Statement (Moher et al., 1999). No date restriction was applied to the search for studies, but because of the lack of funds, only studies reported in English were identified. The databases searched are discussed in Step 4. The researchers did not include grey literature in their review and recognized this as a limitation in their discussion section.

Step 4: Conduct a Comprehensive Search of the Research Literature

The next step for conducting a systematic review or meta-analysis requires an extensive search of the literature focused by the criteria and strategies identified in Step 3. The different databases searched, date of the search, and search results need to be recorded for each database (see Chapter 6 for details on conducting and storing searches of databases). Table 19-1 identifies common databases that are searched by nurses in conducting syntheses of research and in searching for evidence-based guidelines. Usually the key search terms are identified in the report. Sometimes authors of systematic reviews provide a table that identifies the search terms and criteria. The PRISMA Statement recommends presenting the full electronic search strategy used for at least one major database such as CINAHL or MEDLINE (Liberati et al., 2009). The search strategies used to identify grey literature and other unpublished studies need to be identified.

Goulding et al. (2010) searched the following electronic databases: MEDLINE, EMBASE, CINAHL, BNI, PsychINFO, The Cochrane Library (including the Cochrane Database of Systematic Reviews, CENTRAL, and DARE), and Web of Knowledge. They provided an extensive table that detailed their electronic search of the five databases identified. The search criteria included the participants with different types of CHD; interventions of education and cognitive and behavioral therapies; comparisons with control, standard care, and placebo groups; outcomes of health knowledge, attitudes, and illness perceptions; and study design of RCT. The authors noted that the electronic search identified 3526 citations and that they obtained one source from an expert researcher and seven by reviewing the references of other studies.

Step 5: Selection of Studies for Review

The selection of studies for inclusion in the systematic review or meta-analysis is a complex process that initially involves review and removal of duplicate sources. The abstracts of the remaining studies are reviewed by two or more authors and sometimes an external reviewer to ensure they meet the criteria identified in Step 3. The abstracts might be excluded based on the study participants, interventions, outcomes, or design not meeting the search criteria. Sometimes the abstracts are not in English, are incomplete, or are of studies not attainable. If contacting the authors of the abstracts cannot produce essential information, often the abstracts are excluded from the review (Bettany-Saltikov, 2010b; Higgins & Green, 2008; Liberati et al., 2009).

After the abstracts meeting the designated criteria are identified, the next step is to retrieve the full-text citation for each study. It is best to enter these studies into a table and document how each study meets the eligibility criteria. If studies do not meet criteria, they need to be removed with a rationale provided. Two or more authors of the review need to examine the studies to ensure that the eligibility or inclusion criteria are consistently implemented. Often the study selection process includes all members of the review team. This selection process is best demonstrated by a flow diagram that was developed by the PRISMA Group (Liberati et al., 2009). Figure 19-1 shows this diagram, which has four phases: (1) identification of the sources, (2) screening of the sources based on set criteria, (3) determining if the sources meet eligibility requirements, and (4) identifying the studies included in the review.

Goulding et al. (2010) provided the following description of their section of sources and a flow diagram (see Figure 19-2) that documented the final results of the 13 RCTs included in their systematic review.

“Search Outcome

“The electronic search produced 3526 citations, which were reduced to 115 on citation review. A check of 10% of these citations was undertaken by an independent researcher from another university, with 100% concordance on abstracts to be retrieved. A review of abstracts identified 74 papers to retrieve in full. A further seven papers were identified from reference checks, and an additional relevant study was uncovered via contact with an expert in the field. After a consensus meeting between all authors of the review, 13 studies were included. Each of these was a published journal article. The study selection flowchart shown in [Figure 19-2] documents this process.” (Goulding et al., 2010, p. 950)

Step 6: Critical Appraisal of the Studies Included in Review

An initial critical appraisal of the methodological quality of the studies occurs during the selection of the studies to be included in the systematic review. Once the studies are selected, a more thorough critical appraisal takes place. This critical appraisal is best done by constructing a table describing the characteristics of the included studies, such as the purpose of the studies, population, sampling method, sample size, sample acceptance and attrition rates, design, intervention (independent variable), dependent variables, measurement methods for each dependent variable, and major results (Bettany-Saltikov, 2010b; Higgins & Green, 2008; Liberati et al., 2009).

It is best if two or more experts independently review the studies and make judgments about their quality. The authors of the review usually contact the study investigators if needed to obtain important information about the study design or results not included in the publication. Chapter 18 provides guidelines for critically appraising quantitative and qualitative studies. The critical appraisal of the studies is often difficult because of the differences in types of participants, designs, sampling methods, intervention protocols, outcome variables and measurement methods, and presentation of results. The studies are often rank-ordered based on their quality and contribution to the development of the review (Bettany-Saltikov 2010b; Liberati et al., 2009).

Goulding et al. (2010) developed a detailed table of essential content from the 13 studies included in the systematic review and labeled the headings of the columns in the table as (1) authors, year, and country of the study; (2) participants and setting; (3) design, sample size per group, and follow-up; (4) intervention; (5) control or comparison group; (6) study outcomes; and (7) results with statistical significance set at alpha = 0.05. Two reviewers independently assessed the quality of each study. They provided a detailed description of the appraisal process and their findings for each study. You may want to access this systematic review to view their table of studies and examine their critical appraisal process.

Step 7: Conduct a Meta-Analysis If Appropriate

Some systematic reviews include published meta-analyses as sources in the review. Because a meta-analysis involves the use of statistics to summarize results of different studies, it usually provides strong, objective information about the effectiveness of an intervention or solid knowledge about a clinical problem. Other authors conduct a meta-analysis in the synthesis of sources for their systematic review (Liberati et al., 2009). The authors of the review need to provide a rationale for conducting the meta-analysis and detail the process they used to conduct this analysis. For example, a meta-analysis might be conducted on a small group of similar studies to determine the effect of an intervention. The next section provides more details on conducting a meta-analysis.

The systematic review conducted by Goulding et al. (2010) did not include a meta-analysis as a source, and a meta-analysis was not conducted during the review process. A meta-analysis was probably not appropriate because of the limited number of studies that had been conducted to examine the effectiveness of the intervention to change maladaptive illness beliefs in people with CHD.

Step 8: Results of the Review

The results of the authors’ reviews need to include a description of the study participants, types of interventions, and outcomes. The results of the different types of interventions might be best summarized in a table that includes the following: (1) study source; (2) structure of the intervention (stand-alone or multifaceted); (3) specific type of intervention such as physiological treatment, education, counseling, or behavioral therapy; (4) delivery method such as demonstration and return demonstration, verbal, video, or self-administered; and (5) statistical difference between the intervention and the control, standard care, placebo, or alternative intervention groups (Liberati et al., 2009).

The specific outcomes, including primary and secondary outcomes, of the studies might also be best summarized in a table. This table might include (1) the study source; (2) outcome variable, with an indication as to whether it was a primary or secondary outcome in the study; (3) measurement method used for each study outcome variable; and (4) the quality of the measurement methods, such as the reliability and validity of a scale or the precision and accuracy of a physiological measure (see Chapter 16).

Goulding et al. (2010, p. 955) described their participants for their 13 studies as “male and female adults of all ages with a diagnosis of CHD (including people diagnosed with MI [myocardial infarction] or angina or those receiving revascularization). There was no clear link between patient group and effectiveness of interventions to change beliefs.” These authors also provided two detailed tables: one addressing the types of interventions and the other identifying the measurement methods of the study outcomes and quality of these methods. Numerous outcome measures were used in the 13 studies, and some had poor reliability and validity, which limit the results of this systematic review. Goulding et al. also summarized key findings related to the effectiveness of the interventions and reached the following conclusion.

Step 9: Discussion Section of the Review

In a systematic review or meta-analysis, the discussion of the findings needs to include an overall evaluation of the types of interventions implemented and the outcomes measured. The methodological issues or limitations of the review also need to be addressed. The discussion section needs to include a theoretical link back to the studies’ frameworks to indicate the theoretical implications of the findings. Lastly, the discussion section needs to provide recommendations for further research, practice, and policy development (Bettany-Saltikov, 2010b; Higgins & Green, 2008; Liberati et al., 2009). Goulding et al. (2010) provided the following discussion of their findings; review limitations; and recommendations for research, practice, and policy development.

Goulding et al. stressed the importance of interventions to change maladaptive illness beliefs of people with CHD. However, the impact of changing those beliefs on behavior, functional, and psychological outcomes is unclear. Additional research is needed on the use of cognitive behavioral interventions to change maladaptive illness beliefs and how this affects people with CHD health outcomes.

Step 10: Development of the Final Report for Publication

The final step is the development of the systematic review report for publication and presentation. The report should include a title that identifies it as either a systematic review or a meta-analysis for ease of location in database searches. An abstract also needs to be included that identifies background, purpose, data sources, review methods, results, and conclusions. The body of the report needs to include the content discussed in the previous nine steps: (1) clinical question to be addressed by the review; (2) purpose and aims or objectives to be accomplished by the review; (3) search criteria and strategies used to identify essential studies; (4) comprehensive search of the literature and results; (5) selection of studies to be included in the review; (6) critical appraisal process and findings; (7) description of the meta-analysis conducted and results if appropriate; (8) presentation of the review results in table and narrative format; and (9) discussion of the systematic review findings, limitations, and implications for further research, practice, and policy development.

The PRISMA Group developed a 27-item checklist to use in developing the final reports for systematic reviews and meta-analyses. This PRISMA Group checklist is presented in Table 19-3 and discussed on PRISMA Group website (http://www.prisma-statement.org) and in the articles by Liberati et al. (2009) and Moher, Liberati, Tetzlaff, Altman, and PRISMA Group (2009). This checklist is an excellent guide to use in developing the final report of a systematic review or meta-analysis for publication. This checklist was developed to improve the quality, completeness, and consistency of the syntheses developed and published in nursing, medicine, and other healthcare professions.

TABLE 19-3

Checklist of Items to Include When Reporting a Systematic Review or Meta-Analysis

Section/Topic	No.	Checklist Item	Reported on Page No.
Title
Title	1	Identify the report as a systematic review, meta-analysis, or both
Abstract
Structured summary	2	Provide a structured summary including, as applicable, background; objectives; data sources; study eligibility criteria, participants, and interventions; study appraisal and synthesis methods; results; limitations; conclusions and implications of key findings; systematic review registration number
Introduction
Rationale	3	Describe the rationale for the review in the context of what is already known
Objectives	4	Provide an explicit statement of questions being addressed with reference to PICOS (participants, interventions, comparisons, outcomes, and study design)
Methods
Protocol and registration	5	Indicate if a review protocol exists; if and where it can be accessed (e.g., Web address); and, if available, provide registration information including registration number
Eligibility criteria	6	Specify study characteristics (e.g., PICOS, length of follow-up) and report characteristics (e.g., years considered, language, publication status) used as criteria for eligibility, giving rationale
Information sources	7	Describe all information sources (e.g., databases with dates of coverage, contact with study authors to identify additional studies) in the search and date last searched
Search	8	Present full electronic search strategy for at least one database, including any limits used, such that it could be repeated
Study selection	9	State the process for selecting studies (i.e., screening, eligibility, included in systematic review, and, if applicable, included in the meta-analysis)
Data collection process	10	Describe method of data extraction from reports (e.g., piloted forms, independently, in duplicate) and any processes for obtaining and confirming data from investigators
Data items	11	List and define all variables for which data were sought (e.g., PICOS, funding sources) and any assumptions and simplifications made
Risk of bias in individual studies	12	Describe methods used for assessing risk of bias of individual studies (including specification of whether this was done at the study or outcome level), and how this information is to be used in any data synthesis
Summary measures	13	State the principal summary measures (e.g., risk ratio, difference in means)
Synthesis of results	14	Describe the methods of handling data and combining results of studies, if done, including measures of consistency (e.g., I2) for each meta-analysis

Only gold members can continue reading. Log In or Register to continue

Share this:

• Click to share on Twitter (Opens in new window)
• Click to share on Facebook (Opens in new window)

Related

Related posts:

1. Interpreting Research Outcomes
2. Review of Relevant Literature
3. Disseminating Research Findings
4. Selecting a Quantitative Research Design

Stay updated, free articles. Join our Telegram channel

Join