Chapter 10. The Cochrane Database and Meta-analysis
Zena Moore and Seamus Cowman
Introduction
▪ Background to the Cochrane Collaboration
▪ Finding the evidence to appraise
▪ The Cochrane Methodology Groups
▪ Systematic reviews and nursing practice
▪ What is meta-analysis?
▪ Undertaking a systematic review
▪ Conclusion
Introduction
The cornerstone of evidence-based practice is the integration of high quality research evidence into clinical decision making. This evidence is used in combination with clinical judgement and experience to plan the most appropriate patient treatment (Sackett et al 1996). In striving to achieve evidence-based practice, the nurse is faced with an increasing dilemma. There is simply too much published literature and not enough time to appraise it all critically. There was a time when it was possible to read everything related to a particular speciality; however, currently over 6000 health-related articles are published each day (Levin 2001). The Cochrane Collaboration aims to be the medium through which this information is made accessible.
Background to the Cochrane Collaboration
A literature review forms the basis for further research through identification of the gaps in current knowledge (Burns & Grove 2001). On completion of a research project, each researcher’s job is to identify the contribution of their research project to the body of knowledge (Burns & Grove 2001). The traditional literature review lacks a scientific approach to the selection and rejection of material for inclusion in the review (Egger et al 2003). The concern is that these reviews will be based on an inadequate or incomplete analysis of the evidence. For example, inclusion of only published material leads to bias in outcomes reporting as it is known that non-significant research findings take longer to appear in the literature (Misakian & Bero 1998). Non-significant research findings are just as important to be aware of as significant findings. Having only some of the information will result in bias in interpreting the strength, or direction, of the evidence base, leading to inappropriate clinical decisions.
Restriction of article inclusion to only those published in a particular language deliberately rejects important material. For example, in 36 meta-analyses published in leading English language general medical journals from 1991 to 1993, 26 had restricted their search to studies reported only in English (Grégoire et al 1995). These issues among others add bias to the review, reducing the confidence which can be placed in the results.
In essence, the problem lies in the fact that a different conclusion may be drawn if the whole body of knowledge was available. Rossouw et al (1990) noted, in their critique of a published review of cholesterol lowering after myocardial infarction, that inclusion bias led to a completely different meta-analysis than would have been achieved had other studies been included: although one study that met the inclusion criteria was available, it was excluded from the original review; along with 11 other studies where the reviewers had no clear rationale for their exclusion. The initial review found in favour of hormone treatment, whereas inclusion by Rossouw et al (1990) of the omitted studies, demonstrated the opposite.
A good example of the dilemma of trying to integrate a limited knowledge of evidence into clinical decision making was the use of anti-arrhythmic drugs postmyocardial infarction (Chalmers 2003). This treatment led to the deaths of thousands of people in the United States in the early 1980s. A previous systematic review of the treatment had sounded warning bells, but trials continued. Due to the lack of integration of the new trial evidence into the previous systematic review, the danger of these drugs went unrecognised. Thus, the treatment continued to be used (Chalmers 2003). This lack of synthesis and dissemination of trial evidence was the inspiration for the development of the Cochrane Collaboration in 1993 (The Cochrane Collaboration 2005).
Cochrane realised that failure to use evidence appropriately from clinical trials was putting the lives of patients at risk, for example the use of steroids in women expected to deliver prematurely. The result of the meta-analysis of the available RCTs showed a reduction of the odds of death in the babies of these women by 30–50% (Antes & Oxman 2003). Before the publication of these findings, obstetricians were unaware of these outcomes; therefore, steroids were not widely used, resulting in unnecessary suffering and loss of life. The forest plot of this meta-analysis has become the logo for the Cochrane Collaboration (Antes & Oxman 2003).
Cochrane identified the requirement for high quality systematic reviews of literature and the importance of making such information readily available. For that reason, Cochrane began his mission for the development of a database of systematic reviews. The original systematic reviews focused on pregnancy and childbirth. Before his death in 1988, over 600 systematic reviews had been completed (Antes & Oxman 2003). Finally, in 1993 the Cochrane Collaboration was formally launched, bringing Cochrane’s quest to fruition (The Cochrane Collaboration 2005).
What is the Cochrane Collaboration?
The Cochrane Collaboration is an international, not-for-profit organisation. It is established as a limited company and is registered as a charity in the UK. The collaboration is the largest organisation worldwide conducting systematic reviews (Bero & Rennie 1995). The aim is to provide individuals with up-to-date information regarding health care interventions. This is achieved through the production, dissemination and maintenance of systematic reviews, covering a wide variety of health care issues (The Cochrane Collaboration 2006). The work of the Cochrane Collaboration (2006) is based on 10 key principles (Box 10.1).
Box 10.1
▪ Collaboration, by internally and externally fostering good communications, open decision-making and teamwork.
▪ Building on the enthusiasm of individuals, by involving and supporting people of different skills and background.
▪ Avoiding duplication, by good management and coordination to maximise economy of effort.
▪ Minimising bias, through a variety of approaches such as scientific rigour, ensuring broad participation and avoiding conflicts of interest.
▪ Keeping up to date, by a commitment to ensure that Cochrane Reviews are maintained through identification and incorporation of new evidence.
▪ Striving for relevance, by promoting the assessment of health care interventions using outcomes that matter to people making choices in health care.
▪ Promoting access, by wide dissemination of the outputs of the Collaboration, taking advantage of strategic alliances, and by promoting appropriate process, content and media to meet the needs of users worldwide.
▪ Ensuring quality, by being open and responsive to criticism, applying advances in methodology and developing systems for quality improvement.
▪ Continuity, by ensuring that responsibility for reviews, editorial processes and key functions is maintained and renewed.
▪ Enabling wide participation in the work of the Collaboration by reducing barriers to contributing and by encouraging diversity.
The Cochrane Collaboration Steering Group
The Cochrane Collaboration Steering Group (CCSG) is the governing body of the Cochrane Collaboration. The CCSG is divided into three subgroups and seven advisory groups (Hetherington 2005). The purpose of these groups is to ensure that the Cochrane Collaboration achieves its mission statement. For example, the handbook Advisory Group is responsible for the Cochrane Handbook for Systematic Reviews of Interventions and other handbooks, which are used as key sources of advice by reviewers in the preparing and maintenance of Cochrane reviews (Alderson et al 2004).
The format of the Cochrane Collaboration
The Cochrane collaboration is divided into a number of sections: Cochrane centres, review groups, methodology groups, fields, and a Cochrane Consumer Network (Alderson et al 2004). There are 12 Cochrane centres worldwide. The first Cochrane centre was established in Oxford, UK in 1993. Since then, centres have opened in South Africa, USA and many other countries, with the most recent being the Chinese Cochrane Centre which opened in 1999 (Alderson et al 2004).
The role and function of the Collaborative Review Groups
Currently there are 50 Collaborative Review Groups (CRGs) each focusing on a specific area of health care. The purpose of these groups is to produce and maintain systematic reviews (Alderson et al 2004). These systematic reviews are available from the Cochrane Library at http://www. thecochranelibrary.com.
One example of a CRG is the Cochrane Wounds Group, which has been in existence since 1995 (Cullum et al 2000). Their focus is on assessing the impact of interventions, including beds, dressings and bandages on both acute and chronic wounds including surgical and traumatic wounds, leg ulcers, pressure ulcers and diabetic foot ulcers. Evidence from randomised controlled trials using valid objective outcomes, such as healing rates or incidence of wounds, is the main source of data used in the development of the Wounds Group reviews (Cullum et al 2000).
Finding the evidence to appraise
A systematic review aims to summarise all the evidence available, published and unpublished, pertaining to a specific health care issue. Therefore, the reviewer needs to be able to access this evidence. The Cochrane Review Groups set out to make this goal possible. The Cochrane Collaboration has developed a search strategy for MEDLINE (Alderson et al 2004) and is working with the US National Library of Medicine to improve the identification of both old and new published trials through MEDLINE (Alderson et al 2004). The review groups can identify those journals that have been already or are currently being hand searched through a database maintained at the New England Cochrane Centre (Alderson et al 2004). The review groups also hand search journals and conference proceedings to identify further studies not available through the previous search strategies (Alderson et al 2004).
The Cochrane Methodology Groups
The expertise in conducting systematic reviews is growing and is supported by the work of the Methodology Groups which advise on different aspects of systematic reviews, for example, design, coding, analysis, dissemination or implementation. The purpose of the groups is to develop and improve the methodological quality of Cochrane reviews thereby ensuring that the reviews are conducted to the highest standards (Alderson et al 2004).
The Cochrane Fields
Fields differ from the review groups in that their focus is on aspects of heath care other than specific health care problems (Gotham 2005). For example, their interest is in the type of health care setting (e.g. primary or secondary) or the individuals involved (e.g. the very young or the very old). The purpose of the fields is not to conduct systematic reviews, rather it is to ensure that their specific area of interest is represented adequately in the work of the Cochrane Collaboration (Gotham 2005).
The Consumer Network
The Consumer Network was registered in 1995 (Jadad & Haynes 1998). The purpose of this network is to facilitate the dissemination of information from systematic reviews to patients, their carers and advocates. This purpose is achieved by sharing information and making the lay press aware of recent relevant additions to the Cochrane Collaboration review databases (Jadad & Haynes 1998).
Systematic reviews and nursing practice
The Cochrane Library has been traditionally linked to medicine due to its reliance on evidence from RCTs, the gold standard in research methodology. However, the practice of using high quality evidence as a basis for clinical decision making is also of fundamental importance in all aspects of nursing practice. More recently it has become clear that nurses have embraced the challenge of conducting systematic reviews evidenced by a search undertaken in the Journal of Advanced Nursing. The journal was searched for the years 1990–2006, using the term ‘Systematic Review’. Systematic reviews began to appear in 1998 and since then the number included has increased steadily, indeed the search yielded 34 hits.
Further support for the role of the Cochrane database of systematic reviews (CDSR) as a source of evidence for nursing practice has been provided by Mistiaen and colleagues (2004). The authors conducted a search of the CDSR 2002 issue 3 and found 160 completed reviews.
Overall the authors found that 60% of the reviews were inconclusive (Mistiaen et al 2004). Either there was no evidence because no studies existed (8.1%), or there was insufficient evidence to support or refute the intervention (51.9%). Overall, the authors suggest that the outcomes of nursing interventions are well represented within the CDSR (Mistiaen et al 2004). Thus the CDSR is an important reference point for the development of clinical practice and research in the nursing arena.
What is meta-analysis?
Meta-analysis involves quantitatively combining the results from individual trials which have explored a similar research question (Naylor 1997). The origins of meta-analysis stem from the work of Pearson in 1902 (Naylor 1997). In an attempt to overcome the limitations of small sample sizes, Pearson developed a statistical method for combining the results of similar studies. The underlying premise is that the studies must be exploring a set of related research hypotheses. The purpose is to allow for greater accuracy in data analysis when using a group of studies rather than relying on the results of an individual study (Naylor 1997).
Undertaking a systematic review does not automatically imply that a meta-analysis will be conducted. Meta-analysis should only be carried out when it is appropriate, i.e. when the differences between studies have occurred by chance not due to factors such as study design or study quality (Greener & Grimshaw 1996). The rationale for this is that combining poor quality studies will still yield poor results even if the studies are similar (Deeks et al 2005).