Chapter Contents

Introduction

By the time a research report is published in a peer-reviewed journal, it has been critically reviewed by experts. Usually, changes have been made to the initial draft by the author(s) responding to the reviewers’ comments. Nevertheless, even a thorough evaluation procedure doesn’t guarantee the validity of the design or the conclusions presented in a published paper. Ultimately, you as a health professional must be responsible for judging the validity and relevance of published material for your own clinical activities (which is why you are studying health sciences). Evidence-based practice focuses on the ways in which practitioners can improve their clinical practice by using research evidence. The systematic review processes employed by bodies such as the Cochrane Collaboration are intended to assist clinicians in the selection of interventions that are well proven and safe (see Ch. 24).

The proper attitude to take with published material, including systematic reviews, is one of hard-nosed scepticism, whatever the status of the publication. This attitude is based on our understanding of the uncertain and provisional nature of scientific and professional knowledge, as outlined in Chapter 1. In addition, health researchers deal with the investigation of complex phenomena, where it is often impossible for ethical reasons to exercise the desired levels of control or to collect crucial information required to arrive at definitive conclusions. The conduct of a valid critique requires that we compare the methods used by the researchers with the rules of evidence in the context of a research project. Critical evaluation identifies the strengths and weaknesses of a research publication, to ensure that patients receive assessment and treatment based on the best available evidence.

The aim of this chapter is to discuss the evaluation of published research. The chapter is organized around the evaluation of the specific sections of research publications.

The specific aims of this chapter are to:

Guidelines for critical appraisal of research publications

Many textbooks on health research methodology or evidence-based health care provide guidelines for the critical appraisal of published research. For example, Hicks (2009) suggested a list of questions relevant for conducting rigorous appraisals and provided a detailed sample critique of a health research publication. Critical appraisal is an essential step in conducting evidence-based health care (see Ch. 24). Consequently, textbooks in this area devote chapters to explaining critical evaluations. For example, Dawes et al (2005) provide detailed guidelines for critiquing publications based on different types of designs, including randomized controlled trials (RCTs), case-control and cohort studies and systematic reviews.

There are also numerous websites hosted by reputable organizations for facilitating the appraisal of published evidence. A useful online example is the University of South Australia Division of Health Sciences International Centre for Allied Health (http://www.unisa.edu.au/cahe/resources/cat/default.asp) which is dedicated to the critical evaluation of published research.

Some groups created critical appraisal schemes which attempt to produce numerical scores to represent methodological quality. For example, the Downs and Black quality index generates an overall score out of 21 to represent the quality of a publication. Publications with low scores may be excluded from systematic reviews or attributed with poor credibility. While such quantitative approaches are followed by some reviewers to identify risk of bias (e.g. Soares-Weiser et al 2010; see Ch. 4) it is worth noting that the critical appraisal of published research is not easily qualified. Rather, health researchers or practitioners need to make critical judgements based on their personal knowledge, experience and work contexts.

Table 23.1 summarizes some of the potential problems, and their implications, which might emerge in the context-critical evaluation of an investigation. A point which must be kept in mind is that, even where an investigation is flawed, useful knowledge might be drawn from it. The aim of critical analysis is not to discredit or tear down published work, but to ensure that the reader understands its implications and limitations with respect to theory and practice. In this chapter we will focus on the constructive, critical evaluation of each stage of the research process as identified in Chapter 3.

Stay updated, free articles. Join our Telegram channel

Join