Reading and Critiquing Research Articles

Identify and describe the major sections in a research journal article

Characterize the style used in quantitative and qualitative research reports

Read a research article and broadly grasp its “story”

Describe aspects of a research critique

Understand the many challenges researchers face and identify some tools for addressing methodologic challenges

Define new terms in the chapter

Key Terms

Abstract

Bias

Blinding

Confounding variable

Credibility

Critique

Findings

IMRAD format

Inference

Journal article

Level of significance

Placebo

Randomness

Reflexivity

Reliability

Research control

Scientific merit

Statistical significance

Statistical test

Transferability

Triangulation

Trustworthiness

Validity

Evidence from nursing studies is communicated through research reports that describe what was studied, how it was studied, and what was found. Research reports are often daunting to readers without research training. This chapter aims to make research reports more accessible and also provides some guidance regarding critiques of research reports.

TYPES OF RESEARCH REPORTS

Nurses are most likely to encounter research evidence in journals or at professional conferences. Research journal articles are descriptions of studies published in professional journals. Competition for journal space is keen, so research articles are brief—generally only 10 to 20 double-spaced pages. This means that researchers must condense a lot of information about the study into a short report.

Usually, manuscripts are reviewed by two or more peer reviewers (other researchers) who make recommendations about acceptance of or revisions to the manuscript. Reviews are usually blind—reviewers are not told researchers’ names, and authors are not told reviewers’ names. Consumers thus have some assurance that journal articles have been vetted by other impartial nurse researchers. Nevertheless, publication does not mean that the findings can be uncritically accepted. Research method courses help nurses to evaluate the quality of evidence reported in journal articles.

At conferences, research findings are presented as oral presentations or poster sessions. In an oral presentation, researchers are typically allotted 10 to 20 minutes to describe key features of their study to an audience. In poster sessions, many researchers simultaneously present visual displays summarizing their studies, and conference attendees walk around the room looking at the displays. Conferences offer an opportunity for dialogue: Attendees can ask questions to help them better understand what the findings mean; moreover, they can offer the researchers suggestions relating to clinical implications of the study. Thus, professional conferences are a valuable forum for clinical audiences.

THE CONTENT OF RESEARCH JOURNAL ARTICLES

Many research articles follow an organization called the IMRAD format. This format organizes content into four main sections—Introduction, Method, Results, and Discussion. The paper is preceded by a title and an abstract and concludes with references.

The Title and Abstract

Research reports have titles that succinctly convey key information. In qualitative studies, the title normally includes the central phenomenon and group under investigation. In quantitative studies, the title communicates key variables and the population (in other words, PICO components).

The abstract is a brief description of the study placed at the beginning of the article. The abstract answers questions like the following: What were the research questions? What methods were used to address those questions? What were the findings? and What are the implications for nursing practice? Readers can review an abstract to judge whether to read the full report.

The Introduction

The introduction to a research article acquaints readers with the research problem and its context. This section usually describes the following:

The central phenomena, concepts, or variables under study

The study purpose and research questions or hypotheses

A review of the related literature

The theoretical or conceptual framework

The significance of and need for the study

Thus, the introduction lets readers know the problem the researcher sought to address.

Example of an introductory material

“Little is known about how the back-to-school transition following cancer treatment influences adolescents’ developing self-identity and social relationships.” Data from the adolescent’s perspective are particularly limited . . . The purpose of this study was to describe how the return to school affects adolescents’ beliefs about themselves, their self-identity, and their social relationships (Choquette et al., 2015).

In this paragraph, the researchers described the central concept of interest (experiences of adolescents returning to school after cancer treatment), the need for the study (the fact that little is known about the experience directly from adolescents), and the study purpose.

TIP The introduction section of most reports is not specifically labeled “Introduction.” The report’s introduction immediately follows the abstract.

The Method Section

The method section describes the methods used to answer the research questions. In a quantitative study, the method section usually describes the following, which may be presented in labeled subsections:

The research design

The sampling plan

Methods of measuring variables and collecting data

Study procedures, including procedures to protect human rights

Data analysis methods

Qualitative researchers discuss many of the same issues but with different emphases. For example, a qualitative study often provides more information about the research setting and the context of the study. Reports of qualitative studies also describe the researchers’ efforts to enhance the integrity of the study.

The Results Section

The results section presents the findings that were obtained by analyzing the study data. The text presents a narrative summary of key findings, often accompanied by more detailed tables. Virtually all results sections contain descriptive information, including a description of the participants (e.g., average age, percent male, female, and other).

In quantitative studies, the results section also reports the following information relating to statistical tests performed:

The names of statistical tests used. Researchers test their hypotheses and assess the probability that the results are right using statistical tests. For example, if the researcher finds that the average birth weight of drug-exposed infants in the sample is lower than the birth weight of infants not exposed to drugs, how probable is it that the same would be true for other infants not in the sample? A statistical test helps answer the question, Is the relationship between prenatal drug exposure and infant birth weight real, and would it likely be observed with a new sample from the same population? Statistical tests are based on common principles; you do not have to know the names of all statistical tests to comprehend the findings.

The value of the calculated statistic. Computers are used to calculate a numeric value for the particular statistical test used. The value allows researchers to reach conclusions about their hypotheses. The actual value of the statistic, however, is not inherently meaningful and need not concern you.

Statistical significance. A critical piece of information is whether the statistical tests were significant (not to be confused with clinically important). If a researcher reports that the results are statistically significant, it means the findings are probably true and replicable with a new sample. Research reports also indicate the level of significance, which is an index of how probable it is that the findings are reliable. For example, if a report indicates that a finding was significant at the .05 probability level (symbolized as p), this means that only 5 times out of 100 (5 ÷ 100 = .05) would the obtained result be spurious. In other words, 95 times out of 100, similar results would be obtained with a new sample. Readers can thus have a high degree of confidence—but not total assurance—that the results are accurate.

Example from the results section of a quantitative study

Park and coresearchers (2015) tested the effects of a 16-session Patient-Centered Environment Program (PCEP) on a variety of outcomes for home-dwelling patients with dementia. Here is a sentence adapted from the reported results: “Findings showed that agitation (t = 2.91, p < .02) and pain (t = 4.51, p < .002) improved after receiving the PCEP” (p. 40).

In this example, the researchers indicated that both agitation and pain were significantly improved following receipt of the PCEP intervention. The changes in agitation and pain were not likely to have been haphazard and probably would be replicated with a new sample. These findings are very reliable. For example, with regard to pain reduction, it was found that an improvement of the magnitude obtained would occur just as a “fluke” less than 2 times in 1,000 (p < .002). Note that to comprehend this finding, you do not need to understand what a t statistic is, nor do you need to concern yourself with the actual value of the t statistic, 4.51.

TIP Results are more reliable if the p value is smaller. For example, there is a higher probability that the results are accurate when p = .01 (1 in 100 chance of a spurious result) than when p = .05 (5 in 100 chances of a spurious result). Researchers sometimes report an exact probability (e.g., p = .03) or a probability below conventional thresholds (e.g., p < .05—less than 5 in 100).

In qualitative reports, researchers often organize findings according to the major themes, processes, or categories that were identified in the data. The results section of qualitative reports sometimes has several subsections, the headings of which correspond to the researcher’s labels for the themes. Excerpts from the raw data (the actual words of participants) are presented to support and provide a rich description of the thematic analysis. The results section of qualitative studies may also present the researcher’s emerging theory about the phenomenon under study.

Example from the results section of a qualitative study

Larimer and colleagues (2015) studied the experiences, challenges, and coping behaviors of young adults with pacemakers or implantable cardioverter defibrillators. Participants described four categories of challenges, one of which was labeled “Limited support.” Here is an excerpt illustrating that category: “If I go to pediatric doctors, their waiting rooms have blocks and pink elephants. But in cardiopulmonary rehab, I’m the youngest by 60 years. It feels like I’m in a no man’s land, stuck in the middle” (p. 3).

The Discussion Section

In the discussion, the researcher presents conclusions about the meaning and implications of the findings, i.e., what the results mean, why things turned out the way they did, how the findings fit with other evidence, and how the results can be used in practice. The discussion in both quantitative and qualitative reports may include the following elements:

An interpretation of the results

Clinical and research implications

Study limitations and ramifications for the believability of the results

Researchers are in the best position to point out deficiencies in their studies. A discussion section that presents the researcher’s grasp of study limitations demonstrates to readers that the authors were aware of the limitations and probably took them into account in interpreting the findings.

References

Research articles conclude with a list of the books and articles that were referenced. If you are interested in additional reading on a topic, the reference list of a recent study is a good place to begin.

THE STYLE OF RESEARCH JOURNAL ARTICLES

Research reports tell a story. However, the style in which many research journal articles are written—especially for quantitative studies—makes it difficult for some readers to understand or become interested in the story.

Why Are Research Articles So Hard to Read?

To unaccustomed audiences, research reports may seem bewildering. Four factors contribute to this impression:

1. Compactness. Journal space is limited, so authors compress a lot of information into a small space. Interesting, personalized aspects of the investigation cannot be reported, and, in qualitative studies, only a handful of supporting quotes can be included.

2. Jargon. The authors of research articles use research terms that may seem esoteric.

3. Objectivity. Quantitative researchers tend to avoid any impression of subjectivity, so they tell their research stories in a way that makes them sound impersonal. Most quantitative research articles are written in the passive voice, which tends to make the articles less inviting and lively. Qualitative reports, by contrast, are often written in a more conversational style.

4. Statistical information. In quantitative reports, numbers and statistical symbols may intimidate readers who do not have statistical training.

A goal of this textbook is to assist you in understanding the content of research reports and in overcoming anxieties about jargon and statistical information.

HOW-TO-TELL TIP How can you tell if the voice is active or passive? In the active voice, the article would say what the researchers did (e.g., “We used a mercury sphygmomanometer to measure blood pressure”). In the passive voice, the article indicates what was done, without indicating who did it, although it is implied that the researchers were the agents (e.g., “A mercury sphygmomanometer was used to measure blood pressure”).

Tips on Reading Research Articles

As you progress through this book, you will acquire skills for evaluating research articles, but the skills involved in critical appraisal take time to develop. The first step is to comprehend research articles. Here are some hints on digesting research reports.

Grow accustomed to the style of research articles by reading them frequently, even though you may not yet understand the technical points.

Read journal articles slowly. It may be useful to skim the article first to get the major points and then read the article more carefully a second time.

On the second reading, train yourself to become an active reader. Reading actively means that you constantly monitor yourself to verify that you understand what you are reading. If you have difficulty, you can ask someone for help. In most cases, that “someone” will be your instructor, but also consider contacting the researchers themselves.

Keep this textbook with you as a reference when you read articles so that you can look up unfamiliar terms in the glossary or index.

Try not to get bogged down in (or scared away by) statistical information. Try to grasp the gist of the story without letting symbols and numbers frustrate you.

CRITIQUING RESEARCH REPORTS

A critical reading of a research article involves a careful appraisal of the researcher’s major conceptual and methodologic decisions. It will be difficult to criticize these decisions at this point, but your skills will improve as you progress through this book.

What Is a Research Critique?

A research critique is an objective assessment of a study’s strengths and limitations. Critiques usually conclude with the reviewer’s summary of the study’s merits, recommendations regarding the value of the evidence, and suggestions about improving the study or the report.

Research critiques of individual studies are prepared for various reasons, and they vary in scope. Peer reviewers who are asked to prepare a written critique for a journal considering publication of a manuscript may evaluate the strengths and weaknesses in terms of substantive issues (Was the research problem significant to nursing?), theoretical issues (Were the conceptual underpinnings sound?), methodologic decisions (Were the methods rigorous, yielding believable evidence?), interpretive (Did the researcher reach defensible conclusions?), ethics (Were participants’ rights protected?), and style (Is the report clear, grammatical, and well organized?). In short, peer reviewers do a comprehensive review to provide feedback to the researchers and to journal editors about the merit of both the study and the report and typically offer suggestions for revisions.

Critiques designed to inform evidence-based nursing practice are seldom comprehensive. For example, it is of little consequence to evidence-based practice (EBP) that an article is ungrammatical. A critique of the clinical utility of a study focuses on whether the evidence is accurate, believable, and clinically relevant. These narrower critiques focus more squarely on appraising the research methods and the findings themselves.

Students taking a research methods course also may be asked to critique a study. Such critiques are often intended to cultivate critical thinking and to induce students to apply newly acquired skills in research methods.

Critiquing Support in This Textbook

We provide several types of support for research critiques. First, detailed critiquing suggestions relating to chapter content are included at the end of most chapters. Second, it is always illuminating to have a good model, so we prepared critiques of two studies. The two studies in their entirety and the critiques are in Appendices C and D.

Third, we offer a set of key critiquing guidelines for quantitative and qualitative reports in this chapter, in Tables 4.1 and 4.2, respectively. The questions in the guidelines concern the rigor with which the researchers dealt with critical research challenges, some of which we outline in the next section.