Critical Appraisal of Nursing Studies



Critical Appraisal of Nursing Studies


image http://evolve.elsevier.com/Grove/practice/


The nursing profession continues to strive for evidence-based practice, which includes critically appraising studies, synthesizing research findings, and applying sound scientific evidence in practice. Researchers also critically appraise studies in a selected area, develop a summary of current knowledge, and identify areas for future studies. Critically appraising research is essential for evidence-based nursing practice and the conduct of future research. The critical appraisal of research involves a systematic, unbiased, careful examination of all aspects of studies to judge their strengths, weaknesses, meaning, and significance. The ability of a nurse to appraise studies critically is based on the nurse’s previous research experience and knowledge of a topic. To conduct a critical appraisal of research, one must possess analysis and logical reasoning skills to examine the credibility and integrity of a study. This chapter provides a background for critically appraising studies in nursing and other healthcare disciplines. The expanding roles of nurses in conducting critical appraisals of research are addressed. Detailed guidelines are provided to direct you in critically appraising both quantitative and qualitative studies.



Evolution of Critical Appraisal of Research in Nursing


The process for critically appraising research has evolved gradually in nursing from a few to now many nurses who are prepared to conduct comprehensive, scholarly critiques. During the 1940s and 1950s, presentations of nursing research were followed by critiques of the studies. These critiques often focused on the weaknesses or limitations of the studies and tended to be harsh and traumatic for the researcher (Meleis, 2007). As a consequence of these early unpleasant experiences, nurse researchers began to protect and shelter their nurse scientists from the threat of criticism. Public critiques, written or verbal, were rare in the 1960s and 1970s. Nurses responding to research presentations focused on the strengths of studies, and the limitations were either critically not mentioned or were minimized. Thus, the effects of the limitations on the meaning, validity, and significance of studies were often lost.


Incomplete critiques or the absence of critiques may have served a purpose as nurses gained basic research skills. However, the nursing discipline has moved past this point, and it is recognized that comprehensive critical appraisals of research are essential to strengthen the scientific investigations needed for evidence-based practice (EBP; Brown, 2009; Craig & Smyth, 2012; Fawcett & Garity, 2009). As a result of advances in the nursing profession over the last 30 years, many nurses now have the educational preparation and expertise to conduct critical appraisals of research. Nursing research textbooks provide detailed information on the critical appraisal process. Skills in critical appraisal are introduced at the baccalaureate level of nursing education and are expanded at the master’s and doctoral levels. Specialty organizations provide workshops on the critical appraisal process to promote the use of scientific evidence in practice.


The critical appraisal of studies is essential for the development and refinement of nursing knowledge. Nurses need these skills to examine the meaning and credibility of study findings and to ask searching questions. Was the methodology of a study sound to produce credible findings? Are the findings an accurate reflection of reality? Do they increase our understanding of the nature of phenomena that are important in nursing? Are the findings from the present study consistent with findings from previous studies? The answers to these questions require careful examination of the research problem and purpose, the theoretical or philosophical basis of the study, and the study’s methodology. Not only must the mechanics of conducting the study be evaluated, but also the abstract and logical reasoning the researchers used to plan and implement the study (Fawcett & Garity, 2009; Munhall, 2012). If the reasoning process used to develop a study has flaws, there are probably flaws in interpreting the meaning of the findings, decreasing the credibility of the study.


All studies have flaws, but if all flawed studies were discarded, there would be no scientific knowledge base for practice. In fact, science itself is flawed. Science does not completely or perfectly describe, explain, predict, or control reality. However, improved understanding and an increased ability to predict and control phenomena depend on recognizing the flaws in studies and in science. New studies can then be planned to minimize the flaws or limitations of earlier studies. A researcher must critically analyze previous studies to determine their limitations and then interpret the study findings in light of those limitations. The limitations can lead to inaccurate data, inaccurate outcomes of analysis, and decreased ability to generalize the findings. You must decide if a study is too flawed to be used in a systematic review of knowledge in an area. Although we recognize that knowledge is not absolute, we need to have confidence in the research evidence synthesized for practice.


All studies have strengths as well as limitations. Recognition of these strengths is also essential to the generation of sound research evidence for practice. If only weaknesses are identified, nurses might discount the value of studies and refuse to invest time in reading and examining research. The continued work of the researcher also depends on recognizing the strengths of the study. If no study is good enough, why invest time conducting research? The strong points of a study, added to the strong points from multiple other studies, slowly build solid research evidence for practice.



When Are Critical Appraisals of Research Implemented in Nursing?


In general, research is critically appraised to broaden understanding, summarize knowledge for practice, and provide a knowledge base for future studies. In addition, critical appraisals are often conducted after verbal presentations of studies, after a published research report, for an abstract section for a conference, for article selection for publication, and for evaluation of research proposals for implementation or funding. Nursing students, practicing nurses, nurse educators, and nurse researchers all are involved in the critical appraisal of research.



Critical Appraisal of Studies by Students


In nursing education, conducting a critical appraisal of a study is often seen as a first step in learning the research process. Part of learning this process is being able to read and comprehend published research reports. However, conducting a critical appraisal of a study is not a basic skill, and the content presented in previous chapters is essential for implementing this process. Nurses usually acquire basic knowledge of the research process and critical appraisal skills early in their nursing education. Advanced analysis skills are usually taught at the master’s and doctoral levels. Performing a critical appraisal of a study involves (1) identifying the elements or steps of the study, (2) determining the study strengths and limitations, and (3) evaluating the credibility and meaning of study findings for nursing knowledge and practice. By critically appraising studies, students expand their analysis skills, strengthen their knowledge base, and increase their use of research evidence in practice.



Critical Appraisal of Research by Practicing Nurses


Practicing nurses need to appraise studies critically so that their practice is based on current research evidence and not tradition and trial and error (Brown, 2009; Melnyk & Fineout-Overholt, 2011). Nursing actions must be updated in response to the current evidence that is generated through research and theory development. Practicing nurses need to design methods for remaining current in their practice areas. Reading research journals and posting or emailing current studies at work can increase nurses’ awareness of study findings but are insufficient for the purposes of critical appraisal. Nurses need to question the quality of the studies and the credibility of the findings and share their concerns with other nurses. For example, nurses may form a research journal club in which studies are presented and critically appraised by members of the group (Gloeckner & Robinson, 2010). Skills in critical appraisal of research enable practicing nurses to synthesize the most credible, significant, and appropriate evidence for use in their practice. EBP is essential in agencies either seeking or maintaining Magnet status. The Magnet Recognition Program® was developed by the American Nurses Credentialing Center (ANCC, 2012) to recognize healthcare organizations that provide nursing excellence with care based on the most current research evidence (see http://www.nursecredentialing.org/Magnet/ProgramOverview.aspx).





Critical Appraisal of Research Presentations and Publications


Critical appraisals following research presentations can assist researchers in identifying the strengths and weaknesses of their studies and generating ideas for further research. Participants listening to study critiques might gain insight into the conduct of research. Experiencing the critical appraisal process can increase the ability of participants to evaluate studies and judge the usefulness of the research evidence for practice.


At the present time, at least two nursing research journals, Scholarly Inquiry for Nursing Practice: An International Journal and Western Journal of Nursing Research, include commentaries after the research articles. In these journals, other researchers critically appraise the authors’ studies, and the authors have a chance to respond to these comments. Published research critical appraisals often increase the reader’s understanding of the study and the quality of the study findings (American Psychological Association [APA], 2010; Pyrczak, 2008). Another, more informal critique of a published study might appear in a letter to the editor, in which readers have the opportunity to comment on the strengths and weaknesses of published studies by writing to the journal editor.



Critical Appraisal of Abstracts for Conference Presentations


One of the most difficult types of critical appraisal is examining abstracts. The amount of information available is usually limited because many abstracts are restricted to 100 to 250 words. Nevertheless, reviewers must select the best-designed studies with the most significant outcomes for presentation at nursing conferences. This process requires an experienced researcher who needs few cues to determine the quality of a study. Critical appraisal of an abstract usually addresses the following criteria: (1) appropriateness of the study for the program; (2) completeness of the research project; (3) overall quality of the study problem, purpose, methodology, and results; (4) contribution of the study to the knowledge base of nursing; (5) contribution of the study to nursing theory; (6) originality of the work (not previously published); (7) implication of the study findings for practice; and (8) clarity, conciseness, and completeness of the abstract (APA, 2010).



Critical Appraisal of Research Articles for Publication


Nurse researchers who serve as peer reviewers for professional journals evaluate the quality of research articles submitted for publication. The role of these scientists is to ensure that the studies accepted for publication are well designed and contribute to the body of knowledge. Most of these reviews are conducted anonymously so that friendships or reputations do not interfere with the selection process (Pyrczak, 2008; Tilden, 2002). In most refereed journals, the experts who examine the research report have been selected from an established group of peer reviewers. Their comments or summaries of their comments are sent to the researcher. The editor also uses these comments to make selections for publication. The process for publishing a study is described in Chapter 27.



Critical Appraisal of Research Proposals


Critical appraisals of research proposals are conducted to approve student research projects; to permit data collection in an institution; and to select the best studies for funding by local, state, national, and international organizations and agencies. The process researchers use to seek the approval to conduct a study is presented in Chapter 28. The peer review process in federal funding agencies involves an extremely complex critical appraisal. Nurses are involved in this level of research review through the national funding agencies, such as the National Institute of Nursing Research (NINR, 2012), National Institutes of Health, and the Agency for Healthcare Research and Quality. Some of the criteria used to evaluate the quality of a proposal for possible funding include the (1) significance of the research problem and purpose for nursing, (2) appropriate use of methodology for the types of questions that the research is designed to answer, (3) appropriate use and interpretation of analysis procedures, (4) evaluation of clinical practice and forecasting of the need for nursing or other appropriate interventions, and (5) construction of models to direct the research and interpret the findings. The NINR (2012) website (http://www.ninr.nih.gov/ResearchAndFunding/) provides details on grant development and research funding, and Chapter 29 focuses on seeking funding for research.



Nurses’ Expertise in Critical Appraisal of Research


Conducting a critical appraisal of a study is a complex mental process that is stimulated by raising questions. The level of critique conducted is influenced by the sophistication of the individual appraising the study (Table 18-1). The initial critical appraisal of research by an undergraduate student often involves the identification of the steps of the research process in a quantitative study. Some baccalaureate programs include more in-depth research courses that include critical appraisals of the steps of quantitative studies and identification of the aspects of qualitative studies. A critical appraisal of research conducted by a student at the master’s level usually involves description of study strengths and weaknesses and evaluation of the credibility and meaning of the study findings for nursing knowledge and practice. Critical appraisals might focus on quantitative, qualitative, and outcomes studies.



At the doctoral level, students often critically appraise several studies in an area of interest and perform a complex synthesis of the research findings to determine the current empirical knowledge base for the phenomenon. These complex syntheses of quantitative, qualitative, outcomes, and intervention research include (1) systematic review of research, (2) meta-analysis, (3) meta-synthesis, and (4) mixed methods systematic review (see Table 18-1). These summaries of current research evidence are essential for providing EBP and directing future research (Craig & Smyth, 2012; Higgins & Green, 2008; Sandelowski & Barroso, 2007; Whittemore, 2005). Definitions of these types of complex syntheses are presented in Chapter 2, and Chapter 19 provides guidelines for critically appraising and conducting these syntheses.


The major focus of this chapter is conducting critical appraisals of quantitative and qualitative studies. These critical appraisals involve implementing some initial guidelines that are outlined in Box 18-1. These guidelines stress the importance of examining the expertise of the authors, reviewing the entire study, addressing the strengths and weaknesses of the study, and evaluating the credibility of the study findings (Fawcett & Garity, 2009; Marshall & Rossman, 2011; Munhall, 2012; Shadish, Cook, & Campbell, 2002). These guidelines provide a basis for the critical appraisal process for quantitative research that is discussed in the next section and the critical appraisal process for qualitative research discussed later.



Box 18-1   Guidelines for Conducting Critical Appraisals of Quantitative and Qualitative Research




1. Read and evaluate the entire study. A research appraisal requires comprehension of a study that includes the identification and examination of all steps of the research process.


2. Examine the research, clinical, and educational background of the authors. The authors need a clinical and scientific background that is appropriate for the study conducted.


3. Examine the organization and presentation of the research report. The title of the research report needs to indicate clearly the focus of the study. The report usually includes an abstract, introduction, methods, results, discussion, and references. The abstract of the study needs to present the purpose of the study clearly and to highlight the methodology and major results. The body of the report needs to be complete, concise, clearly presented, and logically organized. The references need to be complete and presented in a consistent format.


4. Identify the strengths and weaknesses of a study. All studies have strengths and weaknesses, and you can use the questions in this chapter to facilitate identification of them. Address the quality of the problem, purpose, methodology, results, and findings of quantitative and qualitative studies.


5. Provide specific examples of the strengths and weaknesses of a study. These examples provide a rationale and documentation for your critical appraisal of the study.


6. Be objective and realistic in identifying a study’s strengths and weaknesses. Do not be overly critical when identifying the weaknesses of a study or overly flattering when identifying the strengths.


7. Suggest modifications for future studies. Modifications should increase the strengths and decrease the weaknesses in the study.


8. Evaluate the study. Indicate the overall quality of the study and its contribution to nursing knowledge. Discuss the consistency of the findings of this study with the findings of previous studies. Discuss the need for further research and the potential implications of the findings for practice.



Critical Appraisal Process for Quantitative Research


The critical appraisal process for quantitative research includes three steps: (1) identifying the steps of the research process, (2) determining study strengths and weaknesses, and (3) evaluating the credibility and meaning of a study to nursing knowledge and practice. These steps occur in sequence, vary in depth, and presume accomplishment of the preceding steps. However, an individual with critical appraisal experience frequently performs several steps of this process simultaneously.


This section includes the three steps of the quantitative research critical appraisal process and provides relevant questions for each step. These questions are not comprehensive but have been selected as a means for stimulating the logical reasoning and analysis necessary for conducting a study review. Persons experienced in the critical appraisal process formulate additional questions as part of their reasoning processes. We cover the identification of the steps of the research process separately because persons who are new to critical appraisal start with this step. The questions for determining the study strengths and weaknesses are covered together because this process occurs simultaneously in the mind of the person conducting the critical appraisal. Evaluation is covered separately because of the increased expertise needed to perform this step.



Step I: Identifying the Steps of the Research Process in Studies


Initial attempts to comprehend research articles are often frustrating because the terminology and stylized manner of the report are unfamiliar. Identification of the steps of the research process in a quantitative study is the first step in critical appraisal. It involves understanding the terms and concepts in the report; identifying study elements; and grasping the nature, significance, and meaning of the study elements. The following guidelines are presented to direct you in identifying the elements or steps of a study.



Guidelines for Identifying the Steps of the Research Process


The first step involves reviewing the abstract and reading the study from beginning to end (see the guidelines in Box 18-1). As you read, address the following questions about the presentation of the study: Does the title clearly identify the focus of the study by including the major study variables and the population? Does the title indicate the type of study conducted—descriptive, correlational, quasi-experimental, or experimental (Kerlinger & Lee, 2000; Shadish et al., 2002)? Was the abstract clear? Was the writing style of the report clear and concise? Were the different parts of the research report plainly identified (APA, 2010)? Were relevant terms defined? You might underline the terms you do not understand and determine their meaning from the glossary at the end of this textbook. Read the article a second time and highlight or underline each step of the quantitative research process. An overview of these steps is presented in Chapter 3. To write a critical appraisal identifying the study steps, you need to identify each step of the research process concisely and respond briefly to the following guidelines and questions:



I Introduction



II State the problem.



III State the purpose.


IV Examine the literature review.



V Examine the study framework or theoretical perspective.



VI List any research objectives, questions, or hypotheses.


VII Identify and define (conceptually and operationally) the study variables or concepts that were identified in the objectives, questions, or hypotheses. If objectives, questions, or hypotheses are not stated, identify and define the variables in the study purpose and the results section of the study. If conceptual definitions are not found, identify possible definitions for each major study variable. Indicate which of the following types of variables were included in the study. A study usually includes independent and dependent variables or research variables but not all three types of variables.



VIII Identify demographic variables and other relevant terms.


IX Identify the research design.



X Describe the sample and setting.



A Identify inclusion or exclusion sample or eligibility criteria.


B Identify the specific type of probability or nonprobability sampling method that was used to obtain the sample. Did the researchers identify the sampling frame for the study (Thompson, 2002)?


C Identify the sample size. Discuss the refusal rate and include the rationale for refusal if presented in the article. Discuss the power analysis if this process was used to determine sample size (Aberson, 2010).


D Identify the sample attrition (number and percentage) and rationale for the study.


E Identify the characteristics of the sample.


F Discuss the institutional review board approval. Describe the informed consent process used in the study.


G Identify the study setting, and indicate if it is appropriate for the study purpose.


XI Identify and describe each measurement strategy used in the study. The following table includes the critical information about two measurement methods, the Beck Likert scale to measure depression and the physiological instrument to measure blood pressure. Completing this table allows you to cover essential measurement content for a study (Waltz, Strickland, & Lenz, 2010).



A Identify each study variable that was measured.


B Identify the name and author of each measurement strategy.


C Identify the type of each measurement strategy (e.g., Likert scale, visual analogue scale, physiological measure, and existing database).


D Identify the level of measurement (nominal, ordinal, interval, or ratio) achieved by each measurement method used in the study (Grove, 2007).


E Describe the reliability of each scale for previous studies and this study. Identify the precision of each physiological measure (Bialocerkowski, Klupp, & Bragge, 2010; DeVon et al., 2007).


F Identify the validity of each scale and the accuracy of physiological measures (DeVon et al., 2007; Ryan-Wenger, 2010).



























Variable Measured Name of Measurement Method/Author Type of Measurement Method Level of Measurement Reliability or Precision Validity or Accuracy
Depression Beck Depression Inventory/Beck Likert scale Interval Cronbach alpha of 0.82-0.92 from previous studies and 0.84 for this study. Reading level at 6th grade. Construct validity: Content validity from concept analysis, literature review, and reviews of experts. Convergent validity with Zung Depression Scale. Prediction validity of patients’ future depression episodes. Successive use validity with previous studies and this study.
Blood pressure (BP) Omron BP equipment/Health Care Equipment Agency Physiological measurement method Ratio Test-retest values of BP measurements in previous studies. BP equipment new and recalibrated every 50 BP readings in this study. Average 3 BP readings to determine BP. Documented accuracy of systolic and diastolic BPs to 1 mm Hg by company developing Omron BP cuff. Designated protocol for taking BP. Average 3 BP readings to determine BP.

Only gold members can continue reading. Log In or Register to continue

Related posts:

  1. Interpreting Research Outcomes
  2. Objectives, Questions, Hypotheses, and Study Variables
  3. Using Statistics to Examine Relationships
  4. Selecting a Quantitative Research Design

Stay updated, free articles. Join our Telegram channel
Tags:
Feb 17, 2017 | Posted by in NURSING | Comments Off on Critical Appraisal of Nursing Studies

Full access? Get Clinical Tree
Get Clinical Tree app for offline access