Critical Appraisal of Quantitative and Qualitative Research for Nursing Practice



Critical Appraisal of Quantitative and Qualitative Research for Nursing Practice




The nursing profession continually strives for evidence-based practice (EBP), which includes critically appraising studies, synthesizing the findings, applying the scientific evidence in practice, and determining the practice outcomes (Brown, 2014; Doran, 2011; Melnyk & Fineout-Overholt, 2011). Critically appraising studies is an essential step toward basing your practice on current research findings. The term critical appraisal or critique is an examination of the quality of a study to determine the credibility and meaning of the findings for nursing. Critique is often associated with criticize, a word that is frequently viewed as negative. In the arts and sciences, however, critique is associated with critical thinking and evaluation—tasks requiring carefully developed intellectual skills. This type of critique is referred to as an intellectual critical appraisal. An intellectual critical appraisal is directed at the element that is created, such as a study, rather than at the creator, and involves the evaluation of the quality of that element. For example, it is possible to conduct an intellectual critical appraisal of a work of art, an essay, and a study.


The idea of the intellectual critical appraisal of research was introduced earlier in this text and has been woven throughout the chapters. As each step of the research process was introduced, guidelines were provided to direct the critical appraisal of that aspect of a research report. This chapter summarizes and builds on previous critical appraisal content and provides direction for conducting critical appraisals of quantitative and qualitative studies. The background provided by this chapter serves as a foundation for the critical appraisal of research syntheses (systematic reviews, meta-analyses, meta-syntheses, and mixed-methods systematic reviews) presented in Chapter 13.


This chapter discusses the implementation of critical appraisals in nursing by students, practicing nurses, nurse educators, and researchers. The key principles for implementing intellectual critical appraisals of quantitative and qualitative studies are described to provide an overview of the critical appraisal process. The steps for critical appraisal of quantitative studies, focused on rigor, design validity, quality, and meaning of findings, are detailed, and an example of a critical appraisal of a published quantitative study is provided. The chapter concludes with the critical appraisal process for qualitative studies and an example of a critical appraisal of a qualitative study.



When are Critical Appraisals of Studies Implemented in Nursing?


In general, studies are critically appraised to broaden understanding, summarize knowledge for practice, and provide a knowledge base for future research. Studies are critically appraised for class projects and to determine the research evidence ready for use in practice. In addition, critical appraisals are often conducted after verbal presentations of studies, after a published research report, for selection of abstracts when studies are presented at conferences, for article selection for publication, and for evaluation of research proposals for implementation or funding. Therefore nursing students, practicing nurses, nurse educators, and nurse researchers are all involved in the critical appraisal of studies.



Students’ Critical Appraisal of Studies


One aspect of learning the research process is being able to read and comprehend published research reports. However, conducting a critical appraisal of a study is not a basic skill, and the content presented in previous chapters is essential for implementing this process. Students usually acquire basic knowledge of the research process and critical appraisal process in their baccalaureate program. More advanced analysis skills are often taught at the master’s and doctoral levels. Performing a critical appraisal of a study involves the following three steps, which are detailed in this chapter: (1) identifying the steps or elements of the study; (2) determining the study strengths and limitations; and (3) evaluating the credibility and meaning of the study findings. By critically appraising studies, you will expand your analysis skills, strengthen your knowledge base, and increase your use of research evidence in practice. Striving for EBP is one of the competencies identified for associate degree and baccalaureate degree (prelicensure) students by the Quality and Safety Education for Nurses (QSEN, 2013) project, and EBP requires critical appraisal and synthesis of study findings for practice (Sherwood & Barnsteiner, 2012). Therefore critical appraisal of studies is an important part of your education and your practice as a nurse.



Critical Appraisal of Studies by Practicing Nurses, Nurse Educators, and Researchers


Practicing nurses need to appraise studies critically so that their practice is based on current research evidence and not on tradition or trial and error (Brown, 2014; Craig & Smyth, 2012). Nursing actions need to be updated in response to current evidence that is generated through research and theory development. It is important for practicing nurses to design methods for remaining current in their practice areas. Reading research journals and posting or e-mailing current studies at work can increase nurses’ awareness of study findings but are not sufficient for critical appraisal to occur. Nurses need to question the quality of the studies, credibility of the findings, and meaning of the findings for practice. For example, nurses might form a research journal club in which studies are presented and critically appraised by members of the group (Gloeckner & Robinson, 2010).


Skills in critical appraisal of research enable practicing nurses to synthesize the most credible, significant, and appropriate evidence for use in their practice. EBP is essential in agencies that are seeking or maintaining Magnet status. The Magnet Recognition Program was developed by the American Nurses Credentialing Center (ANCC, 2013) to “recognize healthcare organizations for quality patient care, nursing excellence, and innovations in professional nursing,” which requires implementing the most current research evidence in practice (see www.nursecredentialing.org/Magnet/ProgramOverview.aspx).


Your faculty members critically appraise research to expand their clinical knowledge base and to develop and refine the nursing educational process. The careful analysis of current nursing studies provides a basis for updating curriculum content for use in clinical and classroom settings. Faculty serve as role models for their students by examining new studies, evaluating the information obtained from research, and indicating which research evidence to use in practice. For example, nursing instructors might critically appraise and present the most current evidence about caring for people with hypertension in class and role-model the management of patients with hypertension in practice.


Nurse researchers critically appraise previous research to plan and implement their next study. Many researchers have a program of research in a selected area, and they update their knowledge base by critically appraising new studies in this area. For example, selected nurse researchers have a program of research to identify effective interventions for assisting patients in managing their hypertension and reducing their cardiovascular risk factors.



Critical Appraisal of Research Following Presentation and Publication


When nurses attend research conferences, they note that critical appraisals and questions often follow presentations of studies. These critical appraisals assist researchers in identifying the strengths and weaknesses of their studies and generating ideas for further research. Participants listening to study critiques might gain insight into the conduct of research. In addition, experiencing the critical appraisal process can increase the conference participants’ ability to evaluate studies and judge the usefulness of the research evidence for practice.


Critical appraisals have been published following some studies in research journals. For example, the research journals Scholarly Inquiry for Nursing Practice: An International Journal and Western Journal of Nursing Research include commentaries after the research articles. In these commentaries, other researchers critically appraise the authors’ studies, and the authors have a chance to respond to these comments. Published research critical appraisals often increase the reader’s understanding of the study and the quality of the study findings (American Psychological Association [APA], 2010). A more informal critical appraisal of a published study might appear in a letter to the editor. Readers have the opportunity to comment on the strengths and weaknesses of published studies by writing to the journal editor.



Critical Appraisal of Research for Presentation and Publication


Planners of professional conferences often invite researchers to submit an abstract of a study they are conducting or have completed for potential presentation at the conference. The amount of information available is usually limited, because many abstracts are restricted to 100 to 250 words. Nevertheless, reviewers must select the best-designed studies with the most significant outcomes for presentation at nursing conferences. This process requires an experienced researcher who needs few cues to determine the quality of a study. Critical appraisal of an abstract usually addresses the following criteria: (1) appropriateness of the study for the conference program; (2) completeness of the research project; (3) overall quality of the study problem, purpose, methodology, and results; (4) contribution of the study to nursing’s knowledge base; (5) contribution of the study to nursing theory; (6) originality of the work (not previously published); (7) implication of the study findings for practice; and (8) clarity, conciseness, and completeness of the abstract (APA, 2010; Grove, Burns, & Gray, 2013).


Some nurse researchers serve as peer reviewers for professional journals to evaluate the quality of research papers submitted for publication. The role of these scientists is to ensure that the studies accepted for publication are well designed and contribute to the body of knowledge. Journals that have their articles critically appraised by expert peer reviews are called peer-reviewed journals or referred journals (Pyrczak, 2008). The reviewers’ comments or summaries of their comments are sent to the researchers to direct their revision of the manuscripts for publication. Referred journals usually have studies and articles of higher quality and provide excellent studies for your review for practice.



Critical Appraisal of Research Proposals


Critical appraisals of research proposals are conducted to approve student research projects, permit data collection in an institution, and select the best studies for funding by local, state, national, and international organizations and agencies. You might be involved in a proposal review if you are participating in collecting data as part of a class project or studies done in your clinical agency. More details on proposal development and approval can be found in Grove et al. (2013, Chapter 28).


Research proposals are reviewed for funding from selected government agencies and corporations. Private corporations develop their own format for reviewing and funding research projects (Grove et al., 2013). The peer review process in federal funding agencies involves an extremely complex critical appraisal. Nurses are involved in this level of research review through national funding agencies, such as the National Institute of Nursing Research (NINR, 2013) and the Agency for Healthcare Research and Quality (AHRQ, 2013).



What are the Key Principles for Conducting Intellectual Critical Appraisals of Quantitative and Qualitative Studies?


An intellectual critical appraisal of a study involves a careful and complete examination of a study to judge its strengths, weaknesses, credibility, meaning, and significance for practice. A high-quality study focuses on a significant problem, demonstrates sound methodology, produces credible findings, indicates implications for practice, and provides a basis for additional studies (Grove et al., 2013; Hoare & Hoe, 2013; Hoe & Hoare, 2012). Ultimately, the findings from several quality studies can be synthesized to provide empirical evidence for use in practice (O’Mathuna, Fineout-Overholt, & Johnston, 2011).


The major focus of this chapter is conducting critical appraisals of quantitative and qualitative studies. These critical appraisals involve implementing some key principles or guidelines, outlined in Box 12-1. These guidelines stress the importance of examining the expertise of the authors, reviewing the entire study, addressing the study’s strengths and weaknesses, and evaluating the credibility of the study findings (Fawcett & Garity, 2009; Hoare & Hoe, 2013; Hoe & Hoare, 2012; Munhall, 2012). All studies have weaknesses or flaws; if every flawed study were discarded, no scientific evidence would be available for use in practice. In fact, science itself is flawed. Science does not completely or perfectly describe, explain, predict, or control reality. However, improved understanding and increased ability to predict and control phenomena depend on recognizing the flaws in studies and science. Additional studies can then be planned to minimize the weaknesses of earlier studies. You also need to recognize a study’s strengths to determine the quality of a study and credibility of its findings. When identifying a study’s strengths and weaknesses, you need to provide examples and rationale for your judgments that are documented with current literature.



Box 12-1   Key Principles for Critically Appraising Quantitative and Qualitative Studies




1. Read and critically appraise the entire study. A research critical appraisal involves examining the quality of all aspects of the research report.


2. Examine the organization and presentation of the research report. A well-prepared report is complete, concise, clearly presented, and logically organized. It does not include excessive jargon that is difficult for you to read. The references need to be current, complete, and presented in a consistent format.


3. Examine the significance of the problem studied for nursing practice. The focus of nursing studies needs to be on significant practice problems if a sound knowledge base is to be developed for evidence-based nursing practice.


4. Indicate the type of study conducted and identify the steps or elements of the study. This might be done as an initial critical appraisal of a study; it indicates your knowledge of the different types of quantitative and qualitative studies and the steps or elements included in these studies.


5. Identify the strengths and weaknesses of a study. All studies have strengths and weaknesses, so attention must be given to all aspects of the study.


6. Be objective and realistic in identifying the study’s strengths and weaknesses. Be balanced in your critical appraisal of a study. Try not to be overly critical in identifying a study’s weaknesses or overly flattering in identifying the strengths.


7. Provide specific examples of the strengths and weaknesses of a study. Examples provide evidence for your critical appraisal of the strengths and weaknesses of a study.


8. Provide a rationale for your critical appraisal comments. Include justifications for your critical appraisal, and document your ideas with sources from the current literature. This strengthens the quality of your critical appraisal and documents the use of critical thinking skills.


9. Evaluate the quality of the study. Describe the credibility of the findings, consistency of the findings with those from other studies, and quality of the study conclusions.


10. Discuss the usefulness of the findings for practice. The findings from the study need to be linked to the findings of previous studies and examined for use in clinical practice.


Critical appraisal of quantitative and qualitative studies involves a final evaluation to determine the credibility of the study findings and any implications for practice and further research (see Box 12-1). Adding together the strong points from multiple studies slowly builds a solid base of evidence for practice. These guidelines provide a basis for the critical appraisal process for quantitative research discussed in the next section and the critical appraisal process for qualitative research (see later).



Understanding the Quantitative Research Critical Appraisal Process


The quantitative research critical appraisal process includes three steps: (1) identifying the steps of the research process in studies; (2) determining study strengths and weaknesses; and (3) evaluating the credibility and meaning of study findings. These steps occur in sequence, vary in depth, and presume accomplishment of the preceding steps. However, an individual with critical appraisal experience frequently performs two or three steps of this process simultaneously.


This section includes the three steps of the quantitative research critical appraisal process and provides relevant questions for each step. These questions have been selected as a means for stimulating the logical reasoning and analysis necessary for conducting a critical appraisal of a study. Those experienced in the critical appraisal process often formulate additional questions as part of their reasoning processes. We will identify the steps of the research process separately because those new to critical appraisal start with this step. The questions for determining the study strengths and weaknesses are covered together because this process occurs simultaneously in the mind of the person conducting the critical appraisal. Evaluation is covered separately because of the increased expertise needed to perform this step.



Step 1: Identifying the Steps of the Research Process in Studies


Initial attempts to comprehend research articles are often frustrating because the terminology and stylized manner of the report are unfamiliar. Identifying the steps of the research process in a quantitative study is the first step in critical appraisal. It involves understanding the terms and concepts in the report, as well as identifying study elements and grasping the nature, significance, and meaning of these elements. The following guidelines will direct you in identifying a study’s elements or steps.



Guidelines for Identifying the Steps of the Research Process in Studies


The first step involves reviewing the abstract and reading the study from beginning to end. As you read, think about the following questions regarding the presentation of the study:



• Was the study title clear?


• Was the abstract clearly presented?


• Was the writing style of the report clear and concise?


• Were relevant terms defined? You might underline the terms you do not understand and determine their meaning from the glossary at the end of this text.


• Were the following parts of the research report plainly identified (APA, 2010)?



We recommend reading the research article a second time and highlighting or underlining the steps of the quantitative research process that were identified previously. An overview of these steps is presented in Chapter 2. After reading and comprehending the content of the study, you are ready to write your initial critical appraisal of the study. To write a critical appraisal identifying the study steps, you need to identify each step of the research process concisely and respond briefly to the following guidelines and questions.



1. Introduction



2. State the problem.



3. State the purpose.


4. Examine the literature review.



5. Examine the study framework or theoretical perspective.



6. List any research objectives, questions, or hypotheses.


7. Identify and define (conceptually and operationally) the study variables or concepts that were identified in the objectives, questions, or hypotheses. If objectives, questions, or hypotheses are not stated, identify and define the variables in the study purpose and results section of the study. If conceptual definitions are not found, identify possible definitions for each major study variable. Indicate which of the following types of variables were included in the study. A study usually includes independent and dependent variables or research variables, but not all three types of variables.



8. Identify attribute or demographic variables and other relevant terms.


9. Identify the research design.



10. Describe the sample and setting.



a. Identify inclusion and exclusion sample or eligibility criteria.


b. Identify the specific type of probability or nonprobability sampling method that was used to obtain the sample. Did the researchers identify the sampling frame for the study?


c. Identify the sample size. Discuss the refusal number and percentage, and include the rationale for refusal if presented in the article. Discuss the power analysis if this process was used to determine sample size (Aberson, 2010).


d. Identify the sample attrition (number and percentage) for the study.


e. Identify the characteristics of the sample.


f. Discuss the institutional review board (IRB) approval. Describe the informed consent process used in the study.


g. Identify the study setting and indicate whether it is appropriate for the study purpose.


11. Identify and describe each measurement strategy used in the study. The following table includes the critical information about two measurement methods, the Beck Likert scale and a physiological instrument to measure blood pressure. Completing this table will allow you to cover essential measurement content for a study (Waltz, Strickland, & Lenz, 2010).



a. Identify each study variable that was measured.


b. Identify the name and author of each measurement strategy.


c. Identify the type of each measurement strategy (e.g., Likert scale, visual analog scale, physiological measure, or existing database).


d. Identify the level of measurement (nominal, ordinal, interval, or ratio) achieved by each measurement method used in the study (Grove, 2007).


e. Describe the reliability of each scale for previous studies and this study. Identify the precision of each physiological measure (Bialocerkowski, Klupp, & Bragge, 2010; DeVon et al., 2007).


f. Identify the validity of each scale and the accuracy of physiological measures (DeVon et al., 2007; Ryan-Wenger, 2010).



























Variable Measured Name of Measurement Method (Author) Type of Measurement Method Level of Measurement Reliability or Precision Validity or Accuracy
Depression Beck Depression Inventory (Beck) Likert scale Interval Cronbach alphas of 0.82-0.92 from previous studies and 0.84 for this study; reading level at 6th grade. Construct validity—content validity from concept analysis, literature review, and reviews of experts; convergent validity of 0.04 with Zung Depression Scale; predictive validity of patients’ future depression episodes; successive use validity with the conduct of previous studies and this study.
Blood pressure Omron blood pressure (BP) equipment
(equipment manufacturer)
Physiological measurement method Ratio Test-retest values of BPs in previous studies; BP equipment new and recalibrated every 50 BP readings in this study; average three BP readings to determine BP. Documented accuracy of systolic and diastolic BPs to 1 mm Hg by company developing Omron BP cuff; designated protocol for taking BP average three BP readings to determine BP.


image


12. Describe the procedures for data collection.


13. Describe the statistical analyses used.



a. List the statistical procedures used to describe the sample (Grove, 2007).


b. Was the level of significance or alpha identified? If so, indicate what it was (0.05, 0.01, or 0.001).


c. Complete the following table with the analysis techniques conducted in the study: (1) identify the focus (description, relationships, or differences) for each analysis technique; (2) list the statistical analysis technique performed; (3) list the statistic; (4) provide the specific results; and (5) identify the probability (p) of the statistical significance achieved by the result (Grove, 2007; Grove et al., 2013; Hoare & Hoe, 2013; Plichta & Kelvin, 2013).




































Purpose of Analysis Analysis Technique Statistic Results Probability (p)
Description of subjects’ pulse rate Mean
Standard deviation
Range
M
SD
range
71.52
5.62
58-97
 
Difference between adult males and females on blood pressure t-Test t 3.75 p = 0.001
Differences of diet group, exercise group, and comparison group for pounds lost in adolescents Analysis of variance F 4.27 p = 0.04
Relationship of depression and anxiety in older adults Pearson correlation r 0.46 p = 0.03


image


14. Describe the researcher’s interpretation of findings.



15. What study limitations did the researcher identify?


16. What conclusions did the researchers identify based on their interpretation of the study findings?


17. How did the researcher generalize the findings?


18. What were the implications of the findings for nursing practice?


19. What suggestions for further study were identified?


20. Is the description of the study sufficiently clear for replication?



Step 2: Determining the Strengths and Weaknesses in Studies


The second step in critically appraising studies requires determining strengths and weaknesses in the studies. To do this, you must have knowledge of what each step of the research process should be like from expert sources such as this text and other research sources (Aberson, 2010; Bialocerkowski et al., 2010; Brown, 2014; Creswell, 2014; DeVon et al., 2007; Doran, 2011; Fawcett & Garity, 2009; Grove, 2007; Grove et al., 2013; Hoare & Hoe, 2013; Hoe & Hoare, 2012; Morrison, Hoppe, Gillmore, Kluver, Higa, & Wells, 2009; O’Mathuna et al., 2011; Ryan-Wenger, 2010; Santacroce, Maccarelli, & Grey, 2004; Shadish et al., 2002; Waltz et al., 2010). The ideal ways to conduct the steps of the research process are then compared with the actual study steps. During this comparison, you examine the extent to which the researcher followed the rules for an ideal study, and the study elements are examined for strengths and weaknesses.


You also need to examine the logical links or flow of the steps in the study being appraised. For example, the problem needs to provide background and direction for the statement of the purpose. The variables identified in the study purpose need to be consistent with the variables identified in the research objectives, questions, or hypotheses. The variables identified in the research objectives, questions, or hypotheses need to be conceptually defined in light of the study framework. The conceptual definitions should provide the basis for the development of operational definitions. The study design and analyses need to be appropriate for the investigation of the study purpose, as well as for the specific objectives, questions, or hypotheses. Examining the quality and logical links among the study steps will enable you to determine which steps are strengths and which steps are weaknesses.



Guidelines for Determining the Strengths and Weaknesses in Studies


The following questions were developed to help you examine the different steps of a study and determine its strengths and weaknesses. The intent is not for you to answer each of these questions but to read the questions and then make judgments about the steps in the study. You need to provide a rationale for your decisions and document from relevant research sources, such as those listed previously in this section and in the references at the end of this chapter. For example, you might decide that the study purpose is a strength because it addresses the study problem, clarifies the focus of the study, and is feasible to investigate (Brown, 2014; Fawcett & Garity, 2009; Hoe & Hoare, 2012).



1. Research problem and purpose



2. Review of literature



3. Study framework



4. Research objectives, questions, or hypotheses



5. Variables



6. Design



a. Is the design used in the study the most appropriate design to obtain the needed data (Creswell, 2014; Grove et al., 2013; Hoe & Hoare, 2012)?


b. Does the design provide a means to examine all the objectives, questions, or hypotheses?


c. Is the treatment clearly described (Brown, 2002)? Is the treatment appropriate for examining the study purpose and hypotheses? Does the study framework explain the links between the treatment (independent variable) and the proposed outcomes (dependent variables)? Was a protocol developed to promote consistent implementation of the treatment to ensure intervention fidelity (Morrison et al., 2009)? Did the researcher monitor implementation of the treatment to ensure consistency (Santacroce et al., 2004)? If the treatment was not consistently implemented, what might be the impact on the findings?


d. Did the researcher identify the threats to design validity (statistical conclusion validity, internal validity, construct validity, and external validity [see Chapter 8]) and minimize them as much as possible (Grove et al., 2013; Shadish et al., 2002)?


e. If more than one group was used, did the groups appear equivalent?


f. If a treatment was implemented, were the subjects randomly assigned to the treatment group or were the treatment and comparison groups matched? Were the treatment and comparison group assignments appropriate for the purpose of the study?


7. Sample, population, and setting



a. Is the sampling method adequate to produce a representative sample? Are any subjects excluded from the study because of age, socioeconomic status, or ethnicity, without a sound rationale?


b. Did the sample include an understudied population, such as young people, older adults, or minority group?


c. Were the sampling criteria (inclusion and exclusion) appropriate for the type of study conducted (O’Mathuna et al., 2011)?


d. Was a power analysis conducted to determine sample size? If a power analysis was conducted, were the results of the analysis clearly described and used to determine the final sample size? Was the attrition rate projected in determining the final sample size (Aberson, 2010)?


e. Are the rights of human subjects protected (Creswell, 2014; Grove et al., 2013)?


f. Is the setting used in the study typical of clinical settings?


g. Was the rate of potential subjects’ refusal to participate in the study a problem? If so, how might this weakness influence the findings?


h. Was sample attrition a problem? If so, how might this weakness influence the final sample and the study results and findings (Aberson, 2010; Fawcett & Garity, 2009; Hoe & Hoare, 2012)?


8. Measurements



a. Do the measurement methods selected for the study adequately measure the study variables? Should additional measurement methods have been used to improve the quality of the study outcomes (Waltz et al., 2010)?


b. Do the measurement methods used in the study have adequate validity and reliability? What additional reliability or validity testing is needed to improve the quality of the measurement methods (Bialocerkowski et al., 2010; DeVon et al., 2007; Roberts & Stone, 2003)?


c. Respond to the following questions, which are relevant to the measurement approaches used in the study:



1) Scales and questionnaires



2) Observation



3) Interviews



4) Physiological measures



9. Data collection



10. Data analysis



a. Are data analysis procedures appropriate for the type of data collected (Grove, 2007; Hoare & Hoe, 2013; Plichta & Kelvin, 2013)?


b. Are data analysis procedures clearly described? Did the researcher address any problem with missing data, and explain how this problem was managed?


c. Do the data analysis techniques address the study purpose and the research objectives, questions, or hypotheses (Fawcett & Garity, 2009; Grove et al., 2013; Hoare & Hoe, 2013)?


d. Are the results presented in an understandable way by narrative, tables, or figures or a combination of methods (APA, 2010)?


e. Is the sample size sufficient to detect significant differences, if they are present?


f. Was a power analysis conducted for nonsignificant results (Aberson, 2010)?


g. Are the results interpreted appropriately?

Feb 9, 2017 | Posted by in NURSING | Comments Off on Critical Appraisal of Quantitative and Qualitative Research for Nursing Practice
Premium Wordpress Themes by UFO Themes