Evidence-based practice (EBP) can be defined as “the conscientious use of current best evidence in making decisions about patient care” (Melnyk & Fineout-Overholt, 2011, p. 4). More broadly, EBP may be described as “a lifelong problem-solving approach to clinical practice that integrates … the most relevant and best research, … one’s own clinical expertise … and patient preferences and values” (p. 4). In their definition of evidence-based medicine, Straus, Richardson, Glasziou, and Haynes (2005), include these same three primary components, but also add “patient circumstances” (p. 1) to account for both the patient’s clinical state and the clinical setting.

Traditional EBP models describe evidence generated from systematic reviews of clinically relevant randomized controlled trials as the strongest evidence upon which to base practice decisions, especially decisions about treatment. More recent literature also describes evidence generated from meta-syntheses of rigorously conducted qualitative studies as providing the strongest evidence for informing nurses’ care of patients through a more thorough understanding of the patients’ experience and the context within which care is provided (Flemming, 2007; Goethals, Dierckx de Casterlé, & Gastmans, 2011; Taylor, Shaw, Dale & French, 2011). Evidence from research is also called external evidence, reflecting the fact that it is intended to be generalizable or transferrable beyond the particular study setting or sample.

As is discussed later in this chapter, results of evaluation are not considered external evidence. Evaluations are not intended to be generalizable, but rather are conducted to determine the effectiveness of a specific intervention in a specific setting with an identified individual or group. Although not considered external evidence, results of a systematically conducted evaluation are still important from an EBP perspective. When evidence generated from research is not available, the conscientious use of internal evidence is appropriate. Results of a systematically conducted evaluation are one example of internal evidence (Melnyk & Fineout-Overholt, 2011).

Nurses’ understanding and use of EBP have evolved and expanded over the past two decades. One aspect of that evolution in recent years has been the adoption of the term practice-based evidence (Brownson & Jones, 2009; Girard, 2008; Green, 2008; Horn, Gassaway, Pentz, & James, 2010). Is discussion of practice-based evidence simply another way of addressing clinical expertise and patient circumstances, which are already components of EBP, and therefore redundant? Perhaps. A more useful perspective, however, might be to view practice-based evidence as a systematically constructed two-way bridge between external evidence and the practice setting. Defined as “the systematic collection of data about client progress generated during treatment to enhance the quality and outcomes of care” (Girard, 2008, p. 15), practice-based evidence comprises internal evidence that can be used both to identify whether a problem exists and to determine whether an intervention effectively resolved a problem. Put another way, practice-based evidence can be equally useful for assessment and for evaluation.

The results of evaluations, the outcomes of expert-delivered patient-centered care, and the results of quality improvement projects all represent internal evidence that is gathered by nurses and other healthcare professionals on an ongoing basis as an integral and important component of professional practice. Intentional recognition of these findings about current practice as a source of evidence to guide future practice requires that we think critically before acting and engage in
ongoing critical appraisal during and after each nurse-patient interaction.

Although assessment and evaluation are highly interrelated and the terms are often used interchangeably, they are not synonymous. The process of assessment focuses on gathering, summarizing, interpreting, and using data to decide a direction for action. In contrast, the process of evaluation involves gathering, summarizing, interpreting, and using data to determine the extent to which an action was successful.

The primary differences between these two terms reflect the processes’ timing and purpose. For example, an education program begins with an assessment of learners’ needs. From the perspective of systems theory, assessment data might be called the “input.” While the program is being conducted, periodic evaluation lets the educator know whether the program and learners’ progress are proceeding as planned. After program completion, evaluation identifies whether and to what extent identified needs were met. Again, from a systems theory perspective, these evaluative data might be called “intermediate output” and “output,” respectively.

An important note of caution: Although an evaluation is conducted at the end of a program, that is not the time to plan it. Evaluation as an afterthought is, at best, a poor idea and, at worst, a dangerous one. At this point in the educational program, data may be impossible to collect, be incomplete, or even be misleading. Ideally, assessment and evaluation planning should be concurrent activities. When feasible, the same data collection methods and instruments should be used. This approach is especially appropriate for outcome and impact evaluations, as is discussed later in this chapter. “If only …” is an all too frequently heard lament, which can be minimized by planning ahead.

In planning any evaluation, the first and most crucial step is to determine the focus of the evaluation. This focus then guides evaluation design, conduct, data analysis, and reporting of results. The importance of a clear, specific, and realistic evaluation focus cannot be overemphasized. Usefulness and accuracy of the results of an evaluation depend heavily on how well the evaluation is initially focused.

Evaluation focus includes five basic components: (1) audience, (2) purpose, (3) questions, (4) scope, and (5) resources (Ruzicki, 1987). To identify these components, ask the following questions:

1. For which audience is the evaluation being conducted?

2. For which purpose is the evaluation being conducted?

3. Which questions will be asked in the evaluation?

4. What is the scope of the evaluation?

5. Which resources are available to conduct the evaluation?

The audience comprises the persons or groups for whom the evaluation is being conducted (Dillon, Barga, & Goodin, 2012; Ruzicki, 1987). These individuals or groups include the primary audience, or the individual or group who requested the evaluation, and the general audience, or all those who will use evaluation results or who might benefit from the evaluation. Thus the audience for an evaluation might include patients, peers, the manager of a unit, a supervisor, the nursing director, the staff development director, the chief executive officer of the institution, or a group of community leaders.

When an evaluator reports results of the evaluation, he or she provides feedback to all members of the audience. In focusing the evaluation,
however, the evaluator must first consider the primary audience. Giving priority to the individual or group that requested the evaluation makes it easier to focus the evaluation, especially if the results of an evaluation will be used by several groups representing diverse interests.

The purpose of the evaluation is the answer to the question, “Why is the evaluation being conducted?” For example, the purpose might be to decide whether to continue a particular education program or to determine the effectiveness of the teaching process. If a particular individual or group has a primary interest in results of the evaluation, input from that group can clarify the purpose.

An important note of caution: Why an evaluator is conducting an evaluation is not synonymous with who or what the evaluator is evaluating. For example, nursing literature on patient education commonly distinguishes among three types of evaluations: (1) learner, (2) teacher, and (3) educational activity. This distinction answers the question of who or what will be evaluated and is extremely useful for evaluation design and conduct. The question of why learner evaluation might be undertaken, for example, is answered by the reason or purpose for evaluating learner performance. If the purpose for evaluating learner performance is to determine whether the learner gained sufficient skill to perform a self-care activity (e.g., foot care), the educator might design a content evaluation that includes one or more return demonstrations by an individual learner prior to hospital discharge. If the purpose for evaluating learner performance is to determine whether the learner is able to conduct the same self-care activity at home on a regular basis, the educator might design an outcome evaluation that includes both observing the learner perform the activity in his or her home environment and measuring other clinical parameters (e.g., skin integrity) that would improve and be maintained with regular and ongoing self-care. Content and outcome evaluations are discussed in more detail later in this chapter.

An excellent rule of thumb in stating the purpose of an evaluation is to keep it singular. In other words, the evaluator should state, “The purpose is …,” not “The purposes are …”. Keeping the purpose focused on the audience and singular helps avoid the all too frequently encountered tendency to attempt too much in one evaluation. An exception to this rule is when undertaking a total program evaluation. As discussed later in this chapter, program evaluations are naturally broad in scope, focusing simultaneously on learners, teachers, and educational offerings.

Questions to be asked in the evaluation are directly related to the purpose for conducting the evaluation, are specific, and are measurable. Examples of such questions include “To what extent are patients satisfied with the cardiac discharge teaching program?” and “How frequently do staff nurses use the diabetes teaching reference materials?” Asking the right questions is crucial if the evaluation is to fulfill the intended purpose. As discussed later in this chapter, delineation of evaluation questions is both the first step in selection of evaluation design and the basis for eventual data analysis.

The scope of an evaluation can be considered an answer to the question, “How much will be evaluated?” “How much” includes “How many aspects of education will be evaluated?”, “How many individuals or representative groups will be evaluated?”, and “What time period is to be evaluated?” For example, will the evaluation focus on one class or on an entire program? Will it focus on the learning experience for one patient or for all patients being taught a particular skill? Evaluation could be limited to the teaching process during a particular patient education class, or it could be expanded to encompass both the teaching process and related patient outcomes of learning.

The scope of an evaluation is determined in part by the purpose for conducting the evaluation and in part by available resources. For example, an evaluation addressing learner satisfaction with educators for all programs conducted by a staff development department in a given year is necessarily broad and long term in scope; such a vast undertaking requires expertise in data collection and analysis. By comparison, an evaluation to determine whether a patient understands each step in a learning session on how to self-administer insulin injections is narrow in scope, is focused on a particular point in time, and requires expertise in clinical practice and observation.

Resources needed to conduct an evaluation include time, expertise, personnel, materials, equipment, and facilities. A realistic appraisal of which resources are accessible and available relative to which resources are required is crucial in focusing any evaluation. Someone conducting an evaluation should remember to include the time and expertise required to collate, analyze, and interpret data and to prepare the report of evaluation results.

Evaluation can be classified into different types, or categories, based on one or more of the five components just described. The most common types of evaluation identified include some combination of process, or formative evaluation, and outcome, or summative evaluation. Other types of evaluation include content, outcome, impact, and total program evaluations. A number of evaluation models defining these evaluation types and how they relate to one another have been developed (Abruzzese, 1992; Frye & Hemmer, 2012; Milne, 2007; Noar, 2012; Ogrinc & Batalden, 2009; Rankin & Stallings, 2005; Rouse, 2011; Suhayda & Miller, 2006). Because not all models define types of evaluation in the same manner, the nurse educator should choose the model that is most appropriate for the given purpose and most realistic given the available resources.

Abruzzese (1992) constructed the Roberta Straessle Abruzzese (RSA) evaluation model for conceptualizing, or classifying, educational evaluation into different categories or levels. Although developed in 1978 and derived from the perspective of staff development education, the RSA model remains useful for conceptualizing types of evaluation from both staff development and patient education perspectives. Two more recent examples of use of the RSA model are given by Dilorio, Price, and Becker (2001) in their discussion of the evaluation of the Neuroscience Nurse Internship Program at the National Institutes of Health Clinical Center, and by Underwood, Dahlen-Hartfield, and Mogle (2004) in their study of nurses’ perceived expertise following three continuing education programs. Sumner, Burke, Chang, McAdams, and Jones (2012) used the RSA model to support development of a study to evaluate process, content, and outcome evaluation of a basic arrhythmia course taught to registered nurses.

The RSA model pictorially places five basic types of evaluation in relation to one another based on purpose and related questions, scope, and resource components of evaluation focus (Figure 14-1). The five types of evaluation include process, content, outcome, impact, and total program. Abruzzese describes the first four types as levels of evaluation leading from the simple (process evaluation) to the complex (impact evaluation). Total program evaluation encompasses and summarizes all four levels.