18
PROGRAM EVALUATION AND ACCREDITATION
Program evaluation is the process of judging the worth or value of an educational program. One purpose of program evaluation is to provide data on which to base decisions about the educational program. Another purpose is to provide evidence of educational effectiveness in response to internal and external demands for accountability. With the demand for high-quality programs, development of newer models for the delivery of higher education such as online programs, and public calls for accountability, there has been a greater emphasis on systematic and ongoing program evaluation. This chapter presents an overview of accreditation of nursing education programs, standards for evaluating distance education programs, program evaluation models, and the evaluation of selected program components (curriculum, outcomes, and teaching). The chapter also describes how to develop a program evaluation plan for a nursing education program.
Accreditation of Nursing Education Programs
Accreditation is a means of ensuring that institutions of higher education and nursing education programs meet standards and of indicating to the public that they offer a quality education for students. In this way, accreditation protects the public. Accreditation involves internally reviewing and assessing one’s own programs on a continuous basis to identify areas requiring revision and participating in an external review, including a site visit, to verify the standards are met.
Nursing accreditation is not the same as regulation of nursing education programs. The state boards of nursing are regulatory bodies: They set standards for nursing practice and education in a state. Nursing education programs cannot operate without approval from the state board (Halstead, 2017). Boards of nursing, which are governmental agencies, have this authority because they have responsibility in the state to protect the health and welfare of their citizens by overseeing and ensuring safe nursing practice (National Council of State Boards of Nursing, 2019). Each state board has its own set of standards that nursing education programs within the state need to meet for initial and continuing approval. State boards have the authority 352to close a program or require improvements to continue to operate, for example, if NCLEX® (National Council Licensure Examination) scores or retention rates are below the state’s benchmarks for a designated period of time.
Accreditation is a voluntary process done by nongovernmental agencies. Although accreditation is voluntary, in many ways, it is difficult for nonaccredited nursing education programs to attract students because financial aid for students from both the federal and state governments is contingent on attending an accredited nursing education program.
Types of Accreditation
There are two types of accreditation: institutional (at the university or college level) and programmatic (at the program or discipline level). Institutional accreditation is done by regional accrediting bodies such as the Southern Association of Colleges and Schools Commission on Colleges, and the Middle States Commission on Higher Education. Each region of the United States has its own regional accrediting body. At the institutional level, there also are accrediting bodies for faith-based colleges and career-related schools and programs.
Programmatic or specialized accreditation is performed for a field of study, for example, nursing, business, engineering, and medicine, among others. Programmatic accreditation is typically found in the professions, meeting their responsibility to ensure quality of programs to protect the public. There are three accrediting bodies for nursing education programs in the United States: Accrediting Commission for Education in Nursing (ACEN), Commission on Collegiate Nursing Education (CCNE), and Commission for Nursing Education Accreditation (CNEA). Two of these agencies (ACEN and CNEA) accredit all levels of nursing education programs: practical, diploma, associate, baccalaureate, masters, and DNP (ACEN, 2019b; CNEA, 2019). CCNE accredits baccalaureate and master’s degree nursing programs, practice-focused nursing doctoral programs that award the DNP degree, postgraduate APRN certificate programs, and entry-to-practice residency programs (CCNE, 2019). The Canadian Association of Schools of Nursing serves as the national accrediting body for RN education in Canada. In specialty areas of nursing practice, such as nurse anesthesia, there are specialty accrediting bodies (e.g., Council on Accreditation of Nurse Anesthesia).
PhD in nursing programs are not accredited by any of these agencies because PhD (and other research-focused doctoral) programs usually are offered through the graduate school of the university. They are evaluated using processes determined by the university, which usually include an internal review and self-study, followed by an external review and site visit, similar to nursing accreditation processes.
Program evaluation based on an accreditation model is designed to assess whether the program meets external standards of quality. Although there are differences in the 353names of the standards, the areas evaluated and descriptions of quality of nursing education programs in those areas are similar. Table 18.1 lists the areas addressed by the standards for the three accrediting bodies for nursing education. The focus is on quality improvement—collecting information about the program and its components on a continuous basis to identify what is working well and decide what changes may be needed.
ACEN, Accreditation Commission for Education in Nursing; CCNE, Commission on Collegiate Nursing Education;
CNEA, Commission for Nursing Education Accreditation
354Evaluation of Distance Education Programs Using an Accreditation Model
Although no specific accreditation standards exist for online nursing programs, ACEN (2019a), CCNE (2018), and CNEA (2016) incorporate distance education in their standards. For ACEN accreditation, distance education nursing programs must demonstrate compliance with 10 critical elements in addition to the appropriate accreditation standards for the level of program:
• Congruence with the institutional mission
• Instructional design and course delivery methods
• Competence and preparation of faculty
• Quality and accessibility of support services for students
• Quality and accessibility of support services for faculty members
• Current, relevant, and accessible learning resources
• Current, appropriate offerings relative to the delivery method
• Provision of opportunities for regular faculty–student and student–student interactions
• Ongoing evaluation of student learning
• Processes established for verifying student identity in courses (ACEN, 2019a)
The CCNE accreditation process incorporates specific elements related to effective distance education in three of the four accreditation standards, as the following examples illustrate:
• Standard I, Program Quality: Mission and Governance: Roles of the faculty and students in program governance, including participants in distance education, are clearly defined and promote participation.
• Standard II, Program Quality: Institutional Commitment and Resources: Academic support services, such as library, technology, distance education support, research support, admission, and advising services, support achievement of program outcomes, and there is a defined process for review.
• Standard III, Program Quality: Curriculum and Teaching–Learning Practices: Teaching–learning practices in all environments, including distance education, support student achievement of expected learning outcomes; clinical 355practice experiences are provided for all students, including those in distance education offerings (CCNE, 2018).
The CNEA Standards of Accreditation are applicable to all types of nursing programs, including distance education programs (CNEA, 2016). In some of the standards, specific statements are included to guide faculty and administrators in how the standards apply to online programs. For example, one statement specifies that governance structures in the school should facilitate including distance education students.
In addition to applying the accreditation standards, the Council of Regional Accrediting Commissions (C-RAC) developed guidelines that could be used for evaluating online nursing programs: Interregional Guidelines for the Evaluation of Distance Education (C-RAC, 2011). There are nine guidelines, each with specific areas to address. These are summarized in Exhibit 18.1.
EXHIBIT 18.1 INTERREGIONAL GUIDELINES FOR EVALUATION OF DISTANCE EDUCATION
1. Online programs should be appropriate for the mission and goals of the institution.
2. There are plans for developing, sustaining, and, if appropriate, expanding online offerings, which are part of the regular evaluation processes.
3. Online learning is incorporated into the governance and academic oversight of the institution.
4. Curricula are “coherent, cohesive, and comparable in academic rigor” to traditional programs offered by the institution.
5. There is an evaluation of online learning offerings with its results used for improvement.
6. The faculty is qualified and supported.
7. The institution provides effective student and academic support services.
8. There are sufficient resources to support and expand (if indicated) online course offerings.
9. The institution assures the integrity of online offerings (C-RAC, 2011).
There also are standards from the Distance Education Accrediting Commission (DEAC). Each of these standards, listed in Exhibit 18.2, includes a description of quality practices and expectations. These areas are applicable to assessing online programs in nursing.
356EXHIBIT 18.2 DEAC ACCREDITATION STANDARDS
Standards for evaluating distance education programs relate to:
Institutional mission, effectiveness, and strategic planning
Program outcomes, curriculum, and supplemental materials
Educational and student support services
Assessment plan to track student achievement and satisfaction
Academic leadership and faculty qualificationsAdvertising and promotion of the institution and programs, and recruitment personnel
Admission criteria and practices, and enrollment agreements
Financial disclosures, cancellations, and refunds
Institutional governance
Financial responsibilities
Facilities and supplies (including record keeping; DEAC, 2019)
DEAC, Distance Education Accrediting Commission.
Program Evaluation Models
Saewert (2017) described eight models for program evaluation: objective-based, goal-free, expert-oriented, naturalistic, participative oriented, improvement-focused, success case, and theory-driven. Each of these models has positive and negative considerations for its use. Nurse educators should seek a model that will help organize the program evaluation and produce the most useful information for various stakeholders. Some nursing education programs use an eclectic approach in which they design their own model by selecting features from more than one (Saewert, 2017).
Another type of model is decision oriented. With these models, the goal of the evaluation is to provide information to decision makers for program improvement purposes. However, the existence of assessment data is no guarantee that the program will use the data to improve (Stufflebeam, 2013). Decision models focus more on using assessment data as a tool to improve programs than on accountability. Decision-oriented models usually focus on internal standards of quality, value, and efficacy. An example of a decision-oriented model is Stufflebeam’s Context, Input, Process, Product (CIPP) Model (Stufflebeam, 2013).
Other models are systems oriented. These examine inputs into the program such as characteristics of students, teachers, administrators, and other participants in the program, as well as program resources. These models also assess the operations and processes of the program as well as the context or environment within which 357the program is implemented. Finally, systems models examine the outcomes of the program: Are the intended outcomes being achieved? Are students, graduates, their employers, faculty, staff, and others satisfied with the program and how it is implemented? Is the program of high quality and cost-effective?
Regardless of the specific model used, the process of program evaluation assists various audiences or stakeholders of an educational program in judging and improving its worth or value. Audiences or stakeholders are those individuals and groups who are affected directly or indirectly by the decisions made. Key stakeholders of nursing education programs include students, faculty and staff members, partners (healthcare and community agencies), employers of graduates, and consumers. The purpose of the program evaluation determines which audiences should be involved in generating questions or concerns to be answered or addressed. When the focus is formative, that is, to improve the program during its implementation, the primary audiences are students, teachers, and administrators. Summative evaluation leads to decisions about whether a program should be continued, revised, funded, or terminated. Audiences for summative evaluation include program participants, graduates, their employers, prospective students, healthcare and community agencies, consumers, legislative bodies, funding agencies, and others who might be affected by changes in the program.
When planning program evaluation, an important decision is whether to use external or internal evaluators. External evaluators are thought to provide objectivity, but they may not know the program or its context well enough to be effective. External evaluators also add expense to the program assessment. In contrast, an internal evaluator has a better understanding of the operations and environment of the program and can provide continuous feedback to the individuals and groups responsible for the evaluation. However, an internal evaluator may be biased, reducing the credibility of the evaluation.
Curriculum Evaluation
Curriculum evaluation is not the same as program evaluation. When evaluating the curriculum, the focus is on elements central to the course of studies taken by students. As such, curriculum evaluation is narrower than program evaluation, which includes all the elements needed to offer a quality nursing program such as institutional support, qualified faculty, and adequate resources, among other areas. In evaluating the curriculum, the main areas to assess are:
• Philosophy: Is the philosophy current, and are the beliefs and values in the philosophy guiding teaching and decisions in the school?
• Conceptual framework: Is there a framework for the curriculum, and is it reflected in the program and student learning outcomes, courses, learning activities, and other aspects of the curriculum?
• 358Program outcomes: Do the program outcomes indicate important qualities and characteristics of the graduates on completion of the program, and are they student centered? Are they appropriate to the level of the nursing education program and relevant to the healthcare context? If the program has level objectives, are these congruent with the program outcomes, do they reflect student achievement at designated points in time (levels), and are they different from course outcomes?
• Curriculum design: How well do the curriculum components fit together? Do nursing courses build on foundational and prerequisite courses? Are the courses logically sequenced, and do they build on one another?
• Courses: Are course outcomes or objectives congruent with program and level outcomes? Are students achieving the outcomes of the course and developing essential knowledge, values, and skills for practice and progression through the curriculum? What is the learning environment in the course? Is the course content organized logically, and are exemplars used in the course relevant?
• Teaching–learning strategies: Do teaching strategies promote students’ learning and achievement of course outcomes? Do they promote active learning? Do they respect student diversity? Are students satisfied with the teaching methods, learning activities, assignments, and quality of teaching in the course?
• Assessment methods: Are the assessment methods appropriate for the outcomes to be evaluated and level of students, and are they reasonable? Do they provide for demonstration of relevant types of learning? Do they accommodate students’ diverse learning styles? Do they provide for feedback to improve students’ learning and performance (formative) and periodic summative evaluation?
• Resources and partnerships: Are the resources (physical space, classrooms, technology, simulation, laboratories, clinical sites, etc.) sufficient to implement the curriculum? Are there adequate numbers of qualified faculty members to offer the curriculum and teach the courses? Are there well-prepared clinical instructors, preceptors, and other educators for clinical teaching? Are the number of staff members and their roles and functions appropriate for the curriculum? Are library holdings appropriate and sufficient, and are they available to students in online courses? Are there adequate and readily available support services for students including those in online courses? (Iwasiw, Andrusyszyn, & Goldenberg, 2020; Valiga, 2017).
Although curriculum evaluation is important, an educational program involves more than a curriculum. The success of students in meeting the outcomes of courses 359and the curriculum as a whole may depend as much on the quality of the students admitted to the program or the characteristics of its faculty as it does on the sequence of courses or the instructional strategies used. Similarly, there may be abundant evidence that graduates meet the outcomes of the curriculum, but graduates may not be satisfied with the program or may be unable to pass licensure or certification examinations. As faculty review assessment data, they need to consider all elements of the program before making decisions about program changes.
Evaluation of Courses
Course evaluation is similar to curriculum evaluation because courses are elements of the curriculum. All aspects of course design and implementation should be included in the evaluation. Course evaluation may determine the extent to which:
• Course outcomes are appropriate and relate to the program outcomes.
• Courses reflect the background of students and whether they have prerequisite knowledge and skills.
• Students achieve learning outcomes in the course.
• Content is up-to-date and evidence based.
• The course is logically organized.
• Teaching methods and learning activities are relevant for the course outcomes (or objectives).
• Teaching methods and learning activities facilitate learning and engage students actively in the course.
• Assessment methods are related to the course outcomes.
• Assignments are appropriate and promote learning.
• Grading criteria are clear and adhered to.
• Students are satisfied with the course.
• Faculty are satisfied with the course.
Students typically are asked to evaluate their courses at the end of an academic term, but course evaluations need to extend beyond student ratings. Faculty members have the necessary in-depth knowledge of the curriculum to assess the value of each course in relation to intended curriculum outcomes.
Evaluation of Teaching–Learning Activities
When evaluating the teaching–learning activities used in a course, the faculty should give primary considerations as to whether they facilitated student learning and the 360extent to which they engaged students actively. Teaching–learning strategies should be evidence based and reflect best practices (Oermann & Conklin, 2018; Valiga, 2017). Although student satisfaction with teaching–learning activities is important, it is insufficient. Faculty members should evaluate the effectiveness of the teaching methods and each assignment in a course to ensure it is the best approach for the outcomes to be achieved. Teaching methods should promote active involvement of students and interactions with others, one-to-one and in small groups. Students should be satisfied with the teaching methods and learning activities in a course, but students are not experts in education, and faculty members need to choose the most effective strategies for promoting student learning. Some of those strategies, such as problem-based learning, require more time for students and active involvement, which may affect students’ satisfaction and evaluation of the course. Student satisfaction, although important, is not the only factor to consider.
Teaching
Another area of evaluation involves appraising the effectiveness of the teacher. This addresses the quality of teaching in the classroom, online, in simulation, in skills laboratories, and in clinical settings. Evaluation of teaching also includes other dimensions of the teacher’s role, depending on the goals and mission of the nursing education program. These other roles may involve scholarship and research; service to the nursing program, college or university, community, and nursing profession; and clinical practice. It is beyond the scope of this book to examine these multiple dimensions, but a brief discussion is provided about assessing the quality of teaching in the classroom and the clinical setting.
The National League for Nursing (NLN) Nurse Educator Core Competency describes the knowledge, skills, and attitudes required for effectiveness in the role of nurse educator as identified in research-based literature (NLN, 2019). In the NLN’s The Scope of Practice for Academic Nurse Educators (2012), each competency is followed by tasks that further describe the competency. These competencies address the ability of the nurse educator to guide students’ learning, professional development, and socialization into the nursing role, at whatever level they are preparing for; use assessment and evaluation methods; develop, evaluate, and revise nursing programs and courses; engage in scholarship in the educator role; be a leader and change agent; and contribute to the school of nursing or other educational setting.
The research in nursing education suggests five qualities of effective teaching in nursing: (a) knowledge, (b) clinical competence, (c) teaching skill, (d) interpersonal relationships with learners, and (e) personal characteristics. These findings are consistent with studies about teacher effectiveness in other fields (Oermann, Shellenbarger, & Gaberson, 2018).
361Knowledge
An effective teacher is an expert in the content area and is knowledgeable about the area in which teaching. Nurse educators need to keep current with nursing practice, new developments in their areas of expertise, and research. However, knowledge alone is not sufficient; the teacher must be able to communicate that knowledge to students, assist students in applying knowledge to patient care, and help them learn.
Clinical Competence
If teaching in the clinical setting, the teacher has to be competent in clinical practice (Oermann et al., 2018). The clinical competence of the teacher is one of the most important characteristics of effective clinical teaching in nursing (Lovric, Prlic, Zec, Puseljic, & Zvanut, 2015). The best teachers are expert clinicians who know how to care for patients, can make sound clinical judgments, have expert clinical skills, and can guide students in developing those skills.
In some nursing education programs, especially graduate programs for advanced practice nursing, faculty members are required to maintain certification in clinical practice. In programs without such a requirement, the clinical teacher should maintain clinical competence by regular clinical practice, attending continuing nursing education programs, reading current clinical literature, or other means.
Teaching
Although necessary, knowing what to teach is not sufficient. The teacher also needs to know how to teach. Competencies in teaching involve the ability to:
• Identify students’ learning needs.
• Plan instruction.
• Present content effectively.
• Explain concepts clearly with exemplars that help students apply those concepts in patient care.
• Demonstrate procedures effectively.
• Be skilled in teaching online and promoting interaction in an online environment (if relevant).
• Use sound assessment practices.
The research suggests that the teacher’s skills in evaluation are particularly important to teaching effectiveness. Evaluating learners fairly, having clear expectations and communicating those to students, correcting mistakes without embarrassing students, and giving immediate feedback are important teacher behaviors (Oermann et al., 2018).
362Interpersonal Relations With Learners
Another important characteristic is the ability of the teacher to establish positive relationships with students as a group in the classroom, online environment, and clinical setting, and with students on an individual basis. Effective teaching qualities in this area include showing respect for and confidence in students, being honest and direct, supporting students, and being approachable (Oermann et al., 2018). Effective teachers treat students fairly and create an environment of mutual respect between educator and student.
Personal Characteristics of Teacher
Effective teaching also depends on the personal characteristics of the teacher. Characteristics in this area include enthusiasm, patience, having a sense of humor, friendliness, integrity, perseverance, courage, and willingness to admit mistakes (Oermann et al., 2018). Personal characteristics that convey caring about students support caring as a core value in nursing. Caring about students, recognizing they are learners not yet nurses, and supporting them as they learn and develop their skills are particularly important in clinical teaching, which is stressful for students.
How to Evaluate Teaching Effectiveness
Teaching effectiveness data are available from a variety of sources. These include students, peers, administrators, and others involved in the educational experience such as preceptors.
Student Ratings
Student evaluations are a necessary but insufficient source of information (Oermann, 2017). Because students are the only participants other than the teacher who are consistently present during the teaching–learning process, they have a unique perspective of the teacher’s effectiveness as an educator over time. Students can make valid and reliable interpretations about the teacher’s use of teaching methods, fairness, interest in students, and enthusiasm for the subject.
There are limitations, though, to the use of student ratings. These ratings can be affected by class size, with smaller classes tending to rate teacher effectiveness higher than larger classes (Oermann, Conklin, Rushton, & Bush, 2018). Student ratings can also be influenced by the type of course format; for example, discussion courses tend to receive higher ratings than do lecture courses. Students have a tendency to rate required and elective courses in their own field of study higher than courses they are required to take outside their majors. Lastly, it is questionable whether students can evaluate the accuracy, depth, and scope of the teacher’s knowledge because they 363do not have expertise in the content to make this judgment. Characteristics such as these are best evaluated by peers.
Many colleges and universities have a standard form for student evaluation of teaching that is used in all courses across the institution. These forms generally ask students to rate the teacher’s performance in areas of (a) presentation and teaching skills, (b) interactions with students as a group and individually, (c) breadth of coverage of content, and (d) assessment and grading practices. Students also may be asked to provide a rating of the overall quality of the faculty member’s teaching in the course, the extent of their own learning in the course, and the workload and difficulty of the course. Table 18.2 lists typical areas that are assessed by students on these forms.
TABLE 18.2 Typical Areas Assessed on Student Evaluation of Teaching Forms
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
