7
ASSESSMENT OF HIGHER LEVEL LEARNING
In preparing students to meet the needs of patients within the changing healthcare system, educators are faced with identifying essential content to teach in the nursing program. Mastery of this knowledge alone, however, is not enough. Students also need to develop cognitive skills for processing and analyzing information, comparing different approaches, weighing alternatives, and arriving at sound decisions. These cognitive skills include, among others, the ability to apply concepts to new situations, problem-solving, critical thinking, and clinical judgment. The purpose of this chapter is to present methods for assessing these higher levels of learning in nursing.
Higher Level Learning
One of the concepts presented in Chapter 1, Assessment and the Educational Process, was that outcomes can be organized in a cognitive hierarchy or taxonomy, with each level representing more complex learning than the previous one. Learning extends from simple remembering and understanding, which are lower level cognitive behaviors, to higher level thinking skills. Higher level cognitive skills include applying, analyzing, evaluating, and creating. With higher level thinking, students apply concepts and other forms of knowledge to new situations, use that knowledge to interpret patient needs and identify patient and other types of problems, and arrive at carefully thought-out judgments about actions to take.
The main principle in assessing higher level learning is to develop test items and other assessment methods that require students to apply knowledge and skills in a new situation (Brookhart & Nitko, 2019). Only then can the teacher assess whether the students are able to use what they have learned in a different context. Considering that patient characteristics, problems, and interventions often do not match the textbook descriptions, and health status can change quickly, students need to develop 120their ability to think through clinical situations and arrive at the best possible decisions. By introducing novel materials into the assessment, the teacher can determine whether students have developed these cognitive skills.
Problem-Solving
In the practice setting, students are continually faced with patient and other clinical problems to be solved. Some of these problems relate to managing patient conditions and deciding what actions to take, whereas others involve problems associated with the nurse’s role, collaboration with other providers, and the work environment. The ability to solve patient and setting-related problems is an essential skill to be developed by students. Problem-solving begins with recognizing and defining the problem, gathering data to clarify it further, thinking through possible approaches to use, and evaluating their outcomes. Students faced with patient problems for which they lack understanding and a relevant knowledge base will be impeded in their thinking. This is an important point in both teaching and assessment. When students have an understanding of the problem and possible solutions, they can more easily apply this knowledge to new situations they encounter in the clinical setting.
Past experience with similar problems, either real problems in the clinical setting or hypothetical scenarios in simulation and examples used in teaching, also influences students’ skill in problem-solving. Experience with similar problems gives the student a perspective on what to expect in the clinical situation—typical problems the patient may experience and approaches that are usually effective for those problems.
Well-Structured and Ill-Structured Problems
Brookhart and Nitko (2019) defined two types of problems that students may be asked to solve: well structured and ill structured. Well-structured problems provide the information needed for problem-solving; typically, they have one correct solution rather than multiple ones to consider and in general are “clearly laid out” (p. 226). These are problems and solutions that the teacher may have presented in class or online and then asked students about in an assessment. Well-structured problems provide practice in applying concepts learned in class to scenarios and other exemplars but do not require extensive thinking skills.
In contrast, ill-structured problems reflect real-life problems and clinical situations that students encounter in practice. With these situations, the problem may not be clear to the learner, the data may suggest a variety of problems, or there may be an incomplete data set to determine the problem. Along similar lines, the student may identify the problem but be unsure of approaches to take and how to interpret the clinical situation. Some assessment methods may address well-structured problems, assessing understanding of typical problems and approaches. Other methods assess 121students’ ability to analyze situations to interpret patient needs and concerns, identify possible problems given the data, identify additional data needed, compare and reason through multiple potential approaches, and arrive at an informed judgment as to actions to take (or not) in the situation.
Critical Thinking
There has been extensive literature in nursing for many decades about the importance of students developing the ability to think critically. The complexity of patient needs, the extensive amount of information the nurse has to process and analyze in the practice setting, the types of clinical judgments required for providing quality and safe care, and multiple ethical issues faced by the nurse require the ability to think critically. Many definitions of critical thinking exist. One way to view critical thinking is that it is the ability to use higher level cognitive skills such as analysis to think through situations and arrive at appropriate judgments and actions; it is being deliberate about thinking (Alfaro-LeFevre, 2017; Papp et al., 2014). To engage in critical thinking, students need a knowledge base to understand the situation, analyze clinical information, make informed decisions, and reflect on the thinking process (Von Colln-Appling & Giuliano, 2017). Students cannot engage in critical thinking if they have not learned about the patient problem or concept earlier.
In the clinical setting, critical thinking enables the student to arrive at sound judgments about patient care. Carrying out assessment; planning care; intervening with patients, families, and communities; and evaluating the effectiveness of interventions—all of these require critical thinking. In the assessment process, important cognitive skills include differentiating relevant from irrelevant data, identifying cues in the data, identifying additional data to collect prior to deciding on the problem, and specifying patient needs and problems based on these data.
Critical thinking also is reflected in the ability to compare possible interventions, considering the evidence, to decide on the best approaches to use in a particular situation (Alfaro-LeFevre, 2017; Facione, 2015). Judgments about the quality and effectiveness of care are influenced by the learner’s thinking skills. Nurses, even as they develop their expertise, need to reflect on their clinical reasoning and continue to build their critical thinking skills. Students who demonstrate critical thinking ability:
• Ask questions, are inquisitive, and are willing to search for answers.
• Consider alternate ways of viewing information.
• Offer different perspectives to problems, solutions, and clinical situations.
• Question current practices and express their own ideas about care.
• 122Extend their thinking beyond the readings, course and clinical activities, and other requirements.
• Are open-minded.
These characteristics are important because they suggest behaviors that are to be developed by students as they progress through the nursing program. They also provide a framework for faculty members to use when assessing whether students have developed their critical thinking abilities.
Clinical Judgment
Tanner (2006) viewed clinical judgment as an interpretation of the patient’s needs and problems, and decisions on actions to take, or not take, based on the patient’s responses. The clinical judgment process includes four aspects: (a) noticing, (b) interpreting, (c) responding, and (d) reflecting. Tanner’s model provides a framework for assessing students’ thinking in a clinical situation or scenario. Students can be asked to describe what they would expect to find in the situation, what they noticed first, and other data they need (noticing). To assess students’ ability to interpret a situation, the teacher can ask them to explain specific data and what they mean, and their priorities of care (interpreting). Another series of questions can explore interventions for the patient, why students would select those interventions or not take any actions, and their rationale (responding). The teacher can ask students to reflect on their experiences with patients and discuss what they would do differently next time (reflecting). In addition to written reflections after clinical experiences, and sharing the thought process used to arrive at judgments, simulations are an effective strategy for helping students develop their clinical judgment skills (Bussard, 2018; Victor, 2017).
Context-Dependent Item Sets
In assessing students’ cognitive skills, test items and other methods need to meet two criteria. They should (a) introduce new information not encountered by students at an earlier point in the instruction and (b) provide data on the thought process used by students to arrive at an answer, rather than the answer alone. Context-dependent item sets may be used for this purpose.
Writing Context-Dependent Item Sets
A basic principle of assessing higher level skills is that the test item or other assessment method has to introduce new or novel material for analysis. Without the introduction of new material as part of the assessment, students may rely on memorization from prior discussion or their readings about how to reason through a clinical situation and decide on actions to take for the situation at hand; they may 123simply recall the typical problems and approaches without thinking through other possibilities themselves. In nursing education, this principle is often implemented through clinical scenarios that present a novel situation for students to analyze. Test items that include scenarios in the stem, which students analyze and then answer questions about, are used commonly in nursing education and on licensure and certification examinations. Brookhart and Nitko (2019) referred to these items as context-dependent item sets or interpretive exercises.
In a context-dependent item set, the teacher presents introductory material that students then analyze and answer questions about. The introductory material may be a description of a clinical situation, patient data, research findings, issues associated with clinical practice, and varied types of scenarios. The introductory material also may include diagrams, photographs, tables, figures, and excerpts from reading materials. Students read, analyze, and interpret the introductory material and then answer questions about it or complete other tasks. One advantage of a context-dependent item set is that it offers an opportunity to present new information for students to analyze that is related to clinical practice. In addition, the introductory material provides the same context for analysis for all students.
The questions asked about the introductory material may be selected- or constructed-response items. With selected-response items such as multiple choice, however, the teacher is not able to assess the underlying thought process used by students in arriving at the answer; their responses reflect instead the outcomes of their thinking. If the goal is to also assess the process students used to think through the situation and decide on an approach, then open-ended items such as short answer and essay would be better items to use.
Interpretive Items on the NCLEX®
On the NCLEX (National Council Licensure Exam), items may include multimedia, such as tables, charts, graphics, and audio, for candidates to interpret and respond to questions. Any of the types of item formats may be used with these, including the standard multiple-choice format and alternate formats (National Council of State Boards of Nursing, 2019). Alternate formats include multiple-response, fill-in-the-blank calculation, ordered-response, hot-spot, and exhibit items. Multiple-response items were presented in Chapter 5, Multiple-Choice and Multiple-Response, and fill-in-the-blank and ordered-response items were discussed in Chapter 6, Short-Answer (Fill-in-the-Blank) and Essay. In a hot-spot item, candidates are asked a question about an image; they answer the question by clicking on the image with the mouse. For example, the candidate might be presented with an image of the chest and asked where to place the stethoscope to listen to heart sounds in the mitral area. In exhibit items, candidates are given a problem, and to answer that problem, they need to read and interpret information in an exhibit. Examples of hot-spot and exhibit items are included later in Exhibit 7.2.
124Students should have experience answering these types of questions and other forms of context-dependent items as they progress through a nursing program. Items can be incorporated into quizzes and tests; can be developed for small-group analysis and discussion in class, as out-of-class assignments, and as online activities; and can be analyzed and discussed by students in postclinical conferences. Context-dependent items are also used on nursing certification examinations, and students in graduate programs need to have experience thinking through and answering these types of questions.
Layout
The layout of the context-dependent item set, that is, the way it is arranged on the page, is important so that it is clear to the students which questions relate to the introductory material. Exhibit 7.1 illustrates one way of arranging the material and related items on a page. A heading should be used to indicate the items that pertain to the introductory material, for example, “Questions 1 to 3 refer to the following scenario.” Brookhart and Nitko (2019) suggested that the material for interpretation be centered between the left and right margins of the page so it is readily apparent to the students. If possible, the context and all items pertaining to it should be placed on the same page.
Strategies for Writing Context-Dependent Items
Suggestions follow for writing context-dependent item sets. If the intent is to assess students’ critical thinking or clinical judgment, the introductory material needs to provide sufficient information for analysis without directing the students’ thinking in a particular direction. The first step is to draft the types of questions to be asked about the situation, then to develop a scenario to provide essential information for analysis. If the scenario is designed on the basis of clinical practice, students may be asked to analyze data, interpret the scenario, identify patient problems, decide on nursing interventions, evaluate outcomes of care, and examine ethical issues, among other tasks. Cases, discussed later in this chapter, use a short clinical scenario followed by one or more questions using any type of item.
EXHIBIT 7.1 LAYOUT OF CONTEXT-DEPENDENT ITEM SETS
Questions 1–3 relate to the following scenario:
Scenario (and other types of introductory material) here
1. Item one here
2. Item two here
3. Item three here
125The introductory material should be geared to the students’ level of understanding and experience. The teacher should check the terminology used, particularly with beginning students. The situation should be of reasonable length without extending the students’ reading time unnecessarily.
The questions should focus on the underlying thought process used to arrive at an answer, not on the answer alone. For that reason, short-answer or essay items are frequently used. In some situations, however, the goal may be to assess students’ ability to apply concepts or protocols learned in class without any original thinking about them. In this case, scenarios that are clearly laid out for students and questions, such as multiple-choice with one correct answer, are appropriate. Context-dependent items may be incorporated within a test, completed individually or in small groups for formative evaluation, discussed in class or an online environment for instructional purposes, completed during postclinical conferences, or done as out-of-class assignments, either graded or ungraded.
Item sets focusing on assessment of problem-solving ability may ask students to complete the following tasks:
• Identify the problem and alternate problems possible.
• Develop questions for clarifying the problem further.
• Identify assumptions made about the problem and possible approaches or solutions.
• Identify additional data needed for interpreting the situation.
• Differentiate relevant and irrelevant data.
• Propose interventions and summarize supporting evidence.
• Relate knowledge from different sources to the situation to better understand it.
• Evaluate the effectiveness of approaches to solving problems and the outcomes achieved.
The following item set assesses students’ skill in identifying and thinking through problems. After reading the introductory situation about the patient, students are asked to identify all possible problems and provide data to support them. Other questions ask students about additional data to be collected, again with a rationale for their answer.
126Your 8-year-old patient had a closed head injury 4 weeks ago after falling off his bike. You visit him at home and find that he has weakness of his left leg. His mother reports that he is “getting his rest” and “sleeping a lot.” The patient seems irritable during your visit. When you ask him how he is feeling, he tells you, “My head hurts where I hit it.” The mother appears anxious, talking rapidly and changing position frequently.
1. List all possible problems in this situation. For each problem, describe supporting assessment data.
2. What additional data are needed, if any, to decide on these problems? Provide a rationale for collecting this information.
3. What other data would you collect at this time? Why is this information important to your thinking about what to do?
Context-dependent items may focus on actions to be taken in a situation. For this purpose, the teacher can briefly describe a critical event, then ask learners what they would do next. Because the rationale underlying the thinking is as important if not more important than the decision or outcome, students can include an explanation of the thought process they used. For example:
You are caring for a patient with diabetes whose recent urinalysis revealed 4+ ketones and trace leukocytes, and was negative for nitrites and red blood cells.
1. As the nurse practitioner, what would you do next? Why did you choose this action?
2. Which of the laboratory results in the scenario are most important in your decision? Provide a rationale for your answer.
On a test with selected-response items, the stem can introduce the critical event or clinical situation, followed by a multiple-choice, multiple-response, or other alternate item format. An example of this type of item using the prior scenario is:
You are caring for a patient with diabetes whose recent urinalysis revealed 4+ ketones and trace leukocytes, and was negative for nitrites and red blood cells. Which of the following actions should the nurse practitioner take next?
a. Order an ultrasound of the kidneys to rule out subacute renal failure
b. Check the patient’s blood glucose
c. Order a 24-hour urine test for microalbumin
d. Check for a history of illicit drug and alcohol use
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
