6
SHORT-ANSWER (FILL-IN-THE-BLANK) AND ESSAY
Short-answer and essay questions are examples of constructed-response items. With these items, the test-taker supplies an answer rather than selecting from options already provided. Because students supply the answers, this type of item reduces the chance of guessing.
Short-answer items can be answered with a word, phrase, or number. There are two types of short-answer items: question and completion. One format presents a question that students answer in a few words or phrases. With the other format, completion or fill in the blank, students are given an incomplete sentence that they complete by inserting a word or words in the blank space. In an essay item, the student develops a more extended response to a question or statement. Essay tests and written assignments use writing as the means of expressing ideas, although with essay items the focus of assessment is the content of the answer rather than the writing ability. Short-answer and essay items are described in this chapter. A quick reference guide to writing these items is provided in Appendix A.
Short Answer
Short-answer items can be answered by a word, phrase, or number. The two types of short-answer items—question and completion—also referred to as fill-in-the-blank, are essentially the same except for format.
With the question format, students answer a question in a few words or phrases. Calculations may be included for the teacher to review the process that the student used to arrive at an answer. The questions may stand alone and have no relationship to one another, or comprise a series of questions in a similar content area.
Completion items consist of a statement with a key word or words missing; students fill in the blank to complete it. Other types of completion items ask students to perform a calculation and record the answer, or to order a list of responses. Completion items are appropriate for recall of facts and specific information and for calculations. To complete the statement, the student recalls missing facts, such as a word or short phrase, or records the solution to a calculation problem. Although 102completion items appear easy to construct, they should be designed in such a way that only one answer is possible. If students provide other correct answers, the teacher needs to accept them.
Fill-in-the-blank calculation and ordered-response items are two of the alternate item formats used on the NCLEX® (National Council Licensure Examination). Fill-in-the-blank items ask candidates to perform a calculation and type in the answer. All answers are scored as right or wrong. With ordered-response items, candidates answer a question by rank ordering options or placing a list of responses in the proper order (National Council of State Boards of Nursing, 2019). For example, students might be given a list of characteristics of pressure ulcers or the steps of a procedure and asked to put them in the order in which they occur. On a computerized test, such as the NCLEX, candidates can click an option (unordered side of the screen) and drag and drop it in the correct order (on the ordered response side of the screen), or highlight an option and use the arrow keys to arrange the options in the correct order. However, this same format can be used on a paper-and-pencil test with students writing the order on the answer sheet.
Short-answer items are useful for measuring student ability to interpret data, use formulas correctly, complete calculations, and solve mathematical-type problems. Brookhart and Nitko (2019) described another type of short-answer format, association variety, which provides a list of terms, diagram, or photograph for which students recall relevant labels, numbers, or symbols. For example, students might be given a list of medical terms and asked to list their abbreviations.
Writing Short-Answer Items
Suggestions for developing short-answer items are as follows:
1. Questions and statements should not be taken verbatim from textbooks, other readings, and handouts students received. These materials may be used as a basis for designing short-answer items, but taking exact wording from them may result in testing only recall of meaningless facts out of context. Such items measure memorization of content and may or may not be accompanied by the student’s comprehension of it.
2. Phrase the item so that a unique word, series of words, or number must be supplied to complete it. Only one correct answer should be possible to complete the statement.
3. Write questions that are specific and can be answered in a few words, phrases, or short sentences. The question, “What is insulin?” does not provide sufficient direction as to how to respond; asking instead, “What is the peak action time of NPH insulin?” results in a more specific answer.
4. Before writing the item, think of the correct answer first and then write a question or statement for that answer. Although the goal is to develop an item with only one correct response, students may identify other correct answers. 103For this reason, it is necessary to develop a scoring sheet with all possible correct answers, and rescore student responses as needed if students provide additional correct answers that the teacher did not anticipate.
5. Fill-in-the-blank items requiring calculations and solving mathematical-type problems should include in the statement the type of answer and degree of specificity desired. For example, the question may ask the test-taker to convert pounds to kilograms, rounding the answer to one decimal point.
6. For a statement with a key word or words missing, place the blank at or near the end of the statement. This makes it easier for students to complete. It is also important to watch for grammatical clues in the statement, such as “a” versus “an” and singular versus plural, prior to the blank, which might give clues to the intended response. If more than one blank is included in the statement, they should be of equal lengths. Use of more than two blanks should be avoided because there may be insufficient information remaining to permit students to grasp the nature of the task they are to perform.
7. When students need to write longer answers, provide for sufficient space. In some situations, longer responses might indicate that the item is actually an essay item, and the teacher then should follow principles for constructing and scoring essay items.
8. Even though a blank space is placed at the end of the statement, the teacher may direct the student to record one-word answers in blanks to the left the items, thereby facilitating scoring. For example,
____________ 1. Streptococcus pneumoniae and Staphylococcus aureus are examples of _______________ bacteria.
Following are some examples of question and completion (or fill-in-the-blank) formats of short-answer items:
What congenital cardiac defect results in communication between the pulmonary artery and the aorta? ________________
Two types of metered-dose inhalers used for the treatment of bronchial asthma are:
________________
Six Sigma is a quality-improvement model used in healthcare. List two other models.
1. ________________
2. ________________
104You are caring for a patient who weighs 128 lb. She is ordered 20 mcg/kg of an intravenous (IV) medication. What is the correct dose in micrograms?
Answer: ________________
Essay Item
In an essay test, students construct responses to items based on their understanding of the content. With this type of test item, varied answers may be possible depending on the concepts selected by the student for discussion and the way in which they are presented. Essay items provide an opportunity for students to select content to discuss, present ideas in their own words, and develop an original and creative response to an item. Essay items are useful for assessing complex learning outcomes and higher levels of learning. Higher level responses, however, are more difficult to evaluate and score than answers reflecting recall of facts.
Although some essay items are developed to assess recall of facts and specific information, they are more appropriate for higher levels of learning. Miller, Linn, and Gronlund (2013) recommended that essay items should be used primarily for learning outcomes that cannot be measured adequately through selected-response items. Essay items are effective for evaluating students’ ability to apply concepts, analyze situations and ideas, and develop creative solutions to problems using multiple sources of information. Higher level thinking skills and more complex learning outcomes such as analysis, synthesis, and evaluation are better assessed with essays, because students need to select the information they want to include and formulate their own responses rather than choosing one from a list already provided (Gierl, Latifi, Lai, Boulais, & De Champlain, 2014; Rios & Wang, 2018).
Although essay items use writing as the medium for expression, the intent is to assess student understanding of specific content rather than to judge writing ability in and of itself. Written assignments are better suited to evaluating the ability of students to write effectively; these are described in Chapter 9. Low-level essay items are similar to short-answer items and require precise responses. An example of a low-level essay item is: “Describe three signs of increased intracranial pressure in children younger than 2 years old.” Broader and higher level essay items, however, do not limit responses in this way and differ clearly from short-answer items, such as “Defend the statement ‘parents should be required to follow childhood vaccine recommendations.’” Essay items may be written to assess a wide range of outcomes. These include:
• Comparing, such as comparing the side effects of different medications
• Outlining steps to take and protocols to follow
• Explaining in one’s own words a situation or statement
• 105Discussing topics
• Applying concepts and evidence to a clinical scenario and explaining their impact on potential decisions
• Analyzing patient data and clinical situations
• Critiquing different interventions based on current evidence
• Developing plans and proposals drawing on multiple sources of information
• Analyzing nursing and healthcare trends
• Arriving at decisions about issues and actions to take, with a rationale added
• Analyzing ethical issues, possible decisions, and their consequences
• Developing arguments for and against a particular position or decision
As with other types of test items, the outcome to be assessed provides the framework for developing the essay item. From the outcome, the teacher develops a clear and specific item to elicit information about student achievement. If the outcome focuses on application of concepts to clinical practice, then the essay item should examine the ability to apply knowledge to a clinical situation. The item should be stated clearly so that the students know what they should write about. If it is ambiguous, students may perceive the need to write all they know about a topic.
Issues With Essay Tests
Although essay items are valuable for examining the ability to select, organize, and present ideas and they provide an opportunity for creativity and originality in responding, they are limited by low reliability and other issues associated with their scoring. The teacher should have an understanding of these issues because they may influence the decision to use essay items. Strategies are provided later in the chapter for addressing some of these issues.
Limited Ability to Sample Content
By their nature, essay items do not provide an efficient means of sampling course content as compared with objective items. Often, only a few essay items can be included on a test, considering the time it takes for students to formulate their thoughts and prepare an open-ended response, particularly when the items are intended for assessing higher levels of learning. As a result, it is difficult to assess all of the different content areas in a nursing course using essay items.
When the outcomes are memorization and remembering facts, essay items should not be used because there are more efficient means of measuring such outcomes. Instead, good essay items ask students to use higher level thinking skills and organize and express ideas (Brookhart & Nitko, 2019). Essay items are best used for responses requiring originality.
Unreliability in Scoring
106The major limitation of essay items is the lack of consistency in evaluating responses. Scoring answers is a complex process, and studies have shown that essay responses are scored differently by different teachers and even the same teacher may not be consistent with scoring a response at a different time (Miller et al., 2013; Rios & Wang, 2018). Some teachers are more lenient or critical than others regardless of the criteria established for scoring. Even with preset criteria, teachers may evaluate answers differently, and scores may vary when the same teacher reads the paper again. Miller et al. (2013) suggested that frequently the reasons for unreliability in scoring are the failure of the faculty member to identify the specific outcomes being assessed with the essay item and lack of a well-defined rubric for scoring (p. 238).
Factors such as misspelled words and incorrect grammar may affect scoring beyond the criteria to which they relate. In scoring the student’s response, it is important to focus on the substantive content and not be influenced by how the response is written. Brookhart and Nitko (2019) recommended that writing style, spelling, and grammar should be scored separately to avoid blending this evaluation with a judgment about the student’s knowledge of the subject. These areas can be specified as a separate score on the rubric used for the evaluation and be given less weight than the substantive content of the answer.
The unreliability with scoring depends on the type of essay item. When the essay item is highly focused and structured, such as “Describe three side effects of bronchodilators,” there is greater reliability in scoring. These lower level items also could be classified as short answer. Less restrictive essay items allowing for freedom and creativity in responding have lower rater reliability than more restricted ones. Items asking students to analyze, defend, judge, evaluate, and create products are less reliable in terms of scoring the response. There are steps the teacher can take, though, to improve reliability, such as defining the content to be included in a “correct” answer and using a scoring rubric. These are presented later in the chapter.
Carryover Effects
Another issue in evaluating essay items is a carryover effect in which the teacher develops an impression of the quality of the answer from one item and carries it over to the next response. If the student answers one item well, the teacher may be influenced to score subsequent responses at a similarly high level; the same situation may occur with a poor response. For this reason, it is best to read all students’ responses to one item before evaluating the next one. Miller et al. (2013) suggested that reading all the answers to one item at a time improves scoring accuracy by keeping the teacher focused on the standards of each item. It also avoids carrying over an impression of the quality of the student’s answer to one item onto the scoring of the next response.
107The same problem can occur with written assignments. The teacher’s impression of the student can carry over from one paper to the next. When scoring essay tests and grading papers, the teacher should not know whose paper it is.
Halo Effect
There may be a tendency in evaluating essay items to be influenced by a general impression of the student or feelings about the student, either positive or negative, that create a halo effect when judging the quality of the answers. For instance, the teacher may hold favorable opinions about the student from class or clinical practice and believe that this learner has made significant improvement in the course, which in turn might influence the scoring of responses. For this reason, essay tests should be scored anonymously by asking students to identify themselves by an assigned or selected number rather than by their names. Names can be matched with numbers after scoring is completed.
Rater Drift
Essay tests read early in a scoring session may be scored higher than those read near the end because of teacher fatigue and time constraints. Brookhart and Nitko (2019) described the problem of rater drift, the tendency of the teacher to gradually stray from the scoring criteria. Over time the teacher may not pay attention to the specific criteria or may apply them differently to each response. In scoring essay items, the teacher needs to check that the rubric and standards for grading are implemented equally for each student. Teachers should read papers in random order and read each response twice before computing a score. After scoring the responses to a question, the teacher should rearrange the papers to avoid being influenced by their order. It also is important to stop periodically and confirm that the responses read later are scored consistently with those read early (Brookhart & Nitko, 2019). Another potential issue with scoring is the teacher may tend to award scores near the mean or drift between scoring too severely or leniently regardless of the quality of the response (Rios & Wang, 2018). An awareness by the teacher of potential issues with scoring essay items can lead to better practices with assessing student answers.
Time
One other issue in using essay items is the time it takes for students to answer them and for teachers to score them. In writing essay items, the teacher should estimate how long it will take to answer each item, erring on allowing too much time rather than too little. Students should be told approximately how long to spend on each item so they can pace themselves (Miller et al., 2013).
108Scoring essay items also can be a pressing issue for teachers, particularly if the teacher is responsible for large numbers of students. Considering that responses should be read twice, the teacher should consider the time required for scoring responses when planning for essay tests. Scoring software is available that can scan an essay and score the response; however, this software is not designed for assessing constructed responses in specialized fields such as nursing.
Student Choice of Items
Some teachers allow students to choose a subset of essay items to answer, often because of limited time for testing and to provide options for students. For example, the teacher may include four items on the care of patients with type 1 diabetes and ask students to answer two of them. However, Miller et al. (2013) cautioned against this practice because when students choose different items to answer, they are actually taking different tests. The option to choose items to answer also may affect measurement validity.
Restricted-Response Essay Items
There are two types of essay items: restricted response and extended response. Although the notion of freedom of response is inherent in essay items, there are varying degrees of freedom in responding to the items. At one end of the continuum is the restricted-response item, in which a few sentences are required for an answer. These are short-answer essays. At the other end is the extended-response item, in which students have freedom to express their own ideas and organize them as they choose. Responses to essay items typically fall between these two extremes.
In a restricted-response item, the teacher limits the student’s answer by indicating the content to be discussed, the extent of discussion allowed, or both. For example, a specific patient problem might be identified and students asked questions about that problem, or the directions to the item might limit the response to one paragraph or page. With this type of essay item, the way in which the student responds is structured by the teacher. In writing restricted-response items, the teacher might include specific material with the item, such as patient data, a description of a clinical scenario, a summary of evidence, a description of issues associated with clinical practice, and extracts from the literature, to cite a few. Students read, analyze, and interpret this accompanying material, then answer questions about it.
Examples of restricted-response items follow:
• Define patient-centered care. Limit your definition to one paragraph.
• Select one environmental health problem and describe its potential effects on the community. Do not use an example presented in class. Limit your discussion to one page.
• 109Compare metabolic and respiratory acidosis. Include the following in your response: definitions, precipitating factors, clinical manifestations, diagnostic tests, and interventions.
• Your patient is 76 years old and 1-day postoperative following a femoral popliteal bypass surgery. Name two complications the patient could experience at this time and discuss why they are potential problems. List two nursing interventions for this patient to prevent these complications during the early recovery period with related evidence.
• Describe three pathological changes that characterize chronic obstructive pulmonary disease.
Extended-Response Essay Items
Extended-response essay items are less restrictive and as such provide an opportunity for students to decide how to respond: They can organize ideas in their own ways, arrive at judgments about the content, and demonstrate the ability to communicate ideas effectively in writing. With these types of items, the teacher may assess students’ ability to develop their own ideas and express them creatively, integrate learning from multiple sources in responding, and evaluate the ideas of others based on predetermined criteria. Because responses are not restricted by the teacher, assessment is more difficult. This difficulty, however, is balanced by the opportunity for students to express their own ideas and for the teacher to assess higher level learning. As such, extended-response essay items provide a means of assessing more complex learning not possible with selected-response items. The teacher may decide to allow students to respond to these items outside of class. Sample items include:
• Critique arguments for and against the practice of suspending do-not-resuscitate status when patients undergo surgery. Based on your critique, state which position you believe is the strongest and provide a rationale supporting your choice.
• The fall rate on your unit has increased in the past 3 months. Develop a plan for analyzing this occurrence with a rationale to support your action plan.
• You receive a call in the allergy clinic from a mother who describes her son’s problems as “having stomach pains” and “acting out in school.” She asks you whether these problems may be due to his allergies. How would you respond to this mother? How would you manage this call? Include a rationale for your response with evidence to support your decisions.
• Describe an integrated model of palliative care and why this is important for children living with a life-threatening or terminal condition. What principles serve as a foundation for an integrated model of palliative care? List and discuss each principle and why it is important in pediatric palliative care.
110Writing Essay Items
Essay items should be reserved for outcomes that cannot be assessed effectively through multiple-choice and other selected-response formats. With essays, students can demonstrate their higher level thinking and ability to integrate varied sources of information and concepts. Suggestions for writing essay items are as follows.
1. Develop essay items that require synthesis of the content. Avoid items that students can answer by merely summarizing the readings and class or online discussions without thinking about the content and applying it to new situations. Assessing students’ recall of facts and specific information may be accomplished more easily using other formats, such as true–false and matching, rather than essay.
2. Phrase items clearly. The item should direct learners in their responses and should not be ambiguous. Exhibit 6.1 provides sample stems for essay items based on varied types of learning outcomes. Framing the item to make it as specific as possible is accomplished more easily with restricted-response items. With extended-response items, the teacher may provide directions as to the type of response intended without limiting the student’s own thinking about the answer. In the example that follows, there is minimal guidance as to how to respond; the revised version, however, directs students more clearly as to the intended response without limiting their freedom of expression and originality.
Example: Evaluate an article describing a nursing research study.
Revised version: Select an article reporting the results of a nursing research study. Critique the study, specifying the criteria you used to evaluate it. Based on your evaluation, describe whether the research provides evidence for nursing practice and rate the quality of the evidence. Include a rationale supporting your answer, including the system used for rating the evidence.
3. Prepare students for essay tests. This can be accomplished by asking students thought-provoking questions; engaging students in critical discussions about the content; and teaching students how to apply concepts to clinical situations, to compare approaches, and to arrive at judgments about patients and issues. Practice in synthesizing content from different sources, presenting ideas logically, and using creativity in responding to situations will help students prepare to respond to essay items in a testing situation. This practice may come through discussions in class, in clinical practice, and online; written assignments; and small-group activities. For students lacking experience with essay tests, the teacher may use sample items for formative purposes, providing feedback to students about the adequacy of their responses.
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
