10
ASSEMBLING, ADMINISTERING, AND SCORING TESTS
In addition to the preparation of a test blueprint and the skillful construction of test items that correspond to it, the final appearance of the test and the way in which it is administered can affect the validity of the test results. A haphazard arrangement of test items, directions that are confusing, and typographical and other errors on the test may contribute to measurement error. By following certain design rules, teachers can avoid such errors when assembling a test. Administering a test usually is the simplest phase of the testing process. There are some common problems associated with test administration, however, that also may affect the reliability of the resulting test scores and consequently the validity of inferences made about those scores. Careful planning can help the teacher avoid or minimize such difficulties. After administering a test, the teacher’s responsibility is to score it or arrange to have it scored. This chapter discusses the processes of assembling the test, administering it to students, and obtaining test scores.
Test Design Rules
Allow Enough Time
As discussed in Chapter 3, Planning for Testing, preparing a high-quality test requires time for the design phases as well as for the item-writing phase. Assembling the test is not simply a clerical or technical task; the teacher should make all decisions about the arrangement of test elements and the final appearance of the test even if someone else types or reproduces the test. The teacher must allow enough time for this phase to avoid errors that could affect the students’ test scores.
Arrange Test Items in a Logical Sequence
Various methods for arranging items on the test have been recommended, including by order of difficulty and according to the sequence in which the content was taught. However, if the test contains items using two or more formats, the teacher should 178first group items of the same format together. Because each item format requires different tasks of the student, this type of arrangement makes it easier for students to maintain the mental set required to answer each type of item, and prevents errors caused by frequent changing of tasks. Keeping items of the same format together also requires fewer sets of directions and facilitates scoring if a scannable answer sheet is not used (Miller, Linn, & Gronlund, 2013). Miller et al. recommended arranging sections of item types in the following order, from simplest to most complex:
1. True–false items
2. Matching exercises
3. Short-answer or completion items
4. Multiple-choice items
5. Context-dependent or interpretive exercises
6. Restricted-response essay items
7. Extended-response essay items (2013, p. 334)
Constructing a test with all of the item types is not recommended, even for a test with a large sample of items administered to a highly skilled group of learners. The longer the test, the more item formats can be included, but complex formats require more reading and processing time for the student, so they should be combined with only one or two other types.
Next, within each item format, items may be arranged according to the order in which the content was taught, which may assist students in recalling information more easily. Finally, combining the item format and content-sequence grouping, teachers should arrange items in order of increasing difficulty. Even well-prepared students are likely to be somewhat anxious at the beginning of a test, and encountering difficult items may increase their anxiety and interfere with their optimum performance. Beginning with easier items may build the students’ confidence and allow them to answer these items quickly and reserve more time for difficult items. By having confidence in their ability to answer the beginning items correctly, students may have less anxiety about the remainder of the test (Gronlund, 2006; Miller et al., 2013).
Some nursing faculty members prefer to use test-authoring software to make the process of item writing and test assembly more efficient. However, these applications may or may not allow the teacher to arrange the test items in the manner just described. Many test-development program vendors promote their software’s ability to randomize the arrangement of items on a test as a way to produce different versions of the same test to prevent cheating. However, there is no evidence that scrambling the order of items on a test produces a psychometrically equivalent measurement instrument. To facilitate accurate measurement of students’ knowledge and skills, all students should respond to the same items arranged in the same order. A later section of this chapter describes more effective ways of preventing cheating on tests.
179Write Directions
The teacher cannot assume that the students know the basis on which they are to select or provide answers or how and where to record their answers to test items. Depending on the level of students and their familiarity with the type of items and assessment procedures, it is not reasonable to expect that the assessment will be self-explanatory. This is especially true with students who are non-native English speakers or for those whose primary and secondary education occurred in countries where objectively scored item formats are less common.
The test should begin with a set of clear general directions. These general directions should include instructions on
• How and where to record responses
• What type of writing implement to use
• Whether or not students may write on the test booklet
• The amount of time allowed
• The number of pages and items on the exam
• The types and point values of items
• Whether students may ask questions during the test
• What to do after finishing the exam (Brookhart & Nitko, 2019; Gronlund, 2006; Miller et al., 2013)
Students may need to know some of these instructions while they are preparing for the test, such as whether their answers to items requiring them to supply the names of medications must be spelled accurately to be scored as correct.
Each section consisting of a particular item format should begin with a specific set of instructions. For multiple-choice items, the student needs to know whether to select the correct or best response. Directions for completion and essay items should state whether spelling, grammar, punctuation, and organization will be considered in scoring, and the length of the desired response. For computation items, directions should specify the degree of precision required, the unit of measure, whether to show the calculation work, and what method of computation to use if there is more than one option (Miller et al., 2013). Matching exercise directions should clearly specify the basis on which the match is to be made (Gronlund, 2006). Here is an example: “For each definition in Column A, select the proper term in Column B. Use each letter in Column B only once or not at all.”
Use a Cover Page
The general test directions may be printed on a cover page (Exhibit 10.1). A cover page also serves to keep the test items hidden from view during the distribution of the exam so that the first students to receive the test will not have more time to 180complete it than students who receive their copies later. If the directions on the cover page indicate the number of pages and items, the students can quickly check their test booklets for completeness and correct sequence of pages. The teacher can then replace defective test booklets before students begin answering items.
When a separate answer sheet is used, the cover page may be numbered to help maintain test security; students are directed to record this number in a particular place on the answer sheet. With this system, the teacher can track any missing test booklets after the testing is done. In addition, if the teacher asks students to record responses to short-answer or essay items directly on the test booklet, those answers can be scored anonymously; the score from the answer sheet then can be added to the score from the supply-type items for a total test score that is associated with each student’s name.
EXHIBIT 10.1 EXAMPLE OF A COVER PAGE WITH GENERAL DIRECTIONS
Exam Number ________
BEHAVIORAL HEALTH NURSING FINAL EXAM
Directions
1. This test comprises 12 pages. Please check your test booklet to make sure you have the correct number of pages in the proper sequence.
2. Parts I and II contain 86 multiple-choice and matching items. You may write on the test booklet but you must record your answers to these items on your answer sheet. This part of the test will be machine scored; read carefully and follow these instructions:
a. Use a #2 pencil.
b. Notice that the items on the answer sheet are numbered DOWN the page in each column.
c. Choose the ONE BEST response to each item. Items with multiple answer marks will be counted as incorrect. Fill in the circle completely; if you change your answer, erase your first answer thoroughly.
d. Print your name (last name, first name) in the blocks provided, then completely fill in the corresponding circle in each column. If you wish to have your score posted, fill in an identification number of up to nine digits (DO NOT use your Social Security number) and fill in the corresponding circle in each column.
e. Above your name, write your test booklet number.
3. Part III consists of two essay items. Directions for this section are found on page 12. Write your answers to these items on the lined paper provided. You may use pen or pencil. On each page of your answers, write your TEST BOOKLET NUMBER. DO NOT write your name on these pages.
4. If you have a question during the test, do not leave your seat—raise your hand and a proctor will come to you.
5. You have until 11:00 a.m. to complete this test.
181Avoid Crowding
Test items are difficult to read when they are crowded together on the page; non-native English speakers and students with learning disabilities may find crowding particularly trying. Techniques that allow students to read efficiently and to prevent errors in recording their responses include leaving sufficient white space within and between items and indenting certain elements. Teachers should allow enough blank space between and around items so that each item is distinct from the others. If not, the students might inadvertently read a line from a preceding or following item and think it belongs to the item they are answering. Arranging test items on the page in a two-column format also may cause students to lose their place and skip one or more items. Tightly packing words on a page may minimize the amount of paper used for testing, but facilitating maximum student performance on a test is worth a small additional expense for a few more sheets of paper (Miller et al., 2013).
Optimum spacing varies for each item format. The response options for a multiple-choice item should not be printed in tandem fashion, as the following example illustrates:
1. Which method of anesthesia involves injection of an agent into a nerve bundle that supplies the operative site? a. General; b. Local; c. Regional; d. Spinal; e. Topical
The options are much easier to read if listed in a single column below the stem (Miller et al., 2013), as in this example:
1. Which method of anesthesia involves injection of an agent into a nerve bundle that supplies the operative site?
a. General
b. Local
c. Regional
d. Spinal
e. Topical
Notice in this example that the second line of the stem is indented to the same position as the first line and that the responses are slightly indented. This spacing makes the item number and its content easier to read.
Keep Related Material Together
The stem of a multiple-choice item and all related responses should appear on the same page. Both columns of a matching exercise should also be printed side by side and on one page, including the related directions; using short lists of premises and responses makes this arrangement easier. With context-dependent and interpretive 182exercises, the introductory material and all related items should be contained on the same page, if possible. This facilitates reading the material and related questions (Gronlund, 2006; Miller et al., 2013).
Facilitate Scoring
If the test will be scored by hand, the layout of the test or the answer sheet should facilitate easy scoring. A separate answer sheet can be constructed to permit rapid scoring by comparing student responses to an answer key. If the students record their answers directly on the test booklet, the test items should be arranged with scoring in mind. For example, a series of true–false items should be organized with columns of Ts and Fs, preferably at the left margin (Gronlund, 2006; Miller et al., 2013) so that students need to only circle their responses, as in the following example:
| T | F | 1. A stethoscope is required to perform auscultation. |
| T | F | 2. Physical exam techniques should be performed in the order of least to most intrusive. |
| T | F | 3. When using percussion, it is easier to detect a change from dullness to resonance. |
Circling a letter rather than writing or printing it will prevent misinterpretation of the students’ handwriting. With completion items, printing blank spaces for the answers in tandem, as in the following example, makes scoring difficult:
1. List three responsibilities of the circulating nurse during induction of general anesthesia.
_________________ _________________ _________________
Instead, the blanks should be arranged in a column along one side of the page, preferably on the left, as in this example:
| 1. ____________________________ | 1–3. List three responsibilities of the circulating nurse during induction of general anesthesia. |
| 2. ____________________________ | |
| 3. ____________________________ |
Arrange the Correct Answers in a Random Pattern
Many teachers have a tendency to favor certain response positions for the correct or keyed answer to objective test items, for example, to assign the correct response to 183the A or D position of a multiple-choice item. Some teachers arrange test items so that the correct answers form a pattern that makes scoring easy (e.g., T-F-T-F or A-B-C-D). Students who detect a pattern of correct answers (e.g., the correct answer never appears in the same position two or more consecutive times) may use this information to obtain higher test scores than their knowledge would warrant (Gronlund, 2006).
Many item analysis software programs calculate the number of times the keyed response occurs in each position. While reviewing these reports, teachers may determine whether the correct answer positions occur in approximately equal numbers, keeping in mind that multiple-choice, true–false, and matching items may have differing numbers of response options. Although these reports would not be available until after the test is administered and scored, they could alert teachers to use a different technique to more evenly distribute the correct answer position if the test (in its entirety or with minor item revisions) is used again. The teacher also can tally the number of Ts and Fs, or As, Bs, Cs, and Ds, on the answer key by hand. For true–false items, if either true or false statements are found to predominate, some items may be rewritten to make the distribution more equal (although it is recommended by some experts to include more false than true items).
Gronlund (2006) recommended that the position of the correct response in multiple-choice items be randomly assigned. One method for obtaining a random order is to place all responses to multiple-choice items and all premises and responses in a matching exercise in alphabetical order by the first letter in each, as described in the following section.
Arrange Options in Logical or Numerical Order
The response alternatives for multiple-choice items and the premises and responses of a matching exercise should be arranged according to a logical or meaningful order, such as alphabetical or chronological order, or in order of size or degree. This type of arrangement reduces reading time and helps students who know the correct answer to search through the options to find it. This strategy also tends to randomly distribute the correct answer position as discussed earlier, especially on lengthy tests. When the options are numbers, they should always be in numerical order, preferably ascending (Gronlund, 2006). This principle can be seen in the example shown in Exhibit 10.2.
Number the Items Consecutively Throughout the Test
Although test items should be grouped according to format, they should be numbered consecutively throughout the test. That is, the teacher should not start each new item format section with item number 1 but continue numbering items in a continuous 184sequence. This numbering system helps students to find items they may have skipped and to avoid making errors when recording their answers, especially when using a separate answer sheet.
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
