Writing the Vignette

One of the goals of a multiple-choice item is to determine whether or not students have obtained the knowledge required to perform appropriately and safely as practitioners after they graduate. Providing a real-life clinical scenario in each multiple-choice item is an excellent way to achieve this goal.

It is important to note that not all multiple-choice items can be written with a vignette, requiring an examinee to synthesize information to determine an answer. Sometimes writing a recall item is necessary because there is no way to develop a vignette or scenario based on the objective being assessed. However, use of recall items should be kept to a minimum. These types of items require the student to recall an isolated fact, which means that these items test the student’s memory and not his or her knowledge. A student at any level, whether high performing or low performing, can read a textbook the night before a test, memorize a few facts, and pass a test that includes recall items. Items that require the student to apply knowledge and synthesize information provided in a vignette are much more effective at distinguishing a low-performing student from a high-performing student.

When writing a vignette, a vital element to consider is the objective for the item. This seems like a simple step, but knowing the objective and being very clear about the objective throughout the writing and review process are crucial.

With the objective in mind, the first step of writing the vignette includes adding information about the following components to construct the patient scenario:

• •
  

  Patient age and gender

  
  
• •
  

  Site of care (e.g., hospital, office, emergency department, clinic)

  
  
• •
  

  Presenting symptoms

  
  
• •
  

  Duration of symptoms

Most multiple-choice items that require an examinee to apply knowledge will contain these four components within the first or second sentences of the vignette.

Other components of a vignette that may be appropriate to include, depending on the objective of the item, consist of the following:

• •
  

  Pertinent personal or family history

  
  
• •
  

  Vital signs

  
  
• •
  

  Physical examination findings

  
  
• •
  

  Imaging study findings

  
  
• •
  

  Laboratory findings

Note that this is not an exhaustive list of components for a vignette, and there may be more elements that need to be included based on the objective of the item, but this list is a good guideline to follow.

After the instructor has determined the appropriate components to include in the vignette, he or she needs to keep the following rules in mind:

• 1.
  

  Focus on a single, clearly defined topic. If the instructor presents a patient with several medical issues, it is important to keep the focus of the item on the primary medical concern. For instance, if a patient who has diabetes mellitus and major depressive disorder comes to the office with a presenting symptom of shortness of breath, the shortness of breath should be the symptom that is addressed within the item.

  
  
• 2.
  

  Exclude irrelevant information or unnecessary complexity. If the goal of an item is to determine whether or not a student has appropriate knowledge, then fairness of the item is of most concern. It is unfair to add unnecessary information to an item to confuse the examinee. It is up to the instructor and reviewer to determine what information is needed for the examinee to have a fair chance of answering the item correctly.

  
  
• 3.
  

  Do not use jargon or abbreviations that are not well known. Many students do not speak English as their first language. Using medical jargon can be confusing to these individuals and should be avoided. Unless the instructor has provided a list of acceptable abbreviations to the students, most abbreviations, especially those that can represent several different phrases, should be avoided.

Writing the Lead-in Question

After writing a focused vignette that includes the appropriate information, the lead-in question is the next component to develop. The most important element to keep in mind when writing a lead-in question is that the examinee should be able to answer the question without looking at the options . If the lead-in question is written well, the examinee should have no doubt in his or her mind as to the question being asked.

The following are some rules to keep in mind when developing an effective lead-in question:

• 1.
  

  Ask a focused question. As when writing the vignette, the lead-in question should focus on one clear question and should not include information that in unnecessary or tricky. Here are some examples of simple, focused, and clear lead-in questions:

  
  
  
  
  • •
    

    Which of the following is the most likely diagnosis?

    
    
  • •
    

    Which of the following is the most appropriate next step?

    
    
  • •
    

    Which of the following is the most important additional question to ask the patient?

    
    
  • •
    

    The most appropriate additional management is replacement of the patient’s current drug therapy with which of the following medications?

  
  
  
  
• 2.
  

  Use superlatives such as most or best. All of the options in a multiple-choice item should be plausible, with one option being the most correct (see next section for more information on this). Therefore, including most or best in the lead-in makes it clear to the examinee that she or he is being asked to choose the most correct option.

  
  
• 3.
  

  Use positively worded lead-ins. Words such as least, not, or except within a lead-in question (e.g., Which of the following drugs is least appropriate to administer?) can often be misunderstood or even missed by examinees, which is unfair. These types of questions also require the examinee to read all of the options listed, which as stated previously, is a sign of an unfocused lead-in.

  
  
• 4.
  

  Avoid true/false lead-ins. True/false items are difficult to write, especially the options, and this type of item can be unfair to the examinee. For instance, the examinee may know the information but may get confused by long options with subtle differences that can take up valuable time to read. Also, it is difficult to make options that are either completely true or completely false, which then requires the examinee to guess what the instructor had in mind when developing the item.

  
  
• 5.
  

  Avoid use of may and may happen. Use of indefinite words such as these does not support a focused, clear question and can be ambiguous for examinees.

Writing the Options

After the vignette and the lead-in question are complete, it is time to write the answer, or best/most appropriate option, and the distractors, or plausible but incorrect options. It is important to note that the true mark of a well-written item is when all of the options have been chosen by all of the examinees . Ideally, an item that performs well should have the following statistical characteristics:

• •
  

  The majority of the examinees choosing the correct answer, with the higher performing examinees choosing the correct answer more often than the lower performing examinees.

  
  
• •
  

  Some higher performing examinees choosing the distractors. Not all of the distractors need to be chosen by the higher performers.

  
  
• •
  

  The majority of lower performing examinees choosing the answer but many of the lower performing examinees choosing each of the distractors.

For an example of a statistically reliable item, see Example 1 .

Therefore, it is most appropriate to create distractors that are plausible but incorrect. If the examinee knows the information, she or he should get the answer without even looking at the options. If the examinee does not know the information, she or he will have to read all of the options and will most likely choose one of the distractors, which will all be plausible but incorrect.

The following are the rules for writing options:

• 1.
  

  Write homogeneous options. All options within an item should be similar, as in all tests, all diseases, all drugs, and so on.

  
  
• 2.
  

  Create grammatically parallel options. If the answer starts with a verb, the rest of the options or distractors should start with a verb as well so that the answer does not stick out as correct.

  
  
• 3.
  

  Use options similar in length and complexity. All options should be the same length so that the answer does not stick out as correct. Shorter options are preferred.

  
  
• 4.
  

  Avoid use of all of the above or none of the above. Use of all of the above makes the item unfair; if the examinee chooses one of the options and not all of the options, she or he technically got the item correct, which could be controversial. Use of none of the above is also unfair because if the instructor uses a focused lead-in and asks for the best option, the best option is not listed. Also, using these phrases basically turns the item into a true/false item, which requires the examinee to read all the options to determine the answer.

  
  
• 5.
  

  Avoid indefinite words such as frequently and often. The definition of frequently and often differs from person to person, so including such words in the options can be misleading and confusing.

  
  
• 6.
  

  Avoid use of absolutes such as never, none, no, all, always, and every. In the health sciences, it is almost impossible to say that something will never happen or will always happen.

  
  
• 7.
  

  Use single phrases. Use of dual options (i.e., administration of intravenous fluids and admission to the hospital) is appropriate as long as all of the options are dual. This is difficult to do, so it is recommended that all options are single phrases to avoid this.

  
  
• 8.
  

  Use mutually exclusive options. Each option should be able to stand alone. One option should not be included in another option. This is especially true when using number ranges. For instance, if option A is “10 to 20%” and option B is “15 to 25%” and option B is the answer, then option A can also be correct because the range of 10% to 20% is included in option B.