How to take your multiple-choice questions to the next level

The question

One of the biggest challenges of composing a question is to not give away the right answer. For example, the question “What makes this an unsafe situation” already states that it is unsafe.

To avoid leading questions try to put as much relevant information as possible in the video, rather than in text. And write your question in the simplest and clearest way. This is one of the reasons we love short active questions.

There is a thin line between too easy and impossible to answer. Your question should be designed so the trainees who know what to do, can find the correct answer. And try to only ask one thing per question. This helps you later on when creating distractors.

There are different multiple questions types we use in immersive learning, such as:

• Multiple Answers: the basic A, B, C question
• Yes or No: we use this mostly for conflicts
• Negative Questions: What shouldn’t you do first?
• Combined Response: Mostly used for conversations to build up a sentence as a trainee. Or to ask a review question

The answer

Because we create non-linear stories, it could be that you have more than one correct answer. That way you create more than one way to end the scenario. You should aim to have just one best answer. Having one best answer makes it less confusing for the trainee to understand what you want them to learn. One other thing you could consider is to shuffle the placement of the correct answer(s). Try to mix it up!

The distractors

Writing plausible distractors is one of the most difficult aspects of composing multiple-choice questions. That’s why creating two distractors is most effective. Just because you're less likely to come up with more really good distractors. And even when you can do it, research teaches us that there is no benefit in terms of the learning effect. So save yourself some time!

It’s important to not include distractors that are obviously wrong. Because then your question will not make any impact. Avoid absurd or jokey ones too, they are easily spotted.

So what should you do?

• Make the alternatives plausible and attractive.
• Keep all option lengths similar and each distractor grammatically similar to the correct answer.
• Keep the distractors sufficiently different to the correct answer and not just clever or subtle wording.
• And again; put as much of the necessary information as possible in the video or question, rather than in the alternatives.

What can help is to think of the most common mistakes or misconceptions. And also ask yourself the question; “Why would a trainee select this distractor? Which motive can the trainee have?”. Can’t think of any? Then you should remove your distractor or create something in the story which makes it more likely that the trainee will select that distractor.

An example

Let’s put all the information above to the test with an example. This is a multiple-choice question in an evacuation training:

The alarm sounds, which safest evacuation route do you take?‍

A. Jump out the window
B. Open the door to the stairs
C. Take the door with the neon evacuation sign above it next to the red wall
D. None of above

First the question, you can hear the alarm sound in the video. So there is no need to write that in the question. And the question gives away that you need to evacuate. I would cut this question in two, so first I ask if the trainee wants to evacuate and then ask how:

What do you do?‍

A. Take a look at what is going on
B. Get out of the building
C. Finish my job

Where do you go?‍

A. Jump out the window
B. Take the stairs
C. Take the elevator
D. Take the door with the neon evacuation sign above it next to the red wall

First “D”, the correct answer, stands out because it has much more information than the distractors. It describes a lot of information that you can see in the video. And then the distractors, I think “A” is too obviously wrong so I would remove that one. And I would think of a way to make answers “B” and “C” more plausible. I would talk with some experts within the company to ask about common mistakes or misconceptions. But just to give you an idea: you could have an important colleague walk towards the stairs, to see if the trainee just follows. Or you could talk with a colleague at the beginning of the scenario who tells you that an elevator is much safer than the stairs. The question would look like this:

Where do you go?‍

A. Take the stairs
B. Take the elevator
C. Take the door

To conclude

There's more to creating good multiple-choice questions than you might initially think. Let others test your scenario and questions so you get a feel for how easy or difficult they are. Remember that creating good questions and distractors means going back and forth. It takes some tweaking to find the right wording and lengths. Additionally, you can change the story to make your question more challenging to answer. Good luck stepping up your multiple-choice questions!

When you have any questions, please feel free to reach out through chat on our website or danny@warpvr.com. We also offer review sessions, in which I’ll go through your scenario and give you tips and tricks on how to improve it in an online meeting.