Writing good multiple-choice questions is an art. Writing good answer options even more so. While it is relatively easy to write the correct answer option, writing clever, reasoned, incorrect alternatives can be where your instructional design skills can be stretched.
When learning content relates that to a list of items (e.g. the planets in our solar system) it can be too easy to write a question asking learners to select which of the options (i.e. list items) is correct. Too often coming up with an appropriate incorrect answer/s, that isn’t too obvious can be challenging. To resolve this, learning designers often turn to the overused ‘all of the above’ option, which is to have a list of all the correct options, with the final option being an ‘All of the above’ option and the true, correct answer. This results in a seemingly easy fix to the problem of finding an appropriate incorrect option, but how effective is it as a learning tool?
Why is ‘All of the above’ an ineffective question option?
Using ‘All of the above’ as a question option may be a time saver from an instructional design perspective, but it is not an effective way to measure knowledge retention.
Firstly, if you completed a number of courses on any topic, designed by any instructional designer, in the majority of cases ‘All of the above’ will be the correct option. For any learner who has undertaken enough multiple choice quizzes during their life will know, either consciously or subconsciously, that this option is most likely correct. The result is a high percentage of learners who achieve competency in the question, but no evidence that the learners understand the topic presented in the question.
Another scenario is that the learner recognises one or more of the options as correct. This can lead to two outcomes. In one scenario, the learner assumes that as they know, some of the options are correct, that all answers must all be correct and select ‘All of the above’. The result is that they are marked as correct, but not through mastering of the subject. In another scenario, the learner assumes that not all options could be right, as questions typically have an incorrect option and only select the options, they are certain are correct. While the learner gets the question incorrect, which is appropriate if they have not yet achieved mastery of the topic, the learner is subjected to additional mental strain having to try to guess whether or not the question is a ‘trick’ rather than focusing on their understanding of the topic presented.
Changing our question design to avoid ‘All of the above’
The same learning outcomes can be achieved without using the ‘All of the above’ question option. It is all about rethinking our question design. Try to avoid writing questions that require a list of correct options. These questions will lead to the inevitable use of the ‘All of the above’ or similar ‘None of the above’ option.
For example, take a question on the solar system written with an ‘All of the above’ option:
Which of the following planets are closer to the sun than Saturn?
- All of the above (Correct)
To achieve the same outcome of gauging a learner’s understanding of the order of planets in the solar system, a similar question could be written like the example below:
Which of the following planets is third from the sun?
- Earth (Correct)
While not necessarily the most excitingly written question ever written, this example shows how with a simple re-write a question can shift from a list of correct answers with an ‘All of the above’ option, to a question with a similar test of learner knowledge, but without the inherent issues associated with an ‘All of the above’ question. With this simple change to your question design, your quizzes will take a step forward, improve confidence with topic mastery and provide a better experience for your learners.
- Beware of the gap – Considering the digital literacy of your learners – 20th October 2020
- Is ‘All of the above’ an effective question option? – 20th September 2020
- Asking why? – Solution-focused learning design – 20th August 2020