- Open Access
- Open Peer Review
Adding to the debate on the numbers of options for MCQs: the case for not being limited to MCQs with three, four or five options
BMC Medical Education volume 19, Article number: 354 (2019)
There is a significant body of literature that indicates that the number of options for single-best answer multiple choice questions (MCQs) can be reduced from five to three or four without adversely affecting the quality of the questions and tests. Three or four options equates to two or three distractors respectively.
Whilst these arguments may be true when focusing on psychometric aspects of questions, we should also focus on educational and clinical authenticity aspects of questions. I present reasons for MCQs in tests to have a variable number of options which will usually be more than three, four, or five. These include: decisions related to broad clinical scenarios cannot be limited to a small number of options; options lists should include all possible combinations of option elements; and options that are rarely chosen can provide information regarding students and/or for students.
Finally, given computer based delivery, longer option lists are not impractical for examinees. In the contexts that are appropriate, it is time to consider a move to adopting appropriate and variable numbers of MCQ options and not be limited to MCQs with three, four or five options.
Multiple-choice questions (MCQs) are widely used in assessment within medical education and there are numerous articles comparing the number of response options and distractors . Building further on this, it continues to be postulated that reducing the number of options does not lead to a reduction in assessment parameters [2, 3]. This creates a prevalent opinion as articulated in more recent reviews [4, 5] and primary research reporting that reducing the number of options does not result in significant differences in assessment parameters  or can lead to improvement in the parameters . This body of literature suggests there are potential advantages and no disadvantages to reducing the number of MCQ options.
With this weight of evidence, why would assessment organisers not consider reducing to three or four options? In fact, to the contrary, I propose that assessment organisers consider having a variable number of options, which may mean increasing the number of options for many questions. The basis of this argument is that the evidence to reduce the number of options is based on a psychometric perspective, whereas the argument to have a variable number of options, which can include an increased number of options for many questions, is based on clinical authenticity and educational perspectives. In order to add to the debate, I proceed by way of presenting some reasons based on these perspectives.
Decisions related to broad clinical scenarios cannot be limited to a small number of options
Rarely do the questions faced by a clinicians in practice have exactly three, four or five options . Although primarily developed because of concerns that a limited number of MCQ options would cue candidate to a correct response , longer lists of options are also perceived as being more authentic to clinical practice . A single long list of options, hundreds of options long, could be used for all MCQs in an assessment [8, 10]. Equally, there is no reason the number of options has to be same for all questions in a test .
Extended matching type questions (EMQ) were developed such that without cueing from a short list of response options, clinical reasoning and knowledge may be assessed [12, 13]. An option list, of 5 to more than 25, is used for all questions, for a particular theme eg “What is the most likely diagnosis for a person presenting with chest pain?” . An advantage of EMQs is that changes to the stem (patient scenario) can lead to a change in the correct answer from a longer list of options, thus reflecting clinical practice .
The number of options should not be defined by the format but the content of the question . The number of options for a question should align with authentic clinical practice. There is not always the same number of options in clinical practice, so the number of response options should vary, and is likely to be more than three or four.
Options lists should include all possible combinations of option elements
Some options for an MCQ might be made up of several descriptive elements. Rather than try to select which combinations of elements should be included or not, an alternative is to increase the number of options to ensure all combinations are included.
As an example, this is a question with eight possible combinations of elements:
“A person with breathlessness has the following blood gas analysis … ..”
Which option best describes the blood gas analysis?
Metabolic acidosis with a normal Aa (Alveolar-arterial) gradient
Metabolic acidosis with an increased Aa gradient
Metabolic alkalosis with a normal Aa gradient
Metabolic alkalosis with an increased Aa gradient
Respiratory acidosis with a normal Aa gradient
Respiratory acidosis with an increased Aa gradient
Respiratory alkalosis with a normal Aa gradient
Respiratory alkalosis with an increased Aa gradient
Rather than trying to select which three, four or five options should be included or not, it is possible to have all eight. As will be discussed subsequently, clinically important incorrect answers and psychometrically important incorrect answers might be different.
Where there are two elements with two possibilities, then there are four possible options to include . This will also remove the futile hunt for a fifth option, when four options provides all plausible combinations of elements.
The number of options should include all combinations of elements, rather than limiting these to a set number of options for every question. The number of options will vary with the number of elements and therefore combinations, and is best supported by a policy of variable option numbers.
Options that are rarely chosen can provide information regarding students and/or for students
Do we run the risk of losing important information if we remove rarely chosen options from MCQs? Many of the analyses upon which the recommendations to reduce the number of options are based on the assumption that incorrect responses do not have distinct intrinsic information. This is erroneous, there is significant information in incorrect responses, as there are responses that would be potentially unsafe if chosen in practice [16,17,18,19,20,21]. Panels of clinicians can consider the potential clinical impact of incorrect responses, which can lead to incorrect options being stratified for potential (un) safeness [16,17,18,19,20,21]. The most potentially unsafe responses are rarely selected [18, 20]. Rarely selected distractors are unlikely to be considered psychometrically important. Clinically important distractors are different from psychometrically important distractors . Options that are rarely chosen can represent unsafe practices; it is vital to know which students are selecting these potentially unsafe responses [16,17,18,19,20,21]. Individual misconceptions can be included in feedback with the goal to direct personal learning development [16,17,18,19,20,21]. If they become apparent, cohort level misconceptions can be used with the goal to direct curriculum development. By removing rarely chosen but clinically important incorrect options representing potentially unsafe practices, we deny the opportunity for misinformed examinees to choose such options. The choice of unsafe options across multiple questions would be a concerning pattern that needs to be recognised to target learning and subsequent performance; should the pattern be repeated despite further learning opportunities, this information could be used to inform progression decisions [18, 20]. One postulated reason why examinees might continue to select unsafe options is the paucity of feedback they receive on answers that are unsafe as well as incorrect [18, 20].
The number of options in MCQs should be sufficient to include both psychometrically important distractors and clinically important distractors. As the number of each will not be the same for all content areas, their inclusion is likely to require more than three or four options, and is best supported by a policy of variable option numbers.
Computer based delivery has made longer lists of options more feasible
Assessments do need to be practical and feasible . Longer lists of response options might be difficult to fit on assessment documentation or for candidates to use. As already noted, MCQs with longer lists of options do not lead to impaired performance by examinees [8, 10]. A single long list of options, hundreds of options long, could be used, and such formats have proved feasible when facilitated by computer delivery , though it has also been implemented in a paper-based system .
As long as the question meets the cover test (the correct answer can be determined without seeing the options ), and the options are presented in a consistent logical order (e.g. alphabetical), then long lists are not a problem. Questions not meeting these standards are most likely to be flawed irrespective of the number of options.
With computer delivery of MCQ assessments, there is no space constraint on option lists, and each option is automatically set a corresponding response tick box. Computer marking mitigates errors in reading and grading responses.
Now that many institutions are moving to computer delivery and marking of MCQ examinations, it is time to consider the move to adopting appropriate and variable numbers of MCQ options and not be artificially limited to MCQs with three, four or five options.
Availability of data and materials
Extended matching type questions
Multiple choice question
Rodriguez MC. Three options are optimal for multiple-choice items: a meta-analysis of 80 years of research. Educ Meas Issues Pract. 2005;24(2):3–13.
Fozzard N, Pearson A, du Toit E, Naug H, Wen W, Peak IR. Analysis of MCQ and distractor use in a large first year health Faculty Foundation program: assessing the effects of changing from five to four options. BMC Medical Education. 2018;18(1):252.
Raymond MR, Stevens C, Bucak SD. The optimal number of options for multiple-choice questions on high-stakes tests: application of a revised index for detecting nonfunctional distractors. Adv Health Sci Educ. 2019;24(1):141–50.
Wilson I. What's best for multiple-choice questions: three, four or five? Clin Teach. 2014;11(7):568–70.
Gierl MJ, Bulut O, Guo Q, Zhang X. Developing, analyzing, and using distractors for multiple-choice tests in education: a comprehensive review. Rev Educ Res. 2017;87(6):1082–116.
Royal KD, Stockdale MR. The impact of 3-option responses to multiple-choice questions on guessing strategies and cut score determinations. Journal of Advances in Medical Education & Professionalism. 2017;5(2):84–9.
Kilgour JM, Tayyaba S. An investigation into the optimal number of distractors in single-best answer exams. Adv Health Sci Educ. 2016;21(3):571–85.
Veloski JJ, Rabinowitz HK, Robeson MR, Young PR. Patients don't present with five choices: an alternative to multiple-choice tests in assessing physicians' competence. Acad Med. 1999;74(5):539–46.
Huwendiek S, Reichert F, Duncker C, de Leng BA, van der Vleuten CP, Muijtjens AM, Bosse H-M, Haag M, Hoffmann GF, Tönshoff B. Electronic assessment of clinical reasoning in clerkships: a mixed-methods comparison of long-menu key-feature problems with context-rich single best answer questions. Med Teach. 2017;39(5):476–85.
Schuwirth L, Cvd V, Stoffers H, Peperkamp A. Computerized long-menu questions as an alternative to open-ended questions in computerized assessment. Med Educ. 1996;30(1):50–5.
Rogausch A, Hofer R, Krebs R. Rarely selected distractors in high stakes medical multiple-choice examinations and their recognition by item authors: a simulation and survey. BMC Medical Education. 2010;10(1):85.
Case SM, Swanson DB. Extended-matching items: a practical alternative to free response questions. Teaching and Learning in Medicine: An International Journal. 1993;5(2):107–15.
Beullens J, Struyf E, Van Damme B. Do extended matching multiple-choice questions measure clinical reasoning? Med Educ. 2005;39(4):410–7.
Samuels A. Extended matching questions and the Royal Australian and new Zealand College of Psychiatrists written examination: an overview. Australasian Psychiatry. 2006;14(1):63–6.
Coderre SP, Harasym P, Mandin H, Fick G. The impact of two multiple-choice question formats on the problem-solving strategies used by novices and experts. BMC Medical Education. 2004;4(1):23.
Tweed M, Wilkinson T. A randomized controlled trial comparing instructions regarding unsafe response options in a MCQ examination. Med Teach. 2009;31(1):51–4.
Tweed MJ, Thompson-Fawcett M, Schwartz P, Wilkinson TJ. A confidence and safety approach to MCQ scoring. Focus on Health Professional Education: A Multi-disciplinary Journal. 2012;13(3):84–92.
Tweed M, Schwartz P, Thompson-Fawcett M, Wilkinson TJ. Determining measures of insight and foresight from responses to multiple choice questions. Med Teach. 2013;35(2):127–33.
Curtis DA, Lind SL, Boscardin CK, Dellinges M. Does student confidence on multiple-choice question assessments provide useful information? Med Educ. 2013;47(6):578–84.
Tweed M, Stein S, Wilkinson T, Purdie G, Smith J. Certainty and safe consequence responses provide additional information from multiple choice question assessments. BMC Medical Education. 2017;17(1):106.
Rangel RH, Möller L, Sitter H, Stibane T, Strzelczyk A. Sure, or unsure? Measuring students’ confidence and the potential impact on patient safety in multiple-choice questions. Med Teach. 2017:1–6.
Swanson DB, Holtzman KZ, Allbee K. Measurement characteristics of content-parallel single-best-answer and extended-matching questions in relation to number and source of options. Acad Med. 2008;83(10):S21.
Crossley J, Humphris G, Jolly B. Assessing health professionals. Med Educ. 2002;36(9):800–4.
Ware J, Vik T. Quality assurance of item writing: during the introduction of multiple choice questions in medicine for high stakes examinations. Med Teach. 2009;31(3):238–43.
The author acknowledges constructive comments on earlier drafts of this manuscript from the editor and reviewers (BMC Medical Education), and Tim Wilkinson (University of Otago) and Fiona Hyland (University of Otago).
Ethics approval and consent to participate
Consent for publication
The author declares that he has no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Tweed, M. Adding to the debate on the numbers of options for MCQs: the case for not being limited to MCQs with three, four or five options. BMC Med Educ 19, 354 (2019) doi:10.1186/s12909-019-1801-x
- MCQ distractor