For all their downsides, multiple-choice questions do one thing very well
by Brian Moon
Objections to multiple-choice questions have been central to the rising fervor against standardized testing. Many argue that we shouldn’t be using 100-year-old methods for assessing today’s learners.
However, despite its vocal critics, multiple-choice questions (MCQs) are still the predominant testing tool we use today.
So why do we continue to rely so heavily on them? Figuring out how to move beyond multiple-choice requires understanding the factors that made MC innovative a century ago — and why those benefits still apply today.
Applying mass production to education
The origin story of MCQs takes us back 1914. The Second Industrial Revolution, exemplified by Henry Ford’s invention of the Model T, was ushering in a new era of mass production and accelerated population growth. In the midst of this growing market and increased demands for an educated workforce, educator Frederick Kelly had a goal to improve the efficiency of assessing his students. The result was his Kansas Silent Reading Test.
Kelly’s test contained the essential features of MCQs as they exist today, which require test-takers to choose correct answers to a question from items in a list. It was strict but simple, which made the task of assessing expanding classes much less time-consuming. Presumably, it also eliminated grader bias.
Urgency and technology boost scale and efficiency
Around the same time in the field of psychology, Robert Yurks and Robert Woordsworth established a precedent for broad-scale use of MCQs. They adopted Kelly’s format to help the U.S. Army evaluate recruits’ personalities and intelligence during World War I. The military’s large-scale use set the stage for MCQ to achieve ubiquity.
While the actual value of the exercises was questionable, by the time WWII ended MCQs had won a large following, due in part to the introduction of mechanisms that made them even easier to grade. In 1937, Reynold Johnson pioneered an instrument for scoring the tests electronically. A few decades later, Everett Lindquist — father of the GED and ACT — raised the bar even higher with his contributions to optical mark recognition. His work spawned a whole industry around the practice of MCQ delivery, which continues to thrive today.
For his part, the man who inspired the MCQ movement later become his own critic. While president of the University of Idaho, Frederick Kelly offered a vision for “liberal, integrated, problem-based learning” that pushed students to higher levels of thinking than what MCQs could measure. However, others at the university did not share his new priorities — Kelly would eventually be removed from his post.
Those critical of the broad-scale use of MCQs today echo the same concerns and desires that Kelly expressed shortly after inventing them. And yet, MCQs remain the most widely used assessment item type at all levels of learning and testing for one clear and important reason: administrative efficiency.
With so much to assess, efficiency is king
Once the effort of authoring is invested — which is no small feat — MCQs are remarkably time and cost-effective to distribute, take, and score. This advantage is even more appealing today than a century ago because the number of students is not the only variable that has affected teacher workloads. In fact, though the number of students per teacher has been generally decreasing for more than 50 years, today there is so much more knowledge and understanding to assess than in decades past.
Kelly introduced his method at a time when the average number of yearly school days U.S. students attended had risen from 90 to 120 in just a couple of decades. By 1980, schools were in session about 170 days, prompting an investigation into the effects of curriculum overload on teacher performance. Today, the average number of school days is 180 in most U.S. states.
For all of their downsides, MCQs have helped education institutions keep pace with increased assessment demands that changing laws and social contexts have required. Psychometric advancements have helped mitigate some concerns about test validity, and computerization has further increased administrative efficiency.
The challenge — and opportunity — for alternative assessments
Despite their undeniable efficiency, frustration with MCQs and the mundane teaching strategies they can compel has been growing for some time now, along with calls for novel assessment approaches that can disrupt the status quo. Indeed, some see assessment redesign as a lynchpin for transforming modern approaches to teaching, particularly for student populations that have historically had the worst experiences with MCQs in standardized assessments.
The calls for reform are entirely appropriate. But they must also recognize where the bar is set and why it is there. Alternative assessments that can’t keep pace with the administrative efficiency of MCQs will ultimately be left on the sidelines.
Continuing the legacy of its quest for assessment efficiency, the U.S. Army has recently encouraged competition in one area where MCQs struggle: authoring.
My own company’s approach to this challenge has focused on concept mapping-based assessment authoring. Concept maps, which have been used for assessment since the early 1970s, were themselves created in reaction to inefficient methods for understanding students’ mental models. Increasing concept maps’ overall use efficiency has always been a goal, and new technology developments show promise for automated assessment authoring that could make them a real contender to MCQs.
In the meantime, until alternative approaches can get close to the efficiency that MCQs offer, MCQs will almost surely remain the default when it comes to large-scale, high-stakes, and even classroom testing. Moving beyond multiple-choice will require taking moonshots to create assessments that not only provide deeper insights into learning thinking, but do so faster and cheaper at every scale.
Want to gain more insight into factors shaping the future of assessment?
Register to attend Beyond Multiple Choice 2021 here.