This is part three in my analysis of the changes to the SAT. Part 1. Part 2.

Another forthcoming change to the SAT is the number of answer choices per question: there will be four rather than five options for all questions. This is another way in which the new SAT will more closely resemble the ACT, which already uses four-choice questions for all the tests except Mathematics.

I can already picture the complaints that this makes the SAT easier. And if you are approaching it from the perspective of guessing, it's clearly easier to guess correctly with fewer choices. Your odds for a single question are 25% rather than 20%. But over the course of the whole test, guessing is not a productive strategy, as Randall Munroe pointed out.

The odds of guessing correctly on all the multiple-choice questions on the current test are

\[\frac{1}{5^{160}}\approx \frac{1}{6.84 \times 10^{111}}\]

[Note that Munro got the number of writing questions wrong, so I corrected his figure. The actual odds are even worse than he calculated.]

The proposed test spec just released has fewer multiple-choice questions (141) and fewer options to choose from, so your odds rise to

$$\displaystyle\frac{1}{4^{141}} \approx \frac{1}{7.77 \times 10^{84}}$$

Of course that's still an staggeringly remote probability. If we recalculate Munroe's computer-guessing scenario for the new test, the odds of correctly guessing all the math questions alone after 5 billion years rises to only 5.9%.

There has been a lot of research about the optimal number of choices for multiple-choice questions, and it turns out that, as in other areas of life, having more choices is not necessarily better. In fact, as question writers know, it's very hard to come up with more than two or three plausible wrong answers for many questions. If you must come up with five options for every question, it's often the case that one or two are implausible fillers that few, if any, students will pick. The number of options you have does affect how many questions you need to put on the test to have a reliable measurement. For example, you can make a good test with only two options per question, but you need to have more questions on the test. On a test the length of the SAT, the choice between four or five options doesn't significantly affect the reliability, and it doesn't simplify the test for the student because the deleted option would probably have been an implausible choice in the first place.

Part 4.