In comparison to the commercial SAT-prep books, the material that comes from the College Board is of much higher quality. That's not surprising, of course, since it's the College Board's test, and they get to reuse all those questions that they have spent so much time (and money) writing, editing, and validating.

Beyond the technical quality, my impression of the College Board material is that for the most part it doesn't waste students' mental energy on usage trivia, and it doesn't take positions (for the most part) on areas of disputed usage. Sure, there are points that are tested which I find foolish, but that folly is widely found in most usage books, so I suppose it's defensible. (Whether this sort of multiple-choice grammar test is a valid way to measure writing skills is a separate, deeper issue that I will put off for another time.)

Still, the College Board is not perfect. I had a WTF moment today while reviewing one of the practice tests that appears in The Official SAT Study Guide.

Here's the question (practice test #6, section 10, question 12):

12. Many changes occurred while she was president of the college, and they increased its educational quality as well as effectiveness.
(A) college, and they increased its educational quality as well as effectiveness
(B) college, they both increased the educational quality and effectiveness of the college
(C) college, which both increased its educational quality as well as increased its effectiveness
(D) college; these changes increased its educational quality and effectiveness
(E) college; these changes increased both the educational quality and effectiveness of the college

Take a moment and figure out which one you think is the best version before reading my analysis. [Imagine the Jeopardy theme playing in the background while you work.]

Ready?

The analysis is long, so here's the executive summary: this is a flawed question with two defensible answers.

First, I don't care much for any of the choices. Our task, however, isn't to pick the ideal version, merely the best among the available choices.

The first three (choice A represents the original sentence) are clearly problematic. The use of and in (A) is a bad coordination, as it suggests two separate, sequentially ordered actions, rather than an cause-effect pattern. The use of 'as well as' instead of 'and' also makes 'effectiveness' sound strange. It would be better as its effectiveness. (More on this later). Choice (B) creates a run-on sentence and lacks parallel structure. Choice (C) uses which vaguely (it's not the college that increased its quality but the changes), uses as well as after both (the normal pattern is both...and) and unnecessarily repeats increased.

(D) and (E) are tougher to distinguish. Both appear clumsy to me, but on the surface, it's hard to see any obvious grammatical flaw. One thing I focused on immediately was what appears to be a bare NP effectiveness (that is a noun phrase with no determiner). In (D), I would want to add its. But that doesn't favor (E), because (E) also has the same bare NP. In that case, I would prefer a the to balance things. The first is a possessive pronoun and the second is a definite article, but they are both determiners, and if you're going to argue that one is required, then the other must be as well.

In this respect, I find both versions somewhat awkward but equally acceptable.

Looking for other differences, the obvious one is that (E) is significantly longer than (D). The two changes that bring that about are the addition of both and the shift from its to an explicit noun phrase, the college. In (D), its can only refer to the college, so it's not ambiguous or the wrong number. Further, (E) creates identical endings for each clause: of the college...of the college, which strikes me as monotonous.

Finally, avoiding unnecessary words is explicitly mentioned by the College Board as a desideratum, so that tips the balance for me. I choose the simpler sentence, (D).

Checking the answer key, the College Board, I find, believes that (E) is better. WTF!

OK, back to the drawing board. It's time to rationalize. What could the question writer have been thinking? Could there be some structural ambiguity that I've overlooked?

Adding both to the sentence serves to make the two parts of the coordination more emphatic. It also provides a clear marker of the start of the coordinate pair. Without that marker, there is, in theory, some structural ambiguity.

That is, with the 'both', we know that the structure must be

[the educational quality] and [{the} effectiveness].

In (D), there are theoretically several ways to analyze the structure of the coordination.

1. [its [[educational quality] and [effectiveness]]]

That is, the pronoun applies to the whole conjoined structure. This is a perfectly sensible reading, one that we find all the time in writing of all kinds (cf., "his intelligence and consideration").

2. [[its educational quality] and [{its} effectiveness]]

That is, two whole NPs are conjoined and the second 'its' is elided.

3. [its [educational [quality and effectiveness]]]

That is, the adjective modifies the conjoined nouns together).

Structures 1 and 2 are semantically equivalent. The first one is completely standard syntax, and the test writers can't think that the second is a problem or the missing 'the' in (E) would also rule it out.

So the only thing I can assume is that they believe there is an ambiguity with the scope of the adjective (i.e., between reading 1/2 and 3). In some sentences, I can see this as an arguable ambiguity. For example, does "large cats and dogs" mean [[large cats] and dogs] or [large [cats and dogs]]? But here, I can't see how it makes a difference. In the original, effectiveness and educational quality seem to be treated separately, but in what, pray tell, does a college's effectiveness consist if not education?

If we're going to adopt so strict a notion of structural ambiguity, there is virtually nothing that we can write that would be acceptable. It's a theoretical ambiguity but makes no real difference in comprehension.

Having run out of explanations, I decided to sign up for College Board's on-line course, which gives you access to explanatory answers for all the practice tests in this book.

The official explanation is garbage. If I were a student, I would feel cheated. (As these explanations provide me grist for my analytical mill, I won't complain about the money spent, though.) It is, of course, much simpler than what I've done here, as the explanation is directed at students. They use only minimal technical language and seem to list only one salient problem for each wrong answer, although many answers contain more than one.

What's there, however, is inadequate. Here's the College Board's explanation for (D):

Choice (D) involves the use of a vague pronoun. The meaning of the sentence would be clearer if the pronoun "its" were replaced by a possessive noun, "the college's."

The annoyance of seeing a noun phrase called a noun I will overlook, but vague? Please. The college is the only eligible antecedent for its anywhere in the sentence. This is arrant hand-waving, a non-explanation. When there's only one possible interpretation for its, the argument that it would be clearer to use the equivalent NP could be used to ban any pronoun. So what are we supposed to do, omit all pronouns from our writing? Give me a break. Even within the restrictions of a non-technical explanation, the College Board owes the students taking its tests a better rationale than this.

I showed this question to two colleagues who are both experienced writing teachers, and they agreed with me that any lack of clarity in (D) is illusory. One also picked (D) as the right answer; the other picked (E) but admitted that it was a guess and thought that there was nothing to clearly distinguish one from the other.

So there's really nothing to recommend (E) over (D). At best, the question has two defensible answers. Alternately, if we use the principle of brevity as a tie-breaker, (D) is the right choice. By no stretch of the imagination is (E) unambiguously better than (D). In short, I think the College Board has screwed up.

So what are we to make of this problem?

First, this is not the case of a debatable but clearly articulated point of usage, such as using they to refer to an indefinite pronoun like everyone. This question is flawed by the internal standards of the SAT.

You see such bad questions in commercial test-prep books all the time. Many are far more flawed than this one. But I'm disappointed to find this in College Board material.

One mitigating factor is that the tests in this book are not copies of actual tests. When the SAT was revised in 2005, the College Board put out this book to help students prepare for the new format. And since there were no actual tests that had been released, they created sample tests cobbled together largely from old material. In the case of writing questions, which did not exist on the old SAT, some of the questions at least have come from material released on old SAT-II Writing and PSAT tests, but when I have been able to compare specific questions, it seems that many of them have undergone some alteration since the earlier version. I do not know whether those edits were done for this book and only looked at by the authors and editors or if they went through the complete process of review that operational test items must undergo before appearing on a real, high-stakes test. A number of other minor problems with writing-test items in my book makes me think it is the former.

If that is the case, consider this much-too-long post to be just grumbling. Writing good standardized test questions is a very difficult art. That's why real questions are evaluated repeatedly by committees of experienced test writers, outside content experts, and editors, then pre-tested on students and evaluated by psychometricians.

But if this question appeared on an actual SAT (or any of the predecessor tests that had writing questions), and if students were penalized for picking (D), I would be upset.

Additionally, the College Board's explanatory answers need to be reviewed by someone with a solid understanding of real grammar. While a highly technical account would be inappropriate for the intended audience, it's not all that much to demand accuracy. Calling a noun phrase a noun or referring to the "present progressive tense," as I saw elsewhere on the same test, is simply shoddy.