10 Feb 2014

Test Corruption

Submitted by Karl Hagen
The UK has asked ETS, the company that creates the SAT for the College Board, to stop administering two major tests, the TOEFL (Test of English as a Foreign Language) and the TOEIC (Test of English for International Communication) in Great Britain, for the purposes of fulfilling the UK's immigration requirements. A journalistic investigation revealed test centers that took the test for candidates, who paid large sums for the privilege.

I have no doubt that ETS will put the problems down to the bad actions of corrupt, third-party administrators who ran the test centers in question, but if the accounts in the story I linked are true, ETS has a lot to answer for too. In the testing situation described, ringers gave the answers for the oral portion and listening comprehension portion of the test while answers for the written portion were simply read out to all test takers.

What I want to know is how on earth ETS didn't detect this. Given the answers, every student in the room would get the identical (presumably perfect) score. And this didn't set off any alarm bells with the ETS psychometricians?

The only possible explanation is that they are not checking for such cheating, and for that lapse I have nothing but scorn. The problem of teacher/administrator cheating is well documented. When sufficient motive exists (for example, when teachers and schools are punished for poor performance, or---as here---simple greed), corruption among those giving the test is always a possibility. But this sort of cheating is much less studied than examinee cheating. That's not to say that there has been no work on the issue, though. Metrics do exist to detect such cheating, and I would expect ETS to be at the forefront in developing new methods to detect it.

At the very least, if the answers of all the students taking the test in a particular room are 100% correlated, the security staff of ETS ought to be on a plane heading out to investigate the next day.