The incident of the pilfered test booklet

Georgia bit her lip nervously as she peered out the rear-view mirror of her car. She had already been idling 10 minutes longer than allowed and campus security would be returning shortly. Then, she saw them, exiting the library. Ignacio was detained by a man in uniform. Vincenzo broke into a run, sprinted to the car, and hopped in. “Step on it,” he said. Georgia sped away. “What about Ignacio?” she asked. “Don’t worry. I have it right here,” he replied as he slipped a digital camera from beneath his jacket, extracted a memory card and handed it to Georgia. She grinned. Now, she would be able to pass the test and become an intern at Waldo & Cramer Industries. Once inside W & C and with her computer skills, her current employers would soon be very, very happy.

The above fictionalized account is based upon an incident which Caveon was asked to investigate in 2004. Our client wrote,

“We had an incident over the weekend concerning the XYZ exam …. The examiner contacted our office during the 3rd section of the examination. Two examinees were acting suspiciously throughout the exam. They had questions about how long the breaks were and what would happen if they returned late from the break. During the break, the proctor noticed that one of the test booklets was not on the applicant’s desk.

The proctors noticed that the two examinees went to their car and came back late from the break. When addressed about the booklet, they said they did not have the booklet and then dropped it from their jacket and said, ‘there it is’. They were allowed to continue, although the proctor told them their scores would be invalidated. They were addressed by the proctor and campus police after the exam and questioned. One of the examinees was released as he stated he had nothing to do with the incident. The other fled the scene in a car that was waiting for him, as he was being escorted to check his car to see if there were images on his cell phone of the test booklet. The names of the suspects are Inigo and Vinny.” (Actual names have been changed.)

Results of Investigation

Caveon conducted an investigation into this incident and we discovered that the two individuals, Inigo and Vinny, were enrolled at a nearby university but they were not enrolled in courses of study or college majors that would be consistent with taking the admissions test connected with this incident. Furthermore, we determined that one of these students had lost his passport during the summer and the other had his driver’s license stolen. The information was corroborated and led us to infer that both of these students were victims of identity theft. Some other individuals committed test fraud in their names.

We also discovered that the test thieves were given the opportunity to steal the test because the test site administrator had not collected testing materials during breaks or the lunch period, as per test administration policy and procedures. One of these individuals, “Inigo,” had taken and failed the test approximately six weeks earlier. We presume that this individual determined that an opportunity existed to sneak the test booklet out of the testing site at that time.

In our report, we concluded that the imposters (or identity thieves) took the exam with the intent of exposing the exam content for one or more of the following purposes: for themselves, on behalf of another individual(s), for mass distribution, or for financial gain. We also suggested that, with suitable revision to the test administration policies and procedures, the likelihood of a security breach could be reduced.

Forensics analysis

Another phase of the analysis was to statistically analyze the test responses. It is difficult to infer “intent to steal” from data analysis, but the data are revealing. One of the statistics that we use in Caveon Data ForensicsTM is known as the bimodality statistic. With this statistic, we assume that most individuals answer the test questions consistently according to the observed performance (or a single level of ability). However, we allow the possibility for some individuals to answer the test questions according to two levels of ability (or in two different modes, hence the name bimodality). Using these statistics we found that Vinny’s test was somewhat aberrant (at the probability level of one in 2,000) and that Inigo’s test was extremely aberrant (at the probability level of one in 200 million). These data, along with comparative “normal” data at the same ability levels, are shown in Figures 1 and 2.

Figure 1: Comparison of Vinny’s test with a normal test

Figure 2: Comparison of Inigo’s test with a normal test

The data confirm that both of these individuals took the exam at two levels of ability. The probability of the high level is shown using the yellow line. The probability of the selected response using the low and high levels is shown using the blue and pink lines, respectively. We infer that Inigo demonstrated more information and knowledge about the test content than Vinny, but both of them appeared to be answering the test questions for some other purpose than obtaining a score and an actual measure of their knowledge of this content area. It appears likely that these individuals were connected with the content area being tested.

This incident is extremely instructive. It illustrates that not all test takers are as they appear and that an unfair advantage may be gained in many ways. I had always wondered whether there would be a motive to steal an identity for the purpose of taking a test and now I know.

Dennis Maynes

Chief Scientist, Caveon Test Security

Leave a Reply