Tag: data forensics methods

Trojan Items and Answer-key Arbitrage

Posted on 03/02/08 by Caveon 1 Comment

Today is the first day of the annual ATP Conference (Association of Test Publishers). This afternoon I will present a workshop titled, “Strategies and Tactics for Limiting Item Exposure.” We will be exploring innovative ideas for protecting tests and items from theft. It’s easy to understand why test publishers are concerned about test theft. High-quality [...]

Read more

Are identical answers to exam questions proof of cheating on tests?

Posted on 02/18/08 by Caveon No Comments

When it comes to supporting an allegation of cheating on tests, there is rarely better statistical evidence than having two (or more) tests with identical sets of responses, or identical answers. Having a great interest in this topic, I have read carefully the abstracts of Rice University Honor Council meetings where these types of allegations [...]

Read more

Can you prove cheating on tests using statistics?

Posted on 02/11/08 by Caveon No Comments

There is a children’s game known by various names as “Whisper,” “Secrets,” or “Gossip” where a secret is shared and passed from one player to the next. The last player hearing the secret says it aloud, often with hilarious results. These same distortions happen in the news media, as journalists cite other reports or each [...]

Read more

Trouble in Section K

Posted on 02/07/08 by Caveon No Comments

Elf mistress Heloise entered Elvin’s office (Head of Section K) quickly. “For the eighth week in a row, the reject rate from Section K is three times the rate from the previous twelve months,” she said, handing the weekly quality report to Elvin. She continued, “I was so impressed when your section scored higher on [...]

Read more

The case of the waylaid answer key

Posted on 01/17/08 by Caveon No Comments

Recently there have been many reports of lost databases, stolen computers, and misplaced documents. Is it any wonder that tests and exams are also experiencing the same problems? For example, last November in New Zealand the home of an employee of the Qualification Authority was burglarized and a laptop containing math items for the National [...]

Read more

No-Fly List shenanigans

Posted on 01/14/08 by Caveon No Comments

Just last week a five-year old boy was detained by TSA (Transportation Security Administration) because his name was similar to a suspected terrorist on the no-fly list. The reporter wrote, “A five-year-old boy was taken into custody and thoroughly searched at Sea-Tac because his name is similar to a possible terrorist alias. As the Consumerist [...]

Read more

‘Sabermetrics,’ baseball and steroids

Posted on 01/08/08 by Caveon 1 Comment

Prognostications are that Mark McGwire will not be inducted into Baseball’s Hall of Fame this year again, because of admitted steroid use. Here is the URL to the article: http://www.nationalpost.com/sports/story.html?id=221516 In 2005, McGwire ducked the direct question whether he had used steroids or performance-enhancing drugs (PEDs). Many statisticians think that steroids do not improve performance, [...]

Read more

Benefits-payment cheater caught using statistics

Posted on 11/20/07 by Caveon No Comments

The other day a woman in the UK was caught in a lie where she fabricated the existence of seven children to receive government benefits. She claimed to have given birth to quadruplets in 2005, to twins (who were delivered one week apart) in the same year, and then to a seventh child in 2007. [...]

Read more