Hi, my name is Bob. I’ll be stealing your test items today…
Written by: Jamie Mulkey, Vice President, Caveon Test Development
Who remembers the 1991 Steve Martin movie classic, LA Story? In one particular scene, Steve’s character, Harris Telemacher, goes to an ATM to get some cash. He is greeted by a robber and the following exchange takes place:
“Hi, my name is Bob. I’ll be your robber.”
“Hi, how are you?” Telemacher hands Bob the money.
“Thank you very much.”
What just happened? This is an example of an expected behavior. In this comedic farce, it has become so common to be robbed at an ATM that individuals anticipate and expect this behavior. Mr. Martin’s character accounts for his loss for by simply taking out additional funds to pay the robber.
This behavior is not unlike what some of our certification programs exhibit today with their test items and exams. Because their exams are delivered in less-than-secure test delivery channels, items are easily and quickly stolen. They are then sold on braindump websites where test takers purchase them to prepare for important high stakes exams. As a result, these programs find themselves designing a multitude of strategies in anticipation of the items being stolen – all because their test delivery channel can’t protect their exams.
What types of strategies are we talking about?
Creating extra test items and forms. Now, don’t get me wrong, from a test security perspective it’s always a great idea to have a bank of items in reserve. However, it is very discouraging to see a great deal of effort and resources being used to develop great items; knowing they will be stolen within the first month of publication.
Designing test security strategies that catch test cheaters in the act. This involves intentionally placing mis-keyed items in the item bank to identify individuals who’ve used braindump sites. To the credit of these programs, there are some very clever and statistically valid ideas that have been used successfully.
But these do not solve the real problem: the test delivery vendor isn’t able to deliver test items securely. So like Harris Telemacher, these programs hand over their items to the robber and are thankful they have enough items in reserve to deliver their certification exams.
What can be done? Here are some questions to consider:
- Are there test delivery alternatives? With the continued interest to push test delivery to Internet-based testing, could we see better results in protecting test items with online proctoring models?
- Could testing programs find a more manageable landscape? Could they forgo the franchised testing center model and stick with company-owned test centers?
- Would a hybrid approach work? Could a blend of franchised and company-owned centers be used; placing the company-owned centers in geographic locations where test theft is most likely to occur?
Of course the devil is in the details, but if we can put as much creativity into thinking about how to deliver secure exams as we do trying to outsmart test cheaters, perhaps we can expect to get a better result.