Written by: Jennifer Miller, Data Forensics Coordinator, Caveon Test Security
October 24, 2016
At Caveon, we often share amongst ourselves interesting news articles or stories about test fraud. Some of the stories are very serious while sometimes they just relate an unfortunate instance of cheating (or alleged cheating) that provides insight into the various ways in which people attempt to gain an unfair advantage on high stakes tests. And then sometimes….we share the stories because they are, well, just pretty funny. A colleague of mine forwarded this one to me the other day with the following subject line: “Worst proxy test-taker EVER.”
(In case the link doesn’t work, the impersonator wrote his own name on the testing document in order to take the physical part of the police exam for his friend. The police caught him through self-incrimination!)
I’ll say he’s a pretty bad proxy test taker! One of the pitfalls of the ‘I’ve got the brains, you’ve got the brawn’ arrangement.
C’mon people! I’m no expert on cheating, but I’m pretty sure this guy could have done a better job than this. In fact I KNOW he could have because, as a member of the Caveon Data Forensics team, I have first-hand experience with some very sophisticated test fraud efforts. Cheating and test theft are widespread and big business. Take for example a study we performed where we found that in the course of six months on a single exam, one proxy-test-taking organization fraudulently took the test for more than 500 test takers. We estimated that this represented $500,000 in revenue from approximately six test sites and that the proxy test taking organization received about $1 million for a single exam title. We also have estimated that over 1,500 websites are selling live test content for the IT certifications industry alone!
We like to think that the police are smarter than the criminals, which was luckily the case in the above example, but in many instances this is not true; while those who are engaged in test fraud represent only a fraction of the population of test takers, nevertheless these individuals can be very clever, persistent, and can cause significant harm to our clients. It is our job, as test security professionals, to be smarter.
It is a daily effort to stay ahead of cheaters. From simple professional development efforts like passing around news articles about cheating to more sophisticated efforts like developing new products and new detection techniques, we have to remain on our toes. Here are some ways that you can stay ahead of the game:
Keep Apprised of New Technology
Do you keep an eye out for new technologies and view them from the perspective of, “How could this be used to commit test fraud?” For example, recently released smart watches and smart glasses represent exciting new ways for our social-media-centric society to stay connected, but to us they represent emerging threats to test security. Now examinees not only will be asked to surrender their cellphones upon entering testing centers, their eyeglasses may have to be inspected for cameras and recording devices as well! It’s critical that all of us keep an eye out for new technologies specifically designed for cheating. Cameras and ear pieces, which easily can be connected to smart phones and watches, are continually getting smaller and smaller, making them more difficult to detect. Cameras used for test theft can fit on the end of a ballpoint pen or on the button of a shirt.
Perform and Participate in Research
There are many research opportunities which usually require teamwork between testing professionals and testing organizations. Working together, we can advance the state of the art. For example, we have worked with clients to develop statistics that reliably detect users of braindump content. We also perform vulnerability assessments on new test and item types, looking for ways in which they may be vulnerable to fraud. Also, we take advantage of the new data elements that are provided through computer based testing to look for ways to detect more types of cheating behaviors, such as collaboration between individuals during testing session breaks.
Be Willing to Adopt, Evaluate, and Help Create New Products
We have noticed that the industry continues to innovate and, at Caveon, we are continually improving and introducing security products to help our clients protect their exams and the integrity of their scores. Some recent innovations include:
Discrete Option Multiple Choice items
DMOC is a unique and inherently more secure item type than traditional multiple choice. DOMC helps prevent and deter many types of test fraud, and also can help clients save money in both development and delivery. Try the items for yourself at http://trydomc.com.
Scorpion is a test development environment that enables collaboration among team members, access controls, and streamlined development, all within a secure framework. Check out Scorpion at https://testscorpion.com.
Caveon SEI is a tool to package secure exams. This new product, which will launch in March 2017, gives testing programs the flexibility to work with any test administration provider while adding an additional layer of security and a number of exclusive security enhancements and controls. For more information on SEI, contact Ben Hunter at firstname.lastname@example.org.
New Detection Methods
As new types of test fraud behaviors emerge and as more data elements become available, we continually seek new ways to measure and detect test security threats. Recently developed statistics include timing statistics, which take advantage of new data elements made available through computer based testing to detect various anomalies in the timing of examinees’ responses to items. These can be especially compelling when response latencies are combined with other elements, such as response time stamps and answer change information. These new statistics are able to identify unusual behaviors that have not been easily detected in the past, while also filtering out behaviors that can cause false detections not indicative of actual test fraud.
Consider Blended Security Initiatives
It is often the case that integrated security initiatives complement each other in such a way that a “bundle” of security services is more powerful than the sum of independently implemented security solutions. Have you considered which combination of security initiatives will fortify your testing program against test fraud on the front end (e.g., protection, deterrence, and prevention), supplemented by detection techniques on the back end so that test fraud is found when it does occur? For example, data forensics can supplement web monitoring and exam design and development activities that find and design countermeasures when braindumps of your content are found on the Internet.
As you contemplate the above points, we invite you to partner with us. With over ten years of actively seeking and implementing state-of-the-art test security solutions, Caveon has a lot to offer, and we are not resting on our laurels. While it would be nice if cheaters were as obtuse and easy to catch as the would-be police officer in the story above, we are well aware that this is not the norm and that fraudsters are always looking for new ways to cheat the system. Rest assured that we are continuously reviewing and innovating best practices and solutions to help our clients strengthen test security and administer tests fairly.