Written by John Fremer, President, Caveon Consulting Services
November 18, 2014
I have been asked many times “Can you really make a cheat-proof test?” My view is that a nearly cheat-proof test is theoretically possible, but practically improbable. I believe three protective features, if implemented, could advance the state-of-the-art and make a cheat-proof test a reality:
- Present unique test items each time for every test taker.
- Remove incentives for proctors to help test takers gain an unfair advantage.
- Perform data forensics to ensure that scores are valid before reporting and/or finalizing them.
The extent to which “real world” programs implement these and other protective testing program features very much influences how much cheating they will have to deal with.
Unique Test Items for Every Test Taker – Stealing of test items is a very serious problem for high stakes testing programs, especially if they test internationally and if their tests are offered across a wide span of time. Test theft begins almost immediately and continues until the item pool is completely compromised. Test form validity falls as a consequence. Because of this, test Program managers “refresh” or replace their test material frequently. A very appealing alternative, when permitted by the test content is to provide entirely new and unique test items for every test taker on every testing occasion. The items are generated automatically by item and test creation algorithms. This means no items will be available in advance to be stolen and the items will never be reused so an unfair advantage cannot be obtained through theft.
No Incentive for Proctors to Help Test Takers gain an Unfair Advantage – Most cheating occurs during the testing session. One of our most effective cheating prevention strategies is having a well-trained and conscientious set of test proctors. Many first class test proctoring practices, when diligently employed, successfully deter and detect attempts to cheat. It is essential that test proctors administer tests or monitor the process very carefully. Testing programs need to confirm that test proctors do not have a stake in test taker success. Even though proctors may wish students well, they should have no “personal stake” in testing. The closer a testing program can come to this goal, the easier it will be to prevent and deter cheating.
Data Forensics Performed before Scores are Reported and/or Finalized – Once test scores are reported and have contributed to decisions, rectifying a major testing irregularity becomes problematical. If you cancel scores or decisions based on scores such as admissions or certification, strong resistance can be expected. Delaying the reporting on the front end and possibly requesting clarifying information from test takers is less troublesome and gives the program a great deal of flexibility. Building data forensics directly into the scoring and quality control process can greatly reduce cheating in the program. Test takers who have not yet received scores can be asked to provide additional information that would help to evaluate their test results. When feasible, test takers may be asked to take the test again to confirm the validity of their scores.
If the above ideal protective measures are implemented, we can come close to having exams where we can have great faith in the validity of the results. In other words, the scores will be trustworthy and valid. And, the tests will be practically cheat-proof.