February 4, 2014
By Dennis Maynes, Chief Scientist, Caveon Test Security
The Swiss Army Knife is an amazing object. Packed within a compact design, you might find scissors, pliers, files, magnifying glasses, watches, and even USB drives, in addition to the standard regulation knife blade. The standard assessment is pretty much a single-purpose tool. It is usually designed to measure an individual’s competence as effectively and reliably as possible. But, given today’s complex environment with security compromises and requirements to ensure fairness and non-discrimination, it may be time to redesign your assessments and convert them into multi-purpose instruments.
As I write this, I am specifically thinking about improving test security. But, the concept goes farther than test security. Many assessments are administered to distinct subpopulation groups. Members of these groups may be disadvantaged by the form of some of today’s assessments. Thus, it may be helpful to design into your assessments the ability to discern whether someone has been disadvantaged and not measured fairly.
Returning to the idea of security, there are many security elements that could be designed into your assessments. Some of those elements could add protection for the exam in high-risk environments. Other elements could aid in detection of test security issues. And, it would even be possible to design elements that track where test questions are stolen. It’s merely a matter of being careful, creative, thoughtful, and being willing to take risks. Collectively, I would like to refer to all of these “exam inoculation elements.” By now, you should have realized that many, many things can be designed into exams to allow them to perform their basic function AND extend them so their effectiveness is increased.
We haven’t discovered or invented all of the elements that could be designed into exams for security purposes, but it’s time to seriously think about doing this. Let me list just a few elements which can be designed into exams in order to strengthen exam security.
- Embedded verification items – These items are newly written and newly added to the exam. Because of this, the items can be considered secure. On the other hand, the older items may have been stolen and disclosed. When a test taker has a significantly lower score on the new items as compared with the old items, the embedded verification items can indicate when and where braindump content is being used. Of course, the approach must be designed and implemented very carefully.
- Digital watermarking or steganography – The items are modified on delivery by the computer. The modifications are associated with specific testing sessions. When the information is decoded from recovered, stolen content, the specific testing session where the items were delivered can be identified.
- Stealth republication – Exams that are administered on a continuous basis may be stealthily republished. Data analysis reveals that the pass rates of braindump users decrease and the effects persist.
- Chameleon items – These items have been cloned from compromised items in such a way that the braindump users believe the item is original when in fact it has been refreshed. Braindump users will select incorrect answers and be penalized for using braindump content.
It should be apparent that incorporating security elements into an exam will require some effort and resources. In the same way that the Swiss Army Knife’s design must accommodate its components the exam must be designed to integrate new security elements.
I have presented just a few ideas. Hopefully, I have helped you see the possibilities that exist through careful design of assessments. I believe that assessments can be engineered or designed effectively so that security elements are built into them. If you would like to know more, please attend our session at ATP 2014: Exam Inoculation and Other “Crazy” Ideas Stop Cheaters from Passing Exams.