Using a quantitative approach to assessing risk

Written by Dr. Jamie Mulkey, VP, Client Services

September 18, 2017

What exactly is risk? The Business Dictionary defines risk as, the probability or threat of damage, injury, liability, loss, or any other negative occurrence that is caused by external or internal vulnerabilities, and that may be avoided through preemptive action. When we talk about risk in terms of test security, it is the threat of losing carefully-crafted test items, the threat of liability resulting from unqualified individuals performing in a job, the threat of negative consequences for unattained knowledge and skills of students, and lastly, the threat of unintended consequences, such as damage to your testing program’s reputation, because your test has been compromised.

Often, we get preoccupied by these test security threats. Let’s face it, there are probably potential risks to your testing program that stand out for you more than others. You may be plagued with thoughts of unauthorized access into the item banking system, maybe you are concerned with unsanctioned distribution of items over the Internet or through Social Media. Or, maybe you are worried that a test taker passed an exam when they shouldn’t have.

Wouldn’t it be nice if you could quantify these test security concerns to help make decisions about which test security threats to focus on in terms of taking action to prevent them? We thought so too. We are very pleased to introduce the Caveon Risk Assessment Tool. This free-to-use, web-based tool based on Dr. David Foster’s definitions for test cheating and theft threats can help you quantify your test security concerns and assist you in prioritizing the actions to take to mitigate risk to your program.

Here’s how it works:

There are two main groupings of test security threats: cheating and theft. Think of cheating as the things that an individual might do to increase or better their test score. Think of theft as the stealing of items so that other people can increase their test scores. These two categories have multiple examples of activities that describe how threats to test security occur.

Here’s an example of theft:

For each example, you will provide a score for the likelihood (How likely is this event to occur?) and the damage (If the event were to occur, how much would it damage your program?). Our comprehensive algorithm will calculate the likelihood times the damage to give each threat a combined score. When you’re finished, you will have a prioritized list of test security threats. This will help you confirm or rebuke your uncertainties regarding the security of your program and give you a path to go tackle your most salient test security concerns.

The risk assessment tool can also be used in collaboration with several staff at your organization to provide you with valuable insights. You can compare your results to find answers to questions like “Does everyone on the team view the same test security threats as most important?” “Are there differences?” “And, if so, why?”.

The results of conducting this analysis can provide you with a roadmap to help you prevent and mitigate test security threats. Once you’ve put action plans in place to address these threats, you can come back within a year’s time, complete the risk assessment again, and see if those test security priorities have changed.

The new Caveon Risk Assessment Tool is a way for you to quantify test security threats and help you prioritize actions to be taken. We would love for you to try it and let us know what you think!

Jamie Mulkey

Vice President of Client Services, Caveon Test Security