Healthcare Case Study
Exam Data Forensics in Healthcare Case Study
Caveon Case Study | Healthcare – The Federation of State Boards of Physical Therapy
Caveon Data Forensics™ Helps FSBPT Provide Fair Testing Environment
In its effort to protect the public by providing service and leadership that promote safe and competent physical therapy practice the Federation of State Boards of Physical Therapy (FSBPT) has developed the National Physical Therapy Examinations (NPTE) for physical therapists and physical therapist assistants. Passing the NPTE is required for licensure to practice in every state in the U.S. and the District of Columbia.
“Caveon’s work was invaluable in helping us respond to this very serious and damaging security breach.”
The FSBPT was forced to take legal action against the test prep centers but even while doing so was faced with a perplexing dilemma. The damage had already been done. While the FSBPT had to assume that a number of the examinees who had participated in the Philippine test centers had gained an unfair advantage through pre-knowledge of the test content, there was no way of determining which examinees exactly had gained an advantage. Forcing everyone to retake the exam would unfairly place the burden of proof on legitimate test takers and be very costly in terms of lost income due to delayed licensing. Doing nothing to validate the test scores, on the other hand, would undermine the integrity of the entire testing program. How could the FSBPT protect its own reputation as well as that of prospective physical therapists?
To remove any bias in the investigation Caveon asked the FSBPT for the test results from all candidates and not just candidates that potentially participated in the test preparation course. Caveon used three separate statistical indices to identify aberrant tests. The likelihood that a test would be aberrant on any one of these indices is 1 in ten thousand—the likelihood that any one test would be aberrant on all three is more than 1 in a million. After review of the entire data set Caveon determined that 20 of the tests were statistically inconsistent. All 20 were anomalous on all 3 of the indices Caveon used to identify irregularities. And all 20 of the examinees had access to the implicated test preparation centers.
Caveon’s work with the FSBPT did not end with the invalidation of these test scores. Understanding the real risk from new and innovative threats to test security the FSBPT felt it needed to continue working with Caveon, a company they trusted which could provide new and innovative solutions to their test security issues.
“Caveon’s work was invaluable in helping us respond to this very serious and damaging security breach. They were both responsive and helpful in developing and implementing the forensic analysis plan,” said Mark Lane, Vice President at FSBPT.