Caveon Case Study
(View PDF)

Caveon Data Forensics™ Helps Mississippi Department of Education Provide Fair Testing Environment

The Challenge

The Mississippi Department of Education’s (MDE) mission is to ensure that the testing process and environment is fair for all students. Any indication of an unfair advantage signals a red flag to MDE’s team of testing professionals, including Cindy Simmons, the Director of Student Assessment. Although MDE has had a strong test security and auditing system in place for a number of years and school districts were doing a good job of following the state’s test security guidelines, a significant increase in test administrations and limited staff to conduct analyses of possible testing irregularities prompted Simmons and her colleagues to seek an outside vendor with specialized knowledge in data forensics and test security to provide assistance.

“We realized that our teachers and administrators were limited in their capacity to be everywhere at all times to monitor the test administrations,” says Cindy Simmons, Bureau Director for MDE’s Office of Student Assessment. “For example, although cell phone use is clearly prohibited during testing, students were finding ways to text message answers to one another. We needed help to identify how some students were beating the system and, more importantly, to offer increased, effective support to how our schools monitor test security,” Simmons adds.

Additional manpower was a necessity. The MDE administers four high-stakes multiple-choice examinations to approximately 116,000 students annually. MDE needed new techniques and advanced analytical skills to address irregular test taking but, as with many of the nation’s departments of education, Mississippi’s state and district staff were overbooked. Time was of the essence. Graduation was fast approaching and compromised test scores meant some seniors might receive diplomas they didn’t earn. Finally, budget considerations were paramount. Like its neighboring states, Mississippi was spending millions of dollars toward crucial Hurricane Katrina recovery and relief. A creative, cost-effective solution was the only answer.

The Solution

MDE found such a solution in Caveon Test Security. Through Harcourt Assessment, which holds the contract for the SATP (Subject Area Testing Program), the Department hired Caveon to analyze test data and look for anomalies and response patterns that indicate possible test fraud.

Leveraging their proprietary Data Forensics technology, Caveon conducted a detailed analysis of both the paper/pencil and online tests for the 2006 spring administration. The Caveon team of security experts and statisticians looked at both initial administration and re-administration to spot inconsistencies in the test data. Specifically, they looked for anomalies (deviations from statistical averages) and collusion (students working together) such as identical wrong answers and many erasures on an answer sheet.

The Results

Today Caveon works in conjunction with MDE to set a security risk threshold, and any exam exceeding the threshold is flagged. When districts receive individual student scores, any student with a score that exceeds the security threshold receives a conditional code for a pass/fail status. MDE sends letters to the appropriate districts informing them that a score is in question and that further investigation and explanation regarding the student’s performance is required. Districts then have 15 working days to respond with information that addresses MDE’s specific concerns.

Caveon’s analysis allows MDE to reduce the number of irregularities in score reports and, more importantly, provides the school districts with a higher level of scrutiny on test data. Although many of the districts met current standards of test site security, the information MDE now provides through its partnership with Caveon has prompted districts to re-evaluate in-house procedures and revise them as necessary to support MDE’s mission of providing an equitable test environment.

Caveon’s role in the testing process takes approximately five to seven days, a timeline which Simmons says has not slowed the overall flow and speed of test security. To date, approximately 20 districts have been asked to investigate test results since analysis of the first administration in 2006 was conducted. Testing irregularities continue to be on the decline statewide.

“As educators, our focus is on our students, and a fair test environment allows us to gauge each student’s development,” says Director Simmons. “Our partnership with Caveon Test Security has helped our teachers and administrators better perform their jobs.”

Send this page to a friend.