Different Cultures, Different Test Security Priorities

Written by: Steve Addicott, Vice President of Client Services, Caveon Test Security

In the last 3 months, I have presented at two auspicious test industry events in China: the inaugural conference of the Association of Test Publisher’s (ATP) newest affiliate, Asia ATP, and the annual client conference of SeaSkyLand.   I learned two things from these trips. There are hard-working, determined test professionals in China who invest huge blocks of time, effort, and finances to protect their tests and ensure valid test results, same as in the US.  However, just as our cultures vary dramatically, apparently our test security priorities do, too.

China is the birthplace of standardized testing, with Imperial tests dating from the 6th century CE. Along with the long tradition of testing, cheating on tests has also become entrenched. We’ve all heard the horror stories and read salacious news articles depicting widespread, wholesale test fraud in China.

For example, consider the Chinese College Entrance Exam (CCEE).  Depending upon who you talk to, the CCEE may just be THE most high-stakes test program in the world.  Doing well on this exam can, literally, mean the difference between studying to become an engineer, and toiling away the rest of your life on the family’s small farming plot.  Every spring, nearly ten million Chinese teenagers sit for the CCEE, the culmination of years of preparation and study.  Imagine confronting this pressure-cooker at such a tender age—how I perform over the next few hours will set a trajectory for the rest of life that is nearly irreversible.  No wonder why every year there are reports of cheating attempts being detected, and cheaters (and proctors and others) receiving harsh sanctions.

Of course, the significance of the exam is fully recognized by central and provincial government leaders.  In fact, the CCEE is considered a state secret, on par with any other sensitive, classified national asset, and government leaders extoll every effort they can to protect it as such—They spare no expense to ensure that no physical security measure is missed.  But, no statistical analyses of test results are routinely performed. That’s one of the key differences between US and Chinese test practices. I was informed that culturally, government leaders would be hesitant to base important decisions upon the use of statistics.

To protect the CCEE and other high-stakes exams, a primary focus is on physical security during exam administrations.  Indeed, the list of physical security measures involved in the CCEE’s three day test window is extensive:  In many areas, police and/or army units transport test materials;  most of the 100,000+ test venues install metal detectors to prevent any sort of electronics being brought in; surveillance cameras record every step of the test administration; in addition to proctors, some 1,200 disciplinary inspectors monitor test administrations; and, in some areas where test fraud has been suspected in previous years, thousands of police officers may patrol the administrations, too.  Test vendors must comply with rigorous protocols to ensure the integrity of test materials and results during test administrations, including creation of data centers with scanning equipment in every province for the three day administration of the CCEE.

Traditions are highly prized in Chinese culture, so unless there is a tradition of use, change occurs very, very slowly. Because so much attention and scrutiny are focused on the integrity of important Chinese test programs, I think we’ll be seeing attitudes with respect to using statistics to ensure exam security soften a bit in the next few years.  Augmenting current security methods with data forensics can deliver huge security improvements at fractions of the cost of physical measures.  Also, with so much riding on exams like the CCEE, we may see situations where government leaders are compelled to evolve their score validation methods, if only to quell concerns and suspicions of suspected test improprieties.   In any case, it will be interesting to see how vendors and government officials blend the old with the new to better support Chinese test security mandates.

For me, the chance to learn and experience new cultures is a primary appeal of international travel.  I find that sharing ideas, knowledge, and philosophies with like-minded professionals from varying backgrounds spurs both personal and professional growth.  My trips to China, though very long, have proved enlightening, and ultimately, very rewarding.  And who knows…as Chinese test security measures evolve, perhaps Caveon can play a part in bridging some of the differences between East and West!

Caveon

Leave a Reply