Innovations and Standards: A Natural Pair

Most of you have taken an exam in a testing center, a classroom or even in your home.  Have you ever wondered what it was like to take a test a hundred or more years ago?  (If so, then you probably have way too much free time.) Sir Francis Galton set up one of the first testing “centers” in modern times at the International Health Exposition in 1884 at South Kensington, England. Galton wanted to collect some data to support Charles Darwin’s (his cousin) theory of evolution so he created a number of tests to administer to exposition visitors.

An area 6 feet by 36 feet was fenced off from the rest of a large hall. Visitors paid 3 cents, entered at one end, completed a series of tasks separated on long tables, and exited at the far end. A staff of three, including Galton, led the visitors through the tests, and furnished them with a card of their results, keeping a duplicate. More than 10,000 people were tested during the Exposition. The testing center, or “laboratory” as it was called then, was so successful that a permanent center was established in the South Kensington Museum. Later, other centers were established at Oxford, Eton, Cambridge and Dublin.

Measurements were taken of weight, height, arm span, breathing capacity, strength, force of blow, judgments of length, reaction time, keenness of sight and hearing, and color discrimination. These were clearly not tests of academic knowledge or job skill; however they were the appropriate tests for Galton.

In one way, the testing process was similar to that used in innovative tests today as the equipment and procedures for some of the individual tests were new. For example, Galton soon discovered the undesirability of asking people to remove their shoes in order to measure their height. So he changed the process to measure the height of the heel and subtracted it from the person’s total height. His device for measuring the speed of a blow with the hand kept breaking so he replaced the weaker parts with parts made of oak. Several people then sprained their wrists. Trouble was experienced on several occasions when “rough persons entered the laboratory who were apparently not altogether sober.” Improvements were made to the tests and to the testing procedures, including test security, throughout the period of the Exposition. At the end, Galton had better tests and gathered enough good data to meet his needs.

Given that there were no testing standards, Galton’s tests were as good as he could produce. The people that participated enjoyed the experience and learned a bit about themselves.

Today the standards for creating a certification test are more formal, they are published, and they provide criteria for judging the quality of a test. A new version of The Standards for Psychological and Educational Testing was just published in July 2014. A copy can be obtained from the website for $69.95.

Creating a test with these standards as the guide will help produce a quality test, one that produces results that are reliable, valid and fair. An important feature of these Standards and other standards efforts is that they set the quality goals but don’t prescribe or dictate the methods. The test designer can use whatever test features or procedures are at hand—or invent them if necessary—in order to best meet the quality standards. Of course, evidence must be provided that those goals were met.

Often we confuse a long-standing test feature, such as the use of multiple-choice questions, as a standard, simply because it has endured so long and will likely be used for decades to come. Presenting test questions in the same order, using test booklets and answer sheets, relying on a proctor, or taking a test at a test center, are other examples. Just as Galton discovered during the exposition, some of these things may not work as well as they used to or may need to be replaced by something better. That’s progress. That’s the way innovation and standards come together. It makes perfect sense (at least to me) that all testing methods, procedures, and processes need to be looked at with a skeptical eye, and should be replaced at the proper time so that a test can continue to meet or exceed the standards.

New item types, better security methods, new test designs, more convenient test administration, mostly fueled by advances in technology, are examples of latter-day innovations that Galton would have appreciated. His novel approach to testing in 1884 launched the modern testing movement. That same spirit of innovation 130 years later, combined with solid standards refined over the decades, makes sure that tests will get even better, that testing will be even more convenient and less expensive, and that tests continue to meet the need of companies, educational institutions, certifying organizations, governments, you and me, and many others.

David Foster

President and CEO, Caveon Test Security

Leave a Reply