Examinations
By David Foster
Using Tests Properly

The other day I went to purchase a common claw hammer at a local hardware store. I noticed that there were many different types of hammers, each designed and built for a specific purpose. Tests are like that too. They are made differently for different purposes. In the IT industry, we are most familiar with certification tests, although there are others. Practice tests are available that help a person prepare for the certification test, and there are tests used to evaluate the effectiveness of training courses. Like the hammers, tests are designed differently from one another. They work differently too. Certification tests are designed to determine who is competent for a particular job and who isn't.

Let me give a couple of examples of the improper use of tests from IT certification testing. A certification test is efficiently designed to provide a single score that can be used to make pass/fail decisions. However, test-takers, particularly those who fail the exam, want the test to provide more details on their strengths and weaknesses. Unfortunately, the test was not designed to do that and has very little to say about such things. Second, training centers somehow want to use the results of certification testing from a number of their students to help evaluate the effectiveness of their training. That "effectiveness" is then used in their marketing programs to attract more students. As a third example, some test-takers will take a certification exam the first time to "test the waters," to get a good look at the kinds of questions and content they'll see next time or to help them decide whether to take a training course or not. They don't expect to pass, making the experience a very expensive form of practice.

The table below provides a categorization for the different purposes of tests. I've identified seven major purposes and contrasted these on a wide variety of characteristics. Whether you are a test-taker or you create tests for certification programs, this table can help remind you of the differences among tests and help you to use test results properly.

> Show Table