When Test Prep Goes Too Far
By David Foster
Certification Magazine, 8/2002

Here's just a little bit of testing theory to start with. Don't get bored and bail on me. I'll get back to the interesting stuff soon enough.

Certification test questions are created to measure the skills and knowledge of a job domain. Any single test (called a "form") contains enough questions to measure a representative sample of the domain and to produce a reliable score. Other equivalent forms of the test will do the same thing. There is a good deal of overlap between two equivalent forms of a test, not in using specific questions, but in measuring a broad set of skills.

The implication of the preceding paragraph is this: To do well on any of the test forms that might be selected for delivery, the certification candidate has to know or be experienced with the entire domain. So, what does this have to do with test prep?

There are only two legitimate types of test preparation for computerized certification exams. The first is quality training. To prepare for the test, the smart candidate obtains the best training possible that focuses on the skills and knowledge of the job domain covered by the test. (Of course, this strategy also prepares him or her to do well on the job, a significant "added" value of quality training.) Experience with the technology is usually helpful as well. When thoroughly trained, the candidate is "prepared" to take the test. Barring silly testing mistakes, the candidate should easily pass.

The second type of legitimate test prep is practice tests, which have the sole goal of helping individuals become familiar with the testing format and the types of questions that will be encountered, helping them, for example, reduce testing mistakes that occur because they don't know how to select the correct answer or navigate from question to question. It's most helpful when this type of experience occurs in a technical context similar to that of the actual certification exams.

With training and experience, and familiarity with the computerized testing format and types of questions you will see, you are now fully prepared to take and pass the test.

So, how do I classify test-prep materials that seem to be nothing more than a rapid-fire set of questions covering the content the person will likely see on the certification exam? Sometimes these are presented as computerized mini-tests or quizzes, sometimes simply as a document containing a collection of questions.

Are these efforts simply meant to help the candidate get familiar with the computerized-testing user's interface? I don't think so. So, either they are attempting a type of training, or they are trying to present specific questions obtained from the actual exam (by whatever means). The first approach-that of using test questions as a part of a training program-uses questions that are related to those they might see on the test, but makes no attempt to reproduce actual questions. This approach follows a legitimate training philosophy that answering a bunch of questions about a topic helps you learn about that topic, especially if the answering can be followed by feedback and prescriptions.

The second approach, claiming to have real questions from the actual test, is not training at all and doesn't pretend to be. It is a form of test prep that is illegal (violating copyright laws), unethical (using others' developed materials for business purposes without paying for it) and irresponsible (lowering the value of certification by helping people obtain undeserved certifications). This kind of test prep should be snuffed out, deep-sixed, eradicated and otherwise eliminated. As certification candidates, you should participate in this effort by refusing to spend time or money on organizations that make such claims. Certification programs should make every effort to take legal action against these organizations whenever possible.

But, you ask, "What if an organization makes such lousy exams that the only way to pass them is to get the actual questions?" This is a good question. Let me answer it by asking you a question. What would you do if you went to a restaurant and received bad service and bad food? My guess is that you wouldn't return. And you wouldn't buy their recipes from a disgruntled former employee, either. If a certification program's exams are so bad, the certification program can't be much better. So, you are better off becoming affiliated with a different, higher-quality certification-there are plenty of them out there.