Written by Jamie Mulkey, Vice President and General Manager, Test Development Services
As was the ritual in the Reese house, Susanna and Declan Reese always put their 5-year-old daughter, Hannah, to bed with a bedtime story. As Susanna finished tonight’s book, she gently kissed her daughter good night.
Out of nowhere, Hannah asked, “Mommy, where do test items come from?” Flustered, Susanna didn’t know what to say. “Where’s this coming from?” she asked. “Have the kids at school been talking about tests?”
Susanna’s husband, Declan then walked into the room.
Susanna turned to her husband, “Hannah wants to know where test items come from.”
Declan looked pensive and furrowed his brow. “Well,” he said, “Test items come from a lot of different places. Why do you ask?”
“There is this kid at school who said that unless an item was developed a certain way, it’s not a real test item.”
Declan raised an eye brow. “Oh, is that so? Well, actually there are a lot of different ways to develop test items. Some organizations use teams of Subject Matter Experts or SMEs to develop items. Some organizations request items from SMEs and then they fine-tune them for delivery. Other organizations have SMEs develop test items independently, and then they review items as a team to ensure their accuracy. There are lots of ways to develop test items—why, I’ll bet there are some we haven’t even invented yet.”
Hannah still looked concerned. “No, I mean, where do they come from?”
“Oh.” said her mother, “that’s a really important point. Good items must come from effective learning objectives.”
“So you mean what a teacher would expect me to learn?” said Hannah.
“Well, actually, it’s more like what your teacher would expect you to be able to know and do, like being able to write both upper case and lower case letters of the alphabet or being able to read a sentence.”
“What if the learning objectives aren’t effective?”
“Then nothing gets measured. It would be like asking you to look at the letter P and describe it, without having to have you ever draw it.”
Hannah’s eyes got very big. “Do you think I’ll grow up to be a good item writer?”
Susanna and Declan both looked at each other. “Oh honey, I’m sure you will. You will use your creativity, good reasoning, and sound judgment to write the best items ever. But, let’s get through Kindergarten first, OK?” said her mother.
“Ok,” said Hannah. And with that, she dropped off to sleep, dreaming about how to draw a perfect letter P.