Connection 2013 Issue 3 Technology That Works
People Who Care
Myths of Item Writing Jennifer Reed Marketing Manager
H The most important predictor for a successful item writer is their willingness to serve and their love of the profession.
igh quality items (i.e., test questions) are the cornerstone of any professionally sound, legally defensible credentialing examination. Many things go into creating these items: matching the item to test specifications, avoiding bias, ensuring only one best answer, remaining relevant to practice, etc. For many organizations, an item writing workshop will produce hundreds of draft items, and all of them will not make their way to a live test form. Item writers are typically experts in their field, but they can’t be expected to instinctively understand the nuances of item writing. In fact, some item writers initially have misconceptions about the process or what makes a good item. After more than 30 years working with item writers across a wide range of professions, we have encountered our share of item writing myths.
All Items Should be Difficult Some item writers feel it is their job to stump the candidate – an item is only good if candidates can’t get it correct. Wrong! If every item on the test is too difficult, the test may not appropriately differentiate the knowledge needed to identify competence. While the majority of the items should be considered to be moderately difficult, a range of difficulties (from fairly easy to fairly
difficult) makes the best test. The myth that “we need to include a few items that everyone can get right” is also detrimental to measurement precision. Sometimes item writers have trouble judging the level of difficulty of their items, so when writing items, it is best to not worry too much about the difficulty. Focus on appropriate content. Difficulty will be determined later through pretesting, and the items that are appropriately difficult and differentiate those who have the knowledge from those who do not will be available to compute candidates’ scores.
The More Detailed the Better “Let’s throw in all kinds of detail and see if the candidate can sift through what is important and what isn’t. After all, that is real life.” The reality is candidates only have a set amount of time to take an examination and it is typically most helpful if they can focus on the important issues when responding to items. Item writers should always focus on the question: “What are we trying to test?” We usually aren’t trying to test the candidates’ ability to read and sift through unimportant details; the tasks should be relevant to practice. So, providing enough detail to make an item realistic and ensure the distractors are plausible continued on page 2
New Business
2
www.goAMP.com
A Day in the Life of a Meeting Planner
3
On the Road
4
Stay Connected
4
Published by Applied Measurement Professionals, Inc. © 2013