Your Tests Are in Good Hands

Posted on
Share on Google+Share on LinkedInShare on FacebookShare on RedditTweet about this on TwitterEmail this to someone

When I go to a doctor, I feel much better knowing that he or she is an expert at curing what ails me. Most medical doctors specialize in specific fields, such as internal medicine, facial plastic surgery or dermatology. For example, my brother-in-law is a physiatrist: a medical doctor who specializes in physical medicine and rehabilitation. When he first announced he was going to become a physiatrist, I hadn’t heard of it before and wasn’t sure what he would be doing. Since then, he has explained it to me several times—and has even worked a bit on my sore left shoulder.

I have the same respect for chefs, mechanics, real estate agents, pilots, gardeners, pharmacists and most other professions. When another person benefits my life, or that of my family and friends, I appreciate the fact that they are well-trained, experienced and know what they are doing. It’s even better if they come recommended by someone I trust: The professional then gains my trust as well.

Your information technology certification tests are built by experts like those I described. These experts are called psychometricians, and they affect your life directly by creating the exams you need to take. Psychometricians are specifically trained to design exams, write good questions, score tests properly, determine the necessary passing scores and more. Some of these experts have years of computerized exam experience, and can use technology to measure your knowledge and skills better.

I recently received a phone call from an individual who is in charge of a new certification program in a non-IT field. He and his colleagues had written several hundred test questions and wanted me to review them and make sure they were appropriate. His plan was to create several 200-question exams and start administering them as soon as possible. After speaking for a few minutes, it was clear that as a psychometrician, I could be helpful by lending my expertise. Several suggestions include:



  • Could the exam be more efficient? Right off the bat, I could tell him that 200 questions is too long for a certification test. As a certification candidate, I’m sure you’d rather not answer more questions than you need to.
  • Will the exam be computerized? If so, some of the questions need to be improved.
  • Has he planned a beta, or pilot test, to find out if the questions will perform well? How will that beta operate? Who will do the statistical analyses and report the results?
  • How will the exam be scored? Will the questions simply be right or wrong, or will partial credit be given?
  • What score will be needed to pass? There are specific standards for how this needs to be done. Sometimes the passing score is set arbitrarily, as in “80 percent to pass sounds pretty good to me.” A psychometrican can set a passing score that you, as a competent candidate, will be able to use to separate yourself from those who are less competent.
  • What about security? A psychometrician will ensure that the questions are well-protected so that the exam score actually represents a candidate’s true knowledge and ability.


These are only a few of the areas in which psychometricians are trained—and each one affects your exam preparation and completion, and your use of the score that results after taking an exam. Psychometricians go through extensive training. Usually, they possess a master’s or doctorate degree in psychology, education or a related field, with an emphasis on measurement. This means that the psychometrician takes a great number of testing and statistical courses, so they understand the science behind your certification exams.

But the proof is in the pudding, as they say, and the exams that are created should accurately measure your knowledge and skills. As a test-taker and certification candidate, my guess is that you can tell the difference between a good test and a lousy one. You can tell if questions are poorly written or ambiguous, or are measuring an unimportant or irrelevant skill. Likewise, you can easily judge, after finishing a test, whether that test has fairly and accurately measured what you know and what you can do.

With psychometricians in the background and on your side, the exam should be as good as you are.

David Foster, Ph.D., is president of Caveon and is a member of the International Test Commission, as well as several measurement industry boards. He can be reached at


Share on Google+Share on LinkedInShare on FacebookShare on RedditTweet about this on TwitterEmail this to someone


Posted in Archive|