Questionable Questions

Posted on
Share on Google+Share on LinkedInShare on FacebookShare on RedditTweet about this on TwitterEmail this to someone

I’d like to respond openly to a letter written to Certification Magazine from a concerned certification candidate, about an IT exam he recently took and is preparing to take a second time. He explained in his letter that at his company, keeping his job as an information security officer is completely contingent on passing his certification test on a single try. If he passes, he keeps his job. If he fails, he is gone.

Because the results of the test will be so important, he decided to prepare by taking the test early on his own nickel as a kind of practice run. While I don’t think this kind of practice is effective, his decision is understandable.

After taking the test, his opinion was that “the questions were not worded properly, were convoluted and did not have any type of logic.” Basically, the embattled information security officer was saying that poorly written test questions kept him from passing the test; thus, the test was unfair.

He may be right or wrong—I certainly can’t determine this. But let’s assume that he is right, and that the test has some bad questions. If you’ve read my column in the past, you know that such a thing is not unlikely in some IT certification tests. A few bad questions can make a test unfair.

How would you advise him? What would you do if you were in his position, having to pass a test that seemed unfair?

Here are some options that come to mind:



  • Over-prepare for the test. It is not likely that all of the questions are bad, and over-preparation would increase your chances of obtaining a passing score. The point is that you should study enough to achieve a very high score. Hopefully, even with the unfair influence of the bad questions, you could still achieve a passing score.
  • Find some legitimate practice questions (not those found on a brain-dump site) and test yourself. There are organizations that sell practice questions for almost any IT topic. It’s not a perfect science, but strong performance on the practice tests would give you confidence when the crucial moment arrives. If your performance isn’t good, these tests indicate where you need additional work.
  • Talk to your boss and let him know that the test is difficult to pass because the questions are poorly written, not because the content is difficult. You can cite your first test experience as evidence or use comments about the test found in online forums. Your boss may reconsider the decision to give you only a single chance to pass it.
  • Quickly switch to a competing certification, if there is one. Perhaps you could take a different certification test that satisfies your work requirement.
  • Find a good brain-dump site and get the actual questions from the test. Some people see this action as justified because the test is unfair, but I do not agree. An unfair test is not a good reason to cheat. In fact, there are no good reasons to cheat and use brain-dump sites.
  • Attend a boot camp where the instructors “guarantee” passing, usually by providing more help during the test than is allowed. If you are really lucky, these instructors might actually take the test for you. You can probably tell from the sarcasm that I’m not in favor of this option, either.


I am concerned, though, that a company would make such strict demands of its employees. While the requirement to certify may be the result of federal regulations, it seems punitive, unreasonable and unnecessary.

To help with the long-term improvement of unfair tests, there is something else that can be done. It is important to contact the program manager for the IT certification and provide personal and immediate feedback about the test. Specific information about the test is critical. Instead of just stating that the test or questions were “not worded properly,” give plenty of examples.

While it will take months to fix the test, such feedback is important to most program managers. It encourages them to raise the bar on their exams, to take steps to make sure that great tests are published and that they remain valid as long as they are operational.

I hope that none of you find yourselves in a testing situation as intense as that of our letter writer. But IT certification testing is high-stakes. Jobs are gained and lost. It’s a game that most of us have chosen to play.

David Foster, Ph.D., is president of Caveon ( and is a member of the International Test Commission, as well as several measurement industry boards.


Share on Google+Share on LinkedInShare on FacebookShare on RedditTweet about this on TwitterEmail this to someone


Posted in Archive|