Doing Something About Test Security
As you may recall, test security—or more accurately, the lack of it—is one of my favorite topics for this column. For almost two decades I’ve watched cheating, in all of its forms, grow like a cancer in IT certification programs, stimulated by the new computerized tests and Internet technology. Testing in other certification and licensure markets has been hit hard as well. Recently, Educational Testing Service (ETS) quit giving its computerized version of the Graduate Record Examination (GRE) in certain areas of the world because it had been compromised. Instead, ETS now gives a traditional paper-and-pencil test available only on specific dates. Wherever high-stakes tests are given, including IT certification, that cancer has cost hundreds of millions of dollars in damage each year and, in your case, has also reduced the value of and confidence in IT certifications.
People cheat by buying questions, using cheat sheets, paying someone to take a test for them, memorizing and selling questions to others and in dozens of other ways. In the United States, we seem to be becoming more and more a nation of cheaters, unable to tell the difference between what is right and what is wrong. And there seem to be more than enough justifications to choose from:
- Cheating is not that bad.
- I really need the job (or to get into school or to get a good grade).
- I have to beat the others.
- I don’t have time to study or get experience.
- I’ll be at a disadvantage because everyone else is cheating.
- Cheating is easier than anything else.
- The tests are unfair.
- The tests are too hard.
- The tests are too easy.
- The tests are meaningless.
- The people who make the tests only care about themselves or the money the tests bring.
- No one is trying very hard to stop the cheating.
With that many reasons to cheat, how can anyone resist the temptation? I’m surprised any test is taken legitimately. There must be fewer good reasons not to cheat. And should we all feel a bit sorry for those who study, work hard and prepare honestly? What is all that effort for? To see the value of their new credential disappear? Sorry for the sarcasm, but when is reason going to win out here?
You know what I have heard lately (and it scares me to death)? That IT certification programs don’t really care that much about the cheating that is going on. That they are not important certification programs anyway—not like the automobile mechanics, security people at airports or real estate agents. I’ve heard that IT programs don’t really care whether those who pass are actually competent. I’ve heard also that efforts to improve the tests or test security won’t work because IT certification programs aren’t willing to pay for them.
Of course, none of these are true, but they may reflect a growing crisis of confidence in IT certification programs that we all need to address. And please don’t misunderstand me. I’m not saying that we are too late. We are experiencing some very natural problems because we have left some “holes” in the system over the years. This is not unlike the recent security problems experienced by Microsoft’s Windows, allowing viruses into our systems. And like Microsoft’s quick reaction, we need to find solutions and implement them.
It’s not a total solution to these problems, but I’ve begun a new company called Caveon. It is a beginning effort to turn the tide, to start to reclaim some of what we have lost. Caveon helps organizations protect their existing tests and questions, erecting barriers to unauthorized access and sharing of copyrighted materials, using sophisticated statistical and Web patrolling tools to determine the who, what, when, where and how of cheating, and providing the support to track down the cheaters and put them away for good.
My apologies in advance, but you’ll probably see a greater proportion of security-related columns in the coming months and years. There is no need to worry, however. Being my first passion, I’ll still write about tests and questions, scores and psychometrics, validity and reliability, and new technologies for testing—all the topics you’ve come to love. I just won’t write about them as often.
David Foster, Ph.D., is president of Caveon (www.caveon.com) and is a member of the International Test Commission, as well as several measurement industry boards.