The Costs of Compromise
Recently on the History Channel, there was a story about John Walker, the spy who sold U.S. military secrets to the Soviet Union in the 1980s. What struck me during the broadcast was the statement that the U.S. military had to make more than $1 billion in changes in order to restore the former level of security. I was surprised that a single individual causing a breach in security could have such a powerful financial effect.
I tried but couldn’t find the total for the increased costs of security following the Sept. 11 terrorist attacks. As far as I can tell, the changes in security have cost more than we can even imagine. One obvious example is the cost of changes in airline security.
My business interest in security has to do with the security of high-stakes exams, like those you take toward IT certification. The term “compromise” is used to refer to a breach in exam security. But what does such compromise cost? It was suggested to me recently that no one really gets hurt if the questions from an IT exam are exposed on a brain-dump site. The logic is that people who use the site still have to take (and pay for) the exam. No money is lost. Where’s the victim?
That logic, of course, is faulty.
To start with, it costs on average about $500 to create a single test question for a high-stakes IT test. Writing, editing, reviewing and field-testing the questions take time and money. A test usually has about 250 of these questions on average (although any single exam you take may only have 50 to 100). So replacing test items for a single test, a necessary action when compromise has occurred, can cost more than $100,000. If a program’s many tests are compromised, the effort to replace items can easily cost millions.
But wait, that’s not all. In fact, that might not be the big cost. While more difficult to quantify, a program’s reputation takes a hit when the tests are compromised. Confidence in the ability of the test scores to decide who gets certified may go way down. Over the years, an average program’s investment and good-will value may easily be in the tens of millions of dollars. Even a moderate drop in such value is worth millions and costs even more to restore. If you also consider in lost sales to customers because of the loss of confidence or the actions of incompetent brain-dump-assisted certificants, then we may be talking hundreds of millions of dollars. That’s a lot of money. And victims are everywhere, including you, which makes it personal.
Most certification candidates get it. They understand what is at stake and the effect that brain dumps have had. They see that incompetent individuals get certified and get jobs meant for better people. Not only that, but once the honest certification is obtained, it is suddenly worth less.
So let’s talk about cost for a minute. Say you spend $15,000 or more on training and gaining experience, and another $1,500 on tests. Then when you are certified, all of a sudden, people can’t tell if your credential is any different from the person next to you, the person who spent nothing on training, a few bucks for pirated questions and the money for tests. Much of what you spent may appear to be wasted.
It’s not likely that these cheaters and thieves are reading this column. They are not that interested in actually learning anything or contributing to the industry. So the burden is on you to help out, and you already have. I’ve seen many of you participating in forum discussions recently about brain dumps, and for the most part your reactions are critical and persuasive. Rightly, you are not very tolerant of people who publish brain-dump sites or people who use them. Keep it up! Stay involved! Report instances of cheating to the certification program management. Let your friends and colleagues know about the effect brain dumps have on the value of your certification.
And if you think a certification program needs to change its tests or its security in some way, let them know. In my experience of more than seven years building certification programs, we received some thoughtful and accurate criticisms. We took each one seriously, and they led to important changes in the program. 8
David Foster, Ph.D., is president of Caveon (www.caveon.com) and is a member of the International Test Commission, as well as several measurement industry boards.