Why Aren’t We Still Making History?
“History is a guide to navigation in perilous times. History is who we are and why we are the way we are.”
– David C. McCullough, historian & author
This month’s column is more than a history lesson on certification testing in IT programs. It asks why things aren’t more different today.
Computers were introduced to high-stakes testing when Novell began conducting certification tests via diskettes in 1989. Test questions were written by Novell instructors, compiled into word processing files and mailed to certification candidates in the fledgling program. The candidate would open the files, supply the answers directly in the file, save it and return the diskette. Graders then opened the files and scored the responses. The process took way too long and was difficult to manage. Plus, Novell was a networking company—sending diskettes around the world ran against the grain.
Novell figured that networked computers could be used to deliver tests and serve as the technical backbone of a new certification program. But no one knew exactly how to do it. There were no “experts” on the topic. There were only about 16 computerized testing centers in the entire United States, and none elsewhere. Novell found that a lack of locations and old technology limited its goal to certify hundreds of thousands of people to support a growing array of networking products.
Most of today’s computerized testing innovations came from Novell’s program. Here are just a few:
- First large-scale use of computerized adaptive testing, improving security, keeping test pricing low and making testing simpler (1991).
- Computerized beta tests to try out exams with actual candidates and publish IT tests when needed (1991).
- First global testing center networks, beginning with Drake, then Sylvan Prometric (later Thomson Prometric) and then VUE (starting in 1990 and still going strong today).
- First computerized support for languages other than English (1992), including dual languages available during the same test (1993).
- Use of CDs (new at the time) to support performance testing of a candidate’s ability to use technical libraries and support encyclopedias (1993).
- First large-scale use of simulations of networking software in a certification test to measure actual network installation and administration skills (1995).
- New question types to measure IT skills better: multiple-choice questions that include more than one correct answer (1992), drag-and-drop, hot area (1995) and short-answer free-response (1993).
Now, it seems we’ve hit a dry spell. There has been little innovation recently in any part of the testing process. Testing costs continue to rise, while research efforts are almost nonexistent. Many programs that continue to use paper-and-pencil tests are unwilling to switch to computerized testing for cost and security reasons. The computerization of testing should have resulted in a decreased cost, greater security, greater reach and more convenience. Instead, the opposite happened.
For fun, I re-read my first column for Certification Magazine (October 1999). It talked about the future and the benefits of computerized testing for IT certification. I wrote:
Accurate measurement of knowledge and skills is really the goal of the whole testing effort and of the use of computer technology in testing. If the test can identify the competent individual as efficiently as possible, and help certify that individual, then everyone wins. The competent candidate gets certified properly and quickly. The less competent candidate learns that he or she needs more training or experience, and in some cases finds out the exact areas of strengths and weaknesses. The certifying organization gains confidence in the certified individual and can recommend him or her to customers and hiring departments. And companies all over the world improve their procedures, and their products and services, by relying on the competence of these certified individuals.
As we begin to rely again on technology in testing, you should see the value of technology reflected in better measurement of skills, more respect for IT certifications, more convenience, better security and lower costs. Training should follow suit, helping you prepare more effectively, more quickly and at a lower cost.
I’ve been racking my brain to suggest ways you can help stimulate these new innovations. Maybe you can offer to provide feedback on exams or sit on a program advisory council for free. Certainly, communicating with your program on things you’d like to see can’t hurt. Above all, stay positive—sometimes the tail does wag the dog.
David Foster, Ph.D., is president of Caveon and is a member of the International Test Commission, as well as several measurement industry boards. David can be reached at firstname.lastname@example.org.