As IT credentialing programs get more complex and rigorous, many certification managers are beginning to contemplate using performance-based testing to evaluate candidates, if they haven’t taken the plunge already. This is a positive development for these programs and for the IT industry generally, said Julie Timmcke, executive director of the Performance Testing Council (PTC).
“In past years, as it kind of ramped up and got crazy, everybody and their brother started getting into (certification),” Timmcke said. “Then there were some questions about how valuable and viable those certifications actually were. There were a lot of certifications out there, but what did they mean? What did they prove? I think that’s part of where this group came from. They were inspired by the need to advocate and pursue performance testing. Our tagline is ‘Testing by Doing.’ We believe that performance testing is a far superior way to evaluate and assess than a lot of the other standard testing methods that are out there.”
To promote this methodology, the PTC hosts events such as its upcoming Summit, which will be in Banff, Canada next month. The organization also researches some of the most successful practitioners of performance-based testing and captured both their best practices and caveats. “There are a lot of obstacles to performance testing,” Timmcke said. “That’s really the mission of the Performance Testing Council: to work out those issues and help people know what steps to take and what steps not to take.”
Performance-based exams are valued for their reliability, as well as the peace of mind they give to customers and employers who are served by those test takers, Timmcke said. “If you’re on a plane and the pilot says, ‘Hey folks, last week I passed my multiple-choice test on how to land. Here we go!’ you’re not going to feel too comfortable about that. The same would be true if you’re hiring someone to manage your system security.”
Timmcke pointed out that many of the more respected high-level IT certifications often employ performance-based testing to some extent. “Those are very complex jobs. The people who are doing that kind of work need to be very, very skilled. They have to have a lot of technical smarts. By performance testing them, you’re really assuring that. I think that the respectability that some of those certifications have comes from the fact they have performance-based tests.”
Yet a credential doesn’t have to be on par with, say, the CCIE in order to use performance-based exams, she added. “The principles of performance testing can apply to any level of certification. It doesn’t have to be the most intense job skill you’re trying to test. It’s much more doable than it ever was before. We feel that it’s more valuable because it’s not just testing the knowledge. It’s testing the ability to perform and do.”