Reality Bytes: Preparing for Performance Exams
Times are changing, and not just in regard to technology. Certification exams also are evolving. Performance-based tests that look not at whether a candidate can memorize and recall facts, but how well the person can replicate actual tasks in a real-world environment, are replacing traditional exams.
The shift has occurred partly because employers increasingly demand proof that certification holders are not “paper tigers.” Today’s employers need to know that the candidates they examine for contract and full-time employment have the right skills and are capable of executing a job. Accordingly, preparation for these more intense and rigorous performance- or competency-based exams requires more effort and more time to study.
What Do I Need to Pass This Test?
“The world of performance-based exams and performance-based certifications is entirely different from the knowledge IT certifications of the past,” said Dan Veitkus, global vice president of training services, Novell Inc. “Competency-based exams require a demonstration of competency far beyond what classic IT exams of the past have required. Our customers continue to tell us and research has shown consistently over the years that certification pursuers prefer to use multiple methods to prepare for an exam and on average will use two different types of media or services in preparation for their certification.”
Veitkus said to study effectively for performance-based exams, candidates might need to mesh a combination of instructor-led courses, online labs, self-study kits, mentoring or even authorized books from the major vendors, depending on the certification. However, one of the best ways to prepare for a performance-based exam is to go through an authorized, instructor-led course and lab. Novell provides its students with all of the software and labs they require to prepare for and pass Practicum certification exams. The company also prescribes general hardware requirements, which satisfy 95 percent of the audience. “Once you enter the realm of performance-based, it is critically important that students and certification seekers have access to true subject-matter expertise in the form of an instructor-led, instructor-mentored or some may say a subject-matter-expert-mentored learning experience. We feel that’s the most important thing, providing that access not only to the labs, equipment and the software, but to a true subject-matter-expert mentor in the form of a trainer or an online mentor. That’s key.”
“To me it’s clear that they just have to get lots of hands-on experience with the actual application, the real hardware, whatever it is they’re preparing for,” said Wallace Judd, executive director of The Performance Testing Council. “They need to feel comfortable with that particular interface. Real equipment, real software, the real deal. That’s probably the best preparation they can do.”
Michael Reid, manager of the Cisco Certified Internetwork Expert (CCIE) program at Cisco Systems Inc., agrees. “When it comes to performance exams—we see this with the CCIE specifically because the lab is so strongly performance-based—the strongest indicator of success is how much time a candidate can actually spend configuring real equipment,” Reid said. “Whether that’s in a structured environment like a boot-camp type of thing or self-paced practice, we find both are useful. The key thing is how much time they spend with the real gear. When it comes to the CCIE lab component, book learning is useful but mostly as background. If somebody only touches the book, they will probably score less than 20 percent on the exam, and that’s generous. The reason for that is that in a performance test you are actually testing their hands-on ability. If they don’t practice the hands-on ability, they simply will not perform at all.”
Cisco’s new CCIE Assessor Lab was deliberately designed to provide certification seekers with a workable platform for practicing performance-based tasks. The lab offers four-hour assessments featuring different scenarios so students can try to master different topics from the test. “We’ve produced a four-hour, live-equipment-based practice lab that people can take from anywhere,” Reid said. “The scenarios are written by the same people who write the test, so it looks and feels like the CCIE exam only in a shorter form. We know that customers have a hard time getting equipment, and this is an option for them that also has scenarios that are very similar to the test. Even boot camps produced by training partners and vendors probably won’t look like our test because they’re done independently from the CCIE program.”
Microsoft provides a variety of preparation tools based on a certification seeker’s learning style so those pursuing performance-based certification can choose the best study fit. “We have books, ILT (instructor-led training). We also work with practice test providers to provide additional information regarding our exams and how to prepare for them,” said Keith Loeber, group product manager for certification, Microsoft Learning. “Specifically for performance-based testing, ILT obviously offers the labs and classrooms that give you hands-on experience with the product, which we think is the best way to learn and study for the performance-based exams. Both of our preferred practice test providers, Self Test Software and MeasureUp, have either already incorporated simulation technology within the practice exams or are currently in the process of doing so. For e-learning we also have an offering of v-labs (virtual labs).”
“When you learned to drive a car, you didn’t just read the book. You got behind the wheel and you practiced, and most of us don’t have our own car to practice on. Lots of us took classes, driver’s ed, or you went out and purchased that, and we found people who could mentor us,” said Marc Vaglio-Laurin, manager of certification test development, SAS. “Most of us pursued the quest to learn how to drive pretty seriously. IT performance-based tests are no different. You have to have seat time with the technology. You have to have experience driving. It doesn’t have to be with a live system. There are lots of training classes out there where they use simulators. A number of vendors including SAS offer learning editions of their software where it’s real technology, but it’s designed for individuals to learn on. You get data, exercises and instruction on how to do different procedures.”
Vaglio-Laurin said it can be advantageous to study for performance-based exams in a community college setting rather than an authorized vendor lab because quarter- or semester-long college courses offer a longer period in which to study, practice and acquaint oneself with performance-based tasks. Further, community colleges often give access to training or computer labs at all hours of the day and night—ample opportunity to practice on real equipment, which is critical to pass a performance-based exam. A study buddy can be helpful too. “Find someone who is interested in taking the exam, has taken the exam or is maybe at a similar level in terms of experience, and challenge each other,” Vaglio-Laurin said. “Say, ‘I went into the online system and I took away this person’s access privileges. You need to go in and restore that.’”
Obtain the guidelines or objectives for your particular exam and practice those specific tasks. Those tasks are vetted by certification vendors based on real job-task analyses, as is the case with Microsoft. “We do a job-task analysis and base the exam on that analysis so that we’re validating skills that should be or will be used as they get into the job,” Loeber said. “Then we have people who are already in those jobs validate those skills for us and help us write those exams.”
“A significant amount of investment and sacrifice has been made to elevate the level of