Certification exams: Time to move on from multiple choice

This feature first appeared in the Spring 2014 issue of Certification Magazine.

Multiple-choice certification exams are a dime a dozen, and they really aren’t that great. This popular form of assessment has a lengthy list of advantages for exam creators, but the act of filling in a bubble for A, B, C, or D doesn’t provide proof of the candidate’s true knowledge, skill and ability. This is especially true for IT-oriented certification exams. An unprepared or unscrupulous candidate can essentially cheat his way to a successful score through specific methods, including:

Cramming: Just like many of us used to do in college (or high school), test candidates can cram certification information into the short-term memory. It only has to stay there for as long as it takes to get within arm’s reach of an approved scratch pad at the testing center.

Brain Dumps: The cheat sheets are out there. Even only mildly savvy searchers can find illegal online dumps of official exam questions. Memorizing the correct answers — without knowing why any of them are correct — is a snap.

Process of Elimination: Multiple-choice tests so readily lend themselves to guessing that even people who aren’t encouraging test takers to bend the rules repeat the formula: One or two answers to every question are generally easily identified distractions. Rule those out and many questions practically answer themselves.

Incorporating other question types, such as true/false, drag-and-drop, fill-in-theblank, hotspot, and others, can create an instrument slightly more effective than a straight multiple-choice test. Even more complex pen-and-paper (or mouse-and-radio-button) exams, however, fall short of assessing what a candidate can do with his knowledge. Not in every instance, of course. A drag-and-drop question can accurately assess a candidate’s knowledge of the sequencing of a process. A fill-in-the-blank question that requires typing a command and associated switch comes closer to performing the task in a live environment then choosing A, B, or C. Yet still, this is not sufficient in providing proof of the candidate’s true knowledge, skill and ability to perform business-critical activities.

To assess a candidate’s true knowledge, skill and ability, a significantly different approach is required for IT-oriented certification exams. In other words, the candidate should prove what she can do not only to herself, but to her current or future employer.

The next generation of IT-oriented certifications exams, like those developed and deployed by TestOut Corp., provide simulated IT environments that closely resemble those found in enterprise networks, small offices/home offices, and branch offices. Within these simulated environments, the candidate is assessed on his ability to perform critical skills within a particular IT role and function. By simulating the IT environments, the assessment proves how job-ready a candidate is regarding specific IT functions and tasks, including:

— Implementing security configurations on an array of common hardware and software vendors, including Microsoft, Cisco, Apple, etc.

— Configuring and troubleshooting client systems, such as Windows 7, Windows 8, Linux, and iOS operating systems.

— Building and troubleshooting enterprise networks that include Power over Ethernet (PoE) switches, fault-tolerant routers, physical and virtual servers, rack-mounted power supplies, Intrusion Detection Systems/Intrusion Prevention Systems (IPS/IDS), etc.

— Configuring Cisco Routers and Switches for a Local Area Network (LAN) and a Wide Area Network (WAN) implementation for multi-campus situations, as well as branch office connections.

— Building, maintaining, and troubleshooting end-user workstations for production-oriented businesses environments whether simple or enterprise-class.

Further, the job-ready skill assessment should come as close as possible to what is required in the day-to-day activities of specific IT roles and functions. This includes major and recurring tasks, crucial functions, and even the mundane routines that are required by a job role. With regards to specific IT jobs and functions, TestOut’s Pro Certification line targets and simulates the day-to-day business-critical activities of several roles, including the following:

— Computer Support Technician, Help-Desk Specialist

— Network Technician, Network Administrator, LAN/WAN administrator

— Computer and Network Security Specialist, Network Security Analyst

— Server Engineers and Administrator

In the TestOut Client Pro certification exam, for example, a candidate is assessed on his ability to perform real-world tasks on Windows 7 and Windows 8 operating systems. The real-world tasks focus on configuring and troubleshooting the operating systems’ features and settings, which range from installation and storage to networking and security. To illustrate, the image on this page is a sample question that a candidate could see during the certification exam. The candidate is presented with a simulated operating system (in this case, the Windows 7 operating system), and a set of tasks that they must complete before moving on to the next question in the exam. In this example, the candidate is required to complete three tasks (which are independent from each other):

Standard multiple choice exams are bad for the business of certification.

— Configuring a wireless connection

— Creating a VHD file

— Enabling and configuring remote desktop options

To successfully answer the question, the candidate uses the simulated operating system to complete each task in the exact same manner as they would if they were using a real operating system. All of the necessary icons, diagrams, tools and utilities in the certification exam are available for the candidate to use as they complete the tasks. (The first task is shown in the illustration.)

When you think about it, computer support technicians, network administrators and client/server security specialists are not answering multiple-choice questions in their day-to-day occupations. You don’t get to guess between options when maintaining computers, troubleshooting networking issues, or securing end-user computers and enterprise-class servers. In the real world, regular employees are actually performing the business-critical tasks described in certification exams, proving their value to their employers by what they do. Shouldn’t a certification exam prove each test candidate’s ability to perform those same critical skills? It’s time to stop settling for multiple choice guesswork and move up to the next generation of certification exams.

Like what you see? Share it.Share on Google+Share on LinkedInShare on FacebookShare on RedditTweet about this on TwitterEmail this to someone
Greg Shaffer

ABOUT THE AUTHOR

Greg Shaffer is a product manager and former senior instructional designer at TestOut Corporation. He holds several CompTIA and Cisco certifications, and earned his master’s degree in Instructional Technology from Utah State University.

Posted in Certification|

Comment: