This feature first appeared in the Spring 2017 issue of Certification Magazine. Click here to get your own print or digital copy.

‍

"Which of the following is the correct pin mapping for a rollover cable?"‍

It's been a long time since I last created any type of network cable, and the four multiple-choice responses given for this exam question all look like potential correct answers. There's always a 25 percent chance with any four-response, single-answer, multiple-choice question.

But I want to improve my odds. I open a browser and type "rollover cable pin mapping" into the search field. In less than 60 seconds, I know the answer to the question is D) 1<>8, 2<>7, 3<>6, 4<>5.

Traditionally, this is where I would fill in the (D) circle on my answer sheet with a No. 2 pencil, being careful not to stray outside the circular line. The exam I'm taking is on a computer screen, of course. It still follows the same conventions, however, employed by the first standardized education exams. The ones created in the 1920s.‍

How did we get here?‍

The creator of the "fill in the circle" multiple-choice exam was Benjamin D. Wood, an education expert who did his formative work on standardized testing while at Columbia University. Wood was also instrumental in the development of the IBM 805 Test Scoring Machine, an automated exam checker which could read the marks made by "electrographic pencils." The IBM Test Scoring Machine made its commercial debut in 1937.

The root of Wood's trust in the validity of multiple-choice exams was his belief that thinking is based on the knowledge of facts. This premise may have held merit 90 years ago, but it is arguably not the case today. We currently live in a search-first world where the recall of information has been relegated to a supercomputer we carry in our pockets.

Or sometimes to a small home appliance that responds to our voice commands.‍

"You have been asked to implement Client Hyper-V on a Windows 10 Professional workstation. You have confirmed that the workstation meets the necessary hardware prerequisites. What is the simplest way to implement Client Hyper-V on this workstation? Select the best answer."‍

This exam question comes with five possible answers. I am able to eliminate two of them right off the bat, which still leaves me with just a 1-in-3 chance of answering correctly. I haven't actually implemented Client Hyper-V on a Windows 10 Pro machine before. But I'm fairly certain someone out there has.

My first search ("Windows 10 Hyper-V install") doesn't give me anything promising on the first page of results. I expand my search string a little ("Windows 10 install client Hyper-V") and get a top hit straight from the horse's mouth: the Microsoft MSDN website.

I read for about 30 seconds, and then choose the correct answer: Open Control Panel > Programs and Features > Turn Windows Features on or off.

Total amount of time to find the relevant information: about two minutes.‍

If not multiple choice, then what?

‍IT certification vendors have been tinkering with exam question formats for years, trying to find ways to combine the automated convenience of standardized testing with more dynamic, skill-based questions.

Some vendors have implemented interactive simulations of varying complexity into their exams, where candidates have to click on simulated menus or type the required information into a field. Another popular convention is the "drag-and-drop" question where candidates must select items from a jumbled list and place them in a specific order based on the question's desired outcome.

The experts who develop IT certification exams know they must maintain a fine balancing act. If an exam is too easy, or makes itself open to cheating by relying too heavily on rote memorization, then the value assigned by the industry to the related certification will take a hit.

If an exam is insanely difficult, on the other hand, because it delves into the most obscure corners of a given technology — I'm looking at you, Microsoft — then candidates will be frustrated and rightfully complain that the exam doesn't reflect the majority of use cases encountered in the working world. All of this exam tweaking doesn't begin to address the thought processing of today's search-first generation — to say nothing of the changing value of memorized facts in the modern IT workplace.

Put another way: If a modern, moderately-skilled IT professional were able to access the Internet during a certain certification exam, how much higher a score would they be able to achieve?

Would they be able to achieve more advanced credentials with a search engine at their disposal? And, would passing an exam in this manner make the result any less valid, or less valuable to a modern employer?

The crux of the issue is the waning relevance of IT certification exams primarily based on memorized facts. While certification vendors can add tricky language, drag-and-drop questions, and animated simulations to their exams, this won't change that the core of standardized testing is information storage and retrieval — an activity that has been delegated to our personal technology.

And, as we continue to offload information wrangling to our devices, it is not difficult to see what the next critical issue for IT certification is going to be in the not-too-distant future.‍

"You decide to create a new VLAN with ID 4000 in a Cisco UCS domain and using Cisco Nexus 5000 Series Switches. Reviewing the network documentation, you confirm that the VLAN is not in use but decide to change the VLAN ID to 3900. What could be the reason for not using VLAN 4000?"‍

This multiple-choice exam question may as well be written in Greek. I haven't done any work with Cisco equipment in my lifetime, and the context of the question doesn't lend itself to finding the answer through a search engine.

Four possible answers. Eenie-meenie-miney-mo?

"Hey, Alfred," I say. Alfred is my personal AI-powered assistant. I can access Alfred from anywhere via my phone.

"Here, sir," Alfred says politely.

"Answer this exam question for me," I say. I look at the question, and my eyeglasses scan the text and send the information to my phone.

"The correct answer is "C." Do you want a full summary of the question and answer?" Alfred asks.

"Bookmark it for later review," I say.

"Very good, sir," Alfred says contentedly.‍‍

‍From PDA to PDC (Personal Digital Coworker)‍

If you think the above scenario sounds far-fetched, then you haven't been keeping track of the pace of development of machine learning, artificial intelligence, and human-computer interaction.

We will reach a point — easily within the next 10 years, based on the current growth of technology — where an IT technician will ask a device, "How do I — ," and describe any hardware or software activity performed in the enterprise.

Then the device will spell out the exact steps, in the exact order, necessary to complete the activity. What the technician sees and hears will also be seen and heard by the device, which will further automate the process.

The development of AI, machine learning, and robotics is going to permanently change the IT workplace. The skill sets required to perform network administration, software development and testing, and information security will move from knowledge-based to tool-based — the tool, in this case, being an intelligent machine.

If the search engines of 2017 can already give the average person a decent chance of passing a certification exam, then future technology will make standardized tests look prehistoric. This future doesn't mean the end of IT certification exams — but it will require a necessary change in order to make exams relevant to the industry.

It is time for certification vendors to begin moving away from standardized testing. It's time to come up with a plan for the future of IT credentials.

‍