This feature first appeared in the Fall 2017 issue of Certification Magazine. Click here to get your own print or digital copy.

I've been involved in IT certification for close to two decades now. Which is a funny thing, because when I first encountered certification, I wanted nothing to do with it. It didn't seem like it would have any impact in the IT world. Now, of course, I see it as a strategic and driving force, both for IT and IT professionals.

I've weighed in and played my hand in several areas of certification, and carved out a path that has been both challenging and rewarding. Not least among those rewards is a fair amount of all-purpose wisdom gleaned from various interactions and experiments that has guided me through various challenges and trouble spots.

One of the most important goals for me when first starting down the road of IT certification was to reduce the amount of time it takes to create and publish an exam from months to weeks. One key step in accomplishing this aim was instituting rapid development workshops.

The need for speed

There are product life cycles that only last 6 months or less, so if it takes that long to publish an exam, you are already out of date. Not only did rapid development workshops actually work, despite the fear of losing people from the field to act as Subject Matter Experts (SMEs), they accelerated the time to market, making it less of a drain on those same SMEs.

There are a couple of other decisions I made as well to help extend the life of exams. One was to create role-based exams instead of product-based ones, a common practice today. Another was to write "truisms."

It can be hard to separate a product or a tool from a role-based exam, but if you focus on objectives that won't become obsolete and write questions that will be true today as well as true tomorrow, your exams have a longer shelf life.

Expert advice needed

Without SMEs, a test simply cannot be written. When looking to create my first exam, the best advice I got, given by one of the few certification program managers in the world at that time, was to entice the participation of experts by conducting test writing at a desirable location like Orlando or Amsterdam. It worked. Simple as that.

The other thing I discovered about working with SMEs is that any time you bring people from the field to the home office you will lose them and their focus to others — so avoid doing that. If you have to use a company office, use a satellite location.

Do things differently

Time to market, as mentioned, is a critical factor in creating a certification exam. With changes being made so frequently to products, different thinking is needed today to meet this demand. This includes alternate delivery vehicles and creative "credentialing."

Launching a low-stakes non-proctored "accreditation" program, then having second thoughts and later launching low-stakes "qualification" programs instead, led me to write a piece titled "Certification vs. Qualification" for Certification Magazine in 2005.

Over the years, this concept has come up regularly in conversation with other IT training and certification leaders. Now that digital badging is further blurring the lines, those ideas have become more relevant than ever.

I've even had some engaging discussions about using the word "qualification" instead of "accreditation." Should "accreditation" be used only for an institution, rather than for individuals? I think so. The key thing here is to protect the value of the word "certification," which is tied to highstakes proctored testing.

Words have meaning, but meaning can also be impacted by time. Whether it is best to use "accreditation" or "qualification" for low-stakes credentialing does not seem as important now. Protecting the value of the word "certification," on the other hand, is still an important issue.

Beat the cheat

Time also changes minds. Take certification testing security, for example. I was formerly an active member of the IT Certification Security Council, devoting many hours to figuring out how to catch test cheaters.

Over a number of years, however, my focus switched to designing exams differently so that cheaters or test cheat sites become irrelevant. The trick is to use scenario-based questions — it's harder to create the questions, but they require critical thinking to answer. Memorization is out of the equation.

There are still guardrails to use, and suspicious behaviors to flag, but exam creators' energy should be spent on crafting and updating question banks. The test cheat sites always fall behind, and eventually get a bad reputation. In the end, the people who pass are the ones who deserve to pass, and that is what counts.

Blowing up betas

I quickly discarded beta exams, early on in my certification career, in favor of other methods of setting pass/ fail benchmarks while still ensuring the validity of the exams. The always urgent "time to market" equation is perhaps the biggest factor here — adding a beta release cycle slows everything down.

There are other problems, such as unnecessary exposure of the test questions your team has worked so hard to create. If you want to bypass a beta, then your best bet is to have a small group take the exam prior to its release. That will let you work through any exam delivery kinks as well as get a sense of how exam candidates are actually performing versus how you expected them to perform.

Forging ahead

Many of the things I've learned have come along in the course of creating and launching four major IT certification programs, and closely consulting on others. My clients have ranged from large IT companies (EMC, Hitachi Data Systems), to a volunteer-driven industry association (Storage Networking Industry Association), to a dynamic open-source Drupal company (Acquia) leading the charge for an entire industry. Each of those assignments had unique challenges.

As the certification chair for CEdMA (Computer Education Management Association) for the past couple of years, I've been able to work first hand with both veterans of, and newcomers to, the IT certification realm. As we've shared our experiences and ideas, a vision of the future of certification has begun to emerge.

Test delivery has evolved over time, and now online proctored testing means that exams can be taken anywhere, anytime — including from the comfort of the exam candidate's personal living space. It used to be that one had to travel to a testing center, and that is still a very viable option. But online training and testing tools have been getting better and better, and I'm now an advocate for this method of delivery.

Old-school thinking says we should only use test centers. The reality is that test centers can be as much a source of cheating as a protection against it, and vendors have to police their affiliates. Online proctored testing is not entirely secure. But improved exam design, as noted above, counteracts cheating as much as anything else.

Performance-based exams are still difficult to scale. It takes more money to offer such exams, and therefore generally costs more money to take them. The end result, however, is a more desirable credential that will hold its value in the marketplace.

The golden maxim

What does it boil done to for me? I like to think of it this way: What matters gets tested, what is tested matters.

Everything springs out of that. It gives clarity and direction on how to proceed given the resources available. It determines what people will need to know and learn. Following this rule makes identifying what training needs to be in place a fairly easy task.

When I worked with companies like EMC and Hitachi, we had the proper focus and resources to create our training programs. For SNIA, the exams set the objectives for independent third-party training providers to create the content. For Acquia, it meant developing a training partner network to provide training. In all cases, everyone could test out.

Companies want the following: product adoption, company loyalty, brand recognition, and cost savings by having exam-validated professionals to leverage. Companies that offer a valued certification program are creating a lifelong loyalty program with individuals.

What is important to individuals? It pretty much comes down to having credentials that enhance their career in some capacity. Passing an exam gives some people more confidence in their abilities. For others, it validates their industry or professional experience and makes them stand out from the pack. It keeps individuals viable in a fast-changing marketplace.

Say what you do, do what you say

The marketplace always will determine the value of a certification. If a company puts together a program, quality is still the key. The players at the beginning of my journey in IT certification were Microsoft, Cisco, Oracle, and CompTIA, with the key testing partners being Prometric, Pearson VUE and a little company called Galton.

The reason I bring up Galton is that this company spun out of the certification efforts at Novell, which is really where IT certification got off the ground. I worked with Galton, but revamped the whole time-to-market schedule. Waiting several months for an exam to come out, even back then, was simply not an option.

The other big concern at the time was fear of people passing an exam with having any hands-on experience. The old expression "paper tiger," meaning something with a formidable appearance but no real teeth, got thrown around a lot.

What I learned in this case is that if you have an exam that is designed to get you going down a career path, then that should be explicitly stated as the purpose of the exam. If you have an exam that is intended to validate practical skills and knowledge, ones that can be used to immediately help a company achieve its goal, then you had better be sure the exam will function as advertised.

Still going

At the end of the day, it comes down to an enlightened company or organization seeing the big picture, valuing quality, and writing exams with scenarios.

If a company or organization does not have goals and realistic numbers in mind, then it may never get off the ground. There are so many certification programs out there today. If your certifications don't deliver tangible value, then you are at a competitive disadvantage.

Individuals and companies that furnish consulting face a similar challenge. You should never advertise skills you don't have. If competitors are certified, however, and you are not, then that may be the determining factor that causes you to be passed over.

When someone takes and passes an exam, it should be exhilarating both for them and for you. If you have created an exam of value, the test taker will know it can help them in their career. If someone fails, then they know what they need to do to improve. And if they know there is value in your credential, they will work to overcome any initial failure.

That has been the truth of it for me these last two decades. Hopefully there have been many who have been helped by my efforts, both test takers as well as the companies and organizations I have worked with. That is all I can ask for in this yet-to-be-completed journey