This feature first appeared in the Fall 2016 issue of Certification Magazine. Click here to get your own print or digital copy.

NOTE: Want to create your own certification program? This is Part 1 of 2. The second article addressing this topic was printed in the 2017 Winter Edition of Certification Magazine.

If you are looking to create a global, world-class IT certification program, or if you'd just like to know what goes into such an endeavor, then please consider this article a roadmap for your journey from great idea to declaring continuous success! Some readers may also be faced with taking an existing program to the next level, and these same steps can be used to achieve that as well. We'll also cover some specifics for folks who are merely adding to or enhancing, rather than starting from scratch.

When corporate leaders or the members of an industry organization board decide it is time to create a certification program, there needs to be some education involved, both for the person who has been tapped to lead the effort, and for the leaders who have come up with initiative.

In order to educate leaders, some upfront questions need to be asked of these stakeholders, since the starting point and the number one action item on your to-do list is to have an agreed set of goals.

Setting goals for your program

The first thing to ask is what are the goals of the program? It sounds like a simple question, doesn't it? Answers that let you move ahead with the process — the aims of a stable and flourishing program that you must guide your supervisors toward — are as follows:

• Validate skills and knowledge.
• Raise the visibility of the company or industry body.
• Raise the technical competencies of customers, partners, employees, or membership.
• Create lifelong loyalty among a new corps of advocates for your company, organization or program.
• Be recognized as the industry leader in certification.
• Reduce demand into your customer services organization.
• Create a greater talent pool of expertise.

Once these goals are in place, you must figure out how to achieve them, as well as determine the measurements you'll need to declare success. Without a doubt, establishing parameters of success is a very, very important factor.

For example, what happens if you have a CEO or board chair who expects thousands, if not tens of thousands, of new certified people in the first year of the program? And after an audience analysis, you realize that the likely reality is far different?

Once you come up with a realistic number — realism should be a key feature of all your ambitions — then you need to get sign-off. You may believe you can claim success simply by exceeding numbers for other similar first-year programs, but if that metric was not established up front, then you risk creating a perception of failure.

It is also important to educate with facts and information. Before making any presentation, focus on getting clear and accurate data to illustrate your projections. People who have no certification background are not likely to understand the effort, rigor, and resources required to produce a world-class global certification program. Frankly, many training professionals do not understand as well, and often struggle to put together a decent assessment (or test).
What is also needed when you go for sign-off or approval of the program is to detail the resources that will be needed for Subject Matter Experts (SMEs), as well as the time commitment involved. If you've done your homework, then you should win the hearts and minds of your SMEs by the end of your first exam development workshop. And after that, you'll be on the path to the success.

SMEs become evangelists to your leadership team, and if test takers know that leading experts in their field have been involved with exam development, that lends instant credibility. Hence, we will cover how to win SMEs over — which largely means being prepared and making productive use of their time — and then also how to promote and gain traction with your audience and stakeholders.

We are getting a little ahead of ourselves, but do keep these things in mind as we outline your next steps.

Getting your ducks in a row

You will be proposing a detailed plan. The elements needed for this are as follows:

Audience Analysis — Project the size, scope and location of the potential audience for your certification program. Groups to track include Internal Candidates, Partner Candidates, Customer Candidates and Other (Guests). Without this information, it is difficult to measure the success of the program and determine what your target numbers are. Will your partner or channel programs leverage and use certification? Will your company use the program internally?

Job Analysis — Determine the type of jobs or roles you are seeking to certify, and conduct a Job Analysis to determine the skills and knowledge needed to perform well in those jobs. A job analysis, and some will call it Job Task Analysis, is part art and part science. You need to be able to reach out to star performers and select stakeholders and ask key questions that will help form the basis of your exam blueprints. Essentially, what is the knowledge needed for the job at hand, and what are the key tasks and activities that go into it?

Program Structure — Determine the model and framework of the program. What are the paths available to certification candidates, and how will you roll out the program? Plan for both the short and long term. What exam will bring the most bang for your buck? Once you figure that out, you will have identified the first exam to develop.

Program Framework — What are the paths your certification candidates will follow? What is the name of the program and its credentials? The framework will also give details about plans for the future of the program. Will there be a layer of low-stakes exams on top of high-stakes exams? An article written several years back for Certification Magazine, "Qualification vs. Certification," explores this area in depth and is still relevant today.

Training — If you build your exams properly, then the training will fall into place. If there is existing training, it will likely need to be updated. If there is no relevant training material available, then there is an opportunity to create it. You may also decide to just point people to existing content.

Your exam blueprint is the driving force for letting people know what is on each exam, and in turn will drive the training content. This is a critical factor for training development, and will be the revenue-centric part of the program.

Determine a Timeline — The launch date and all of the activities and deliverables need to be in place. A company or organizational event is a great place to launch a new program, release a key exam, or unveil a reboot of an existing program.

Finance Model — Determine whether the program is part of a Profit Center — or part of a Profit Center that is supposed to at least break even — or a Cost Center. If the latter, then the program is overhead and a solid justification of the cost needs to be in place. The farther away from revenue, the more vulnerable the program is, so the justification of your new certification program needs to be strong and clearly backed by leaders and stakeholders.

Determine Delivery Method and Pricing — Perform a price comparison and research the test vendors and costs. Certification is high-stakes proctored testing, but there is now online proctored testing to consider as an alternative to traditional testing centers. Different vendors have different costs. Stick to those with true certification experience, as opposed to an LMS (Learning Management System) vendor who merely claims to offer certification testing. Also, keep in mind how to track your certifications.

Will there be low-stakes non-proctored exams — essentially open book tests? How will those be tracked? What if multiple exams are required to gain a credential? Can something be created in-house or should a third-party product layer on top of your LMS? Again, beware the LMS vendor who merely states that they have this covered.

Exam Development — Line up six-to-eight SMEs, conduct an in-person exam development workshop, and have your exam blueprint in place. Write and tech-review your item bank in a four-and-a-half-day workshop. (I'll have more to say about this in a bit, and we will focus on how to conduct the most effective workshop you can.)

The key to great exam items is to create questions that incorporate a scenario (we'll discuss this in greater depth shortly). Remote exam development takes far longer. Once you have experienced SMEs at work on your tests, it is far easier to bring in new SMEs.

Determine All Costs — This should include travel and expenses for SMEs, in-service analysis, marketing, and conference room rentals for exam development. It is far easier to get an SME to go someplace nice like Orlando, and it can actually be done cheaper there than flying them to the home office. The exam item bank is dependent on great SME test writing, and getting them into one room will be a critical factor in your success.

Your preliminary budget should include:

• Exam development costs (including Subject Matter Expert time and expense)
• Training development costs (if needed; remember, however, that this is where potential revenue resides)
• Program manager and operations costs (people and systems)
• LMS add-ons — Since no LMS really does certification operations well, you will need to bring in a third-party system to layer on top, or start off with a spread sheet.
• Web resources and marketing (Three things to bear in mind: 1 — Your website is how you communicate your program. 2 — Your website is your call deflection center. 3 — Build an FAQ immediately.)

As you do your budgeting, bear the following caveat in mind: Never give away anything for free. At the very least, trade any favors like free exams for testimonials. Remember, FREE is four-letter word! At the end of the day, the exams you create have value! Value to the test taker, value to partner programs, and value in the marketplace. Don't sell yourself or your program short, or else you risk being seen as having little value.

The proposal to your decision makers is now ready. You need them to sign off. Review the following list to be sure you're covered:

Goals — Have them firmly established.
Audience Analysis (Expectations need to be set properly) — Know your numbers and where they are.
Timeline — Set a realistic frame.
Job Analysis — Be swift and do not delay.
Exam Development — Be efficient, conduct it on-site, and remember that remote development takes much longer.
Exam Delivery — Figure out your vendor, as that determines price.
Operations — You need to figure out how you are going to track and report and, if you have vouchers, how are they going to be managed. How are people going to get their badges and certificates? Are you going to have a registry? Managing your test vendor is also a time commitment.
Marketing/Branding — You will need digital badges (some may call them logos), certification credentials (electronic PDF), naming conventions for your credentials, and the name of your program. Develop a marketing plan from the get-go.
Budget — If you have no money, then there is no program.
Metrics — You need to track and report, get your target numbers agreed to and signed off. Plan to conduct in-service analysis for your exams, rather than conducting a beta.

Now let's cover a few of these items in greater depth.

Time to Market — The next step in your journey

To streamline exam development and speed the time to market, the key is to conduct an in-person exam development workshop. By being as efficient as possible, you can streamline the days required of your SMEs. There is a cost involved for bringing a cadre of SMEs together, but if a new exam is created end-to-end in four days, then everyone wins.

Leadership is happy that costs were kept down, and the SMEs are happy since they were productive with their time. They created something of value, something to be proud of, and — critically, for you purposes — something that they will be proud and eager to promote.

If SMEs are brought to your home office for exam development, they are likely be pulled into different meetings. When an SME goes in and out of a workshop, they become less productive and oftentimes disruptive. So let's take a look at the ideal exam development workshop.

Look to have six-to-eight SMEs from various groups. If all the SMEs are from one group, then the test will have just one point of view, and that will sink the exam before you've even gotten started. More than eight and it becomes too cumbersome. Less than six and you run the risk of not being able to create enough exam items.

Experience has shown there are usually two out of six who really struggle to write a good item, but they can be used for tech review and sourcing. Each of your questions needs a reference or source. If you have only five SMEs, and two of them are struggling, then you will have to reevaluate how many items you can generate.

Keep in mind that a 60-question exam can properly determine a candidate's skills and knowledge, so a 70- or 90-item test is not necessary. You should, however, at least aim to have 1.5 times the size of your exam blueprint in your test item count. So if you are creating a test following a 60-question blueprint, then you need to generate at least 90 items, and they need to be proportional to each section of your blueprint. We will go over the key points to creating an exam blueprint in a follow-up article.

This is essentially a rapid development workshop, which is a far cry from the between several weeks and several months that it used to take to get an exam created and published. That all changed back in 2000, when the demand was to create exams in a short time frame and keep them viable for a longer time on the shelf. If it takes six months to create an exam now, then it will be out of date before you know it — sometimes before it is even published.

To beta or not to beta, that is the question

Not to be too dramatic, but if you want to slow your time to market to a crawl, then do a beta. There are also other cons to consider, like unnecessary test item exposure. Plan on creating a great exam and have a small group take the exam prerelease so you can test your delivery system and see if everything is working properly.

It is also ideal to get a few top performers (people expected to do well on the exam), a few novices (people who are not), and a few midrange professionals (people who should be right around the pass/fail mark and, with some additional study, will likely pass the exam). This will give you a very good idea whether the exam is on target.

Now that a strong exam is in place, one which validates skills and knowledge, and the pass/fail mark is set accordingly, let the exam into the marketplace and plan on conducting an in-service analysis. The analysis will let you know how the exam is actually performing.

Are the top performers scoring well on the exam as expected? Are the novices getting the scores they should be getting? Are the borderline candidates getting borderline scores (and in that case is borderline OK)?

It is ideal to run an exam testing center at a company or industry event so you can meet and talk with the candidates. You can get a feel on where they stand in terms of experience with the job or product and see whether the test validates properly.

Exam Security — How much time do you need to spend on this?

An exam that consists entirely of scenario-based questions is an ideal goal to have. This not only creates an exam that truly measures skills and knowledge, but one that is harder to cheat on.

Even if a candidate had all the questions ahead of time, and the answer options (but not the answer key), he or she would still need to parse through the information. A good scenario-based test item should be between three and five sentences long with a question at the end. This may seem obvious, but you still need to ask the question. The scenario gives the parameters and context for the question.

This makes it very hard to memorize much of anything, as the scenario will shift the direction of the answer. Of course, scenario-based questions are the hardest to write, and it is not until the second day of the workshop that most test item writers start to get the hang of it.

To stay ahead of test cheaters and test cheat sites, don't announce any updates to your exams. Simply refresh them. The test cheat sites will fall behind and be ravaged by the people who purchased their illicit wares. Also, set parameters for your test vendor to flag, or for you to look for. Parameters could be an abnormally high (or low) score, or a time element, like spending next to no time on each question and then taking 50 minutes to respond to the last question.

Legal Concerns and NDA Agreements

Most IT certification programs have a splash screen that comes up when taking a test, essentially asking the candidate to abide by the rules of the program and to agree to them. Not agreeing will result in the test being exited.

Post the rules of the program on your website for reference. Create an NDA (non-disclosure agreement) for all of your SME test writers to sign. These SMEs need to know not to share any of the information they have gained during the test writing workshop, and an NDA will help do that and emphasize the seriousness of the test's integrity.

Do not share answers with anyone for any reason; it will invalidate your exam. Document the processes on how the exams are created.

Marketing — You built the program, now what?

Get the word out! Tie it to a company meeting or event, get your website up (with an FAQ) and build up testimonials (perhaps by trading for favors, usually a voucher or two). A few musts:

• Get your framework and badging some visibility via the website.
• Write a blog, write a blog for the CTO or CEO, do a podcast or four. Get a certification e-mail box, and make sure all the questions are answered promptly (and then up- date your FAQ).
• If you have a great FAQ, update it regularly and send the link out in every response.
• Create training if needed, or update training content as needed, or create a training partner net- work. Also, make sure to update exams in a timely fashion.

Feeling intimidated? Embrace the challenge! Live it, breathe it, enjoy it — and keep at it.

In my next article addressing this topic, we will go into more detail on improving your program and taking it to the next level. We'll go into marketing, ROI, leveraging surveys, establishing benefits, recertification, credential management, and creating a buzz!

NOTE: Want to create your own certification program? This is Part 1 of 2. The second article addressing this topic was printed in the 2017 Winter Edition of Certification Magazine.