One of the most common occurrences at the conclusion an IT certification exam development workshop is when a subject matter expert (or SME) comes up to me and says what a great experience they had — and that they can't wait for the next one.

Oftentimes, these SMEs were assigned to participate and go into the exam development process with little to no idea what to expect. For a first-time participant in an exam development workshop, I typically give a 15-to-20 minute prep session that includes a review of the exam blueprint and guidelines for writing questions.

First-timers are usually asked to write three to five items (exam questions) ahead of time, picking whatever objective or objectives they would like to try their hand at. From there, they send items to me for review. I edit and make suggestions. The folks who do this are always a half-day ahead of the others, who were unable to write questions before the workshop.

Once the development workshop is in full swing, we review the exam blueprint and then review initial items. After the initial tech review, the test writers sign up for the objectives they would like to address and off we go.

In the zone

Now we come to the crux of why so many first-timers enjoy the experience. For every exam, I strive to have at least six subject matter experts participate, but no more than 10. I want them to be from different roles or organizations, and I want partners (people from outside the company or association creating the exam) to participate.

Over the course of a weeklong workshop, the SMEs write and review each item in a systematic process — one that I have refined over the years. At the end of the week, an exam bank is created. For a 60-question exam, the exam bank will vary between 90 and 115 potential questions. This will enable us to create more than one test form.

To meet this goal of creating a test bank, it takes an incredibly focused effort for a four-to-five-day workshop. My recent experience is geared toward exam items that are all scenario-based. These are difficult to create, but the payoff is an exam that tests skills and knowledge and requires critical thinking. This ability to think things through is a highly valued characteristic when working in the field — or anywhere for that matter.

To create such items takes skill and knowledge. The test writer becomes immersed. Many subject matter experts learn quite a bit during the week, as each item is defended to be technically accurate. Writing must be clear, and everyone improves their writing skills though the week. This is a skill that enhances any career, and one I am most proud to help with.

Lasting relationships

The tight team of SMEs forms a bond. It is perhaps the most collaborative setting one can experience. Everyone’s professional network grows. Internal company folks get to meet with partners from the field and vice-versa. Services people get to know support people. Everyone can share their point of view.

The bonds that are formed this way can last years. There are SMEs from many years ago who can reach out to others, including myself, and the reconnection is immediate. Each SME can see the strengths in skills and knowledge in others. Everyone can contribute and for those who struggle to write items, others are there to help. The folks who struggle can also help in other ways, checking resources, giving perspectives, and contributing ideas.

There are usually a couple of standouts in every group and their reputation grows within the community. There are exam writers often become very interested in working with the certification provider's education group after going through this process. People from the education group also learn quite a bit from the process and from real-world practitioners. They also get to see whether there are any gaps in the training they have been providing.

Other rewards

The experience is typically a break from each SME's normal responsibilities. They get to have a full understanding of the skills and knowledge applied to the job associated with a role-based exam. They themselves will have an easier time passing the exam. They can earn recognition as an exam writer.

I also try to hold workshops in desirable locales, to help in the recruiting of participants. Once people come out of the workshop, they become advocates of the exam and the certification program. They plan on participating in any updates to exams.

And at the end of the week, they will come up to me and say they had no idea it would be this much fun. They will tell me how satisfying it was to create an exam. (And it is quite an accomplishment.) So, if you ever get a chance to be part of an in-person exam development workshop, then go for it!