SMEs: Where to Find Them, How to Manage Them
Certification exams do not write themselves. There are many pieces to manage from exam design to assessment delivery, and subject-matter experts (SMEs) play a key role in the development process. There are a few tips for program managers to handle SMEs so they meet can meet their deadlines, tap into the best resources and nurture the creativity needed to produce those exams certification candidates are always gnashing their teeth over. First, you have to find the right SME to handle the project.
Where the Experts Are
SMEs can be authors, technical editors, proofreaders, reviewers, etc., but they must have the right knowledge and be the right fit for the right task. Many organizations have established and extensive databases of subject-matter experts with a variety of skills to draw on, and those resources are kept up-to-date. But on occasion, those in-house tools can hit a dry spell. When that happens, many program managers use the Web to find subject-matter experts and network for referrals. Publishers and other stakeholders often will have specific requirements that need to be met, thus narrowing the search field a bit.
“I’m given a course or a book from a publisher, and they want it to be for a certain certification, for instance,” said Kim Lindros, program manager, Thomson NETg. “Some publishers will require that the person writing the book and the tech editor must have that certification already. Or say, ‘I want someone with 24 months’ experience minimum in this subject or topic.’ Whatever they want, I go out and search for that. I have a wonderful tool available to me as a Thomson NETg employee—an extensive author database of people we’ve worked with in the past—and I draw from that. We do keep records on who has what experience, and that’s how I do a search. I go to them and ask, ‘When’s the last time you worked on Exchange Server? Have you been doing that for the last two years, three years?’ It’s mainly a kind of interview process. With some authors and tech editors, they’re folks that I work with all the time and they tell me, ‘Oh, I just recently got my CISSP certification or I just recently got this or that, or upgraded to MCSE 2003.’ It depends on who you’re doing the job for and what they require. A new author or someone I need to find, first I’ll go to the Web site of the organization that’s sponsoring the certification and see if they have any contact people I could get to for a list of people to contact or ask, ‘Would you send an e-mail out? I’m looking for authors.’”
Amazon.com is a popular spot to search for leads on potential SMEs. You also can ask subject-matter experts in your existing database for referrals. Lindros said that some of the best subject-matter experts she’s worked with have been recommended by other SMEs. “You continue to use those people once you find them, and you get a really good pool of subject-matter experts,” said Mary Burmeister, managing editor, Powered Inc. “Sometimes you can e-mail them and they might know somebody. It’s a big networking issue. Before I go on Amazon.com, I usually ping all the writers I have in my database and say, ‘Are you or do you know anyone who is familiar with XYZ? If so, can you have them send me a resume and writing sample.’ That’s a very effective way to find writers.”
Erik Ullanderson, manager of certification exam development at Cisco Systems works with stakeholders to gather or find SMEs. “Our responsibility is the actual exam development, coordinating and working with stakeholders when they’ve decided that they want to create an exam. We work with them to train their SMEs on complex item development. We do not have any subject-matter experts for every possible subject that comes to us, because the technology is moving so rapidly, we really need to have a strong working relationship with the stakeholder. We help them with the exam process, making sure that once they’ve identified subject-matter experts, that they’ve been trained in and given the tools necessary to do what they do best, which is know subjects.”
Do the Groundwork
Once you’ve located potential SMEs, they must be carefully screened to ensure the right fit and confidence so that you, the program manager, can facilitate the level of collaboration needed to successfully complete a project. Burmeister said that she goes through an interview process to screen for candidates who are very knowledgeable about the subject and whose writing is clear, concise and transitions well. Obviously, they must be able to work with deadlines and submit material on time. But sometimes, after all the initial contact and exchange is over, she goes with her gut and takes a chance. Lindros also goes through the interview process. “I require a resume and an unedited writing sample so I can see exactly what I’m dealing with. Raw material is always better than dealing with something that’s been polished by an editor because you don’t know how many people have massaged that.”
To avoid a bad fit, or worse, the need to switch SMEs mid-project, program managers should take the time to check references, ask pointed questions about information listed on resumes and provide as many details as possible before any work begins. “Initially when you have the first meeting about the project, lay the stepping-stones,” Lindros said. “Really go through their resume. Ask, ‘Is it fine if I follow up on references?’ With an author you have to go over the whole plagiarism thing, unfortunately. Interestingly enough, a lot of people don’t understand that you can’t simply go to the Internet and pull and copy and paste. They think because it’s out there its public, it’s free domain, but it’s not. That’s a sticky point, and you have to be real careful about it. For me it’s more of a nuts-and-bolts thing. Deadlines, discuss them ahead of time. If you feel yourself coming up against a deadline, you have to give me a heads-up as soon as possible because I have a whole other group of people waiting on the other side: the copy editor, the development editor, the proofreader, everyone’s scheduled.”
Communicate and Collaborate
Once you’ve got the SME on board, take the time to communicate deadlines and guidelines before work commences. And once things get rolling, it’s important to maintain a continuous and collaborative communication stream. Stay in touch. Ask SMEs where they’re at in the project. Are there any problems? Is the subject matter, now that they’ve delved into it, over their head? Asking the tough questions first can allay problems later.
“I work with that author or tech editor throughout the course of the project,” Lindros said. “Submissions come to me, I review them. I create a schedule of submissions. I offer feedback as quickly as possible if something’s not up to snuff. I almost always try to find backup authors also, backup SMEs to make sure I’ve got another level of protection in case the first one doesn’t work out. That doesn’t happen very often, but it does happen.”
Make sure the subject-matter experts are clearly aware of what the project goals are. SMEs should have a clear line of sight from beginning to end. Or more simply, it should be obvious what you want done, and when the various pieces must be completed. “We want to really give the person a target to aim for,” Ullanderson said. “That really helps a lot. Another thing that’s really important is to make sure that the subject-matter expert understands the value of the upfront decision. To work with each other not just at task-writing time, writing the item, but talking about the design of the assessment, because design is going to affect the tasks that are written, which will affect the exam that is delivered, which will affect the results being generated through the psychometricians, w