Since its inception in 1989, the SANS (SysAdmin, Audit, Network, Security) Institute has provided computer security and professional certification training for thousands of burgeoning IT professionals. In addition, it’s one of the largest research institutions of information security in the world, producing and maintaining a vast number of documents as references for the benefit of anyone in the industry at large. As an independent vendor-neutral organization, SANS course alums are in varied positions around the globe, in organizations from universities to corporations, working together to assist the entire information security community.
One reason for SANS’s continued respect and success is its commitment to self-improvement. Every single course SANS has held is followed up by a trainer evaluation. Certainly this is commonplace in any higher education and trade school institution now, but considering SANS’s loyalty to the process, it’s amassed a breadth of knowledge on what makes a good instructor since 1989.
Stephen Northcutt, who’s been monitoring and running evaluations at SANS for 10 years, feels it’s a process that has improved and been consolidated since his arrival there.
“We’ve made progress over the years,” Northcutt said. “We’re measuring four discrete, distinct things: One is the overall satisfaction level, second is the performance of the content itself, third is the performance of the instructor and last is the performance of the venue. You can have the world’s greatest instructor, but if you’re in a big city and rats come running through the room or the food is terrible or the air conditioning is too cold, it’s obviously going to detract from the course experience.”
Immersed in trainer evaluations, Northcutt is a near-encyclopedia of what’s been evaluated poorly and highly, as well extremes of both over the years. All things being even in terms of the instructor, when asked if certain courses are rated higher than others on a consistent basis, he was quick to respond.
“It’s without a doubt the technical courses,” Northcutt said. “When there’s a lot of hands-on stuff that they’ve never seen before, [students] generally score higher than less technical courses, such as our policy course, our legal course — those are going to come in a little bit lower than stuff they’ve never seen before, like hacking techniques or intrusion detection.”
Regardless of the content of a course, however, it’s widely known all things are never equal when talking about instructors. Some can make even the most boring subject come alive to crackle the brain waves; others can make the most exciting topic dry as a bone. SANS makes an effort to teach instructors to make the material engaging, but as the organization has extended its international reach, some of it gets lost in translation.
“Most of it is a reflection of cultural differences, since we’ve been coaching instructors to be animated, because hexadecimal and subjects like that are kind of boring. So, we try to make it lively by injecting some humor in the course, but then you show up in London and try to make the crowd laugh and it doesn’t work. It’s just nothing but crickets.”
Failed attempts at humor aside, evaluations are serious business at SANS. It prides itself on relentlessly evaluating its instructors and courses, not only for the student’s benefit but for its own as well. SANS gives students a chance to be as elaborate as they can on their evaluations and always follow up when necessary.
“You give people a chance to go beyond just multiple-choice answers; the other thing that’s really important is to be ready to pick up the phone so there are no dissatisfied customers,” Northcutt said. “Stuff does happen in the training world — an instructor gets sick, a lab doesn’t work, the projector’s too dim, the room is too hot, etc. When someone is unhappy, you have to be ready to pick up the phone and call them. It is amazing; when you do that, the people always say, ‘I didn’t even know you read these things,’ and I always counter with ‘Oh, you bet we do.’”