It Shouldn’t Be So Hard
I’ve been noticing something I think you’ll appreciate. People, and I’m talking about my family here, have never really understood what I do. I’ve been helping people become certified since 1990, first at Novell until 1997 and then at Galton. I can explain certification easily enough, and they have a certain grasp of what certified professionals are certified to do.
But my role is still a mystery. From your own experiences, I’ll bet you understand that.
Once they learned I had a job at Novell, my family–starting with my parents and then quickly moving to my wife and children–expected me to be able to fix hardware, install software and set up networks. They found it much easier to believe I was a computer network expert than a psychometrician. Just using the term “psychometrician” would make them smirk. After more than 12 years in IT certification, here is what I hear from my kids when they are telling someone else what I do: “He does something with computers.”
Actually, I say, I am a psychometrician, helping build good tests that measure the skills and knowledge of IT professionals. (Mostly these days, though, I direct others, write, meet, talk on the phone and travel. I’m in what they call “senior management,” which seems to mean that I don’t do anything I’m really good at anymore. But why muddy the waters more?)
Psychometricians can create certification programs, but we don’t have certification programs. We are never tested to make sure we are good psychometricians. We do, though, get lots of education at universities, even Ph.D.s. And later, we get “training” on the job.
Our only goal from all this education and training is to make good tests. Our version of the doctors’ Hippocratic oath could be: “First we will make no test that causes harm.” Not as well written as the Hippocratic oath, but you get the picture.
Now, maybe you’re thinking that I haven’t done a great job of explaining to them what I do. Here’s what I think: Maybe they just don’t listen long enough when I get started. Ever see your friends’ eyes glaze over when you talk about your day?
On the rare occasions when my children ask me what I really do at work, I sit them down and start out with the basic concepts of test reliability and validity. Then I explain how important the Standards for Psychological and Educational Testing are. By now, and it’s been less than five minutes, they are completely tuned out, squirming in their seats and asking to go to the bathroom.
I don’t get it. It’s not as if I didn’t start with the most interesting stuff. Or did I? Perhaps I should have begun with some of the more practical topics, like setting cut scores, performing item analyses and creating equivalent test forms. And don’t forget adaptive testing and using simulations. Or maybe it’s not the content at all, but the presentation. Perhaps I should use PowerPoint, some hand-outs and a demo or two.
They do begin to get interested again when I talk about helping catch cheaters and other testing ne’er-do-wells.
They’re not terribly interested in the details of data capture and analysis, Bayesian statistical models or the industry-wide efforts going on. What is important to them is that their dad is no longer a computer network expert. Now it begins to sound more like I am a high-paid virtual security guard standing alone against the forces of evil. And now something happened that had never occurred before: I’d risen a notch in their eyes (apologies to all you networking experts out there), so I left things at that.
Believe it or not, I envy you. When you want to show somebody what you are good at, you can pull out the hard-earned certificate. You can also pull out documents describing the certification, even show actual test score reports.
And your job functions are easier to simplify: I make computers talk to each other; I control the flow of business information; I stand watch on firewalls (that’ll certainly grab attention). There is no end to your ability to make people understand. Some of those people may even hire you or promote you.
Without a certification program, it’s going to be some time yet before my role as psychometrician is understood. Until then, I’ll keep my chin up. And maybe I’ll just tell my kids: Dad works in an office.
David Foster, Ph.D., is a member of the International Test Commission and sits on several measurement industry boards.