Editor’s Note: The Editor’s Letter in the June 2005 issue reported that the National Association of Communication Systems Engineers (NACSE) plans to start using open-book exams for its certification program. Plenty of readers weighed in on both sides of the issue. Here’s what they had to say:
That is a ridiculous idea. Certifications have become more and more “wanted” and less and less “mandated” by employers. Certifications are a must in my book. However, allowing people to use books during the test will just cripple any meaning certifications have left. As for the “pluses” for this, who said certifications were supposed to be easy? I’d rather certifications be extremely hard! That way, those people with certifications are considered to be experts.
Information gathering is a daily occurrence for all IT people, but that does not mean we should be tested on it. Information gathering is something IT people don’t study for—it’s something we pick up on the way. If the NACSE goes this direction, I might as well stop studying, because the certifications I have been working on will be worth nothing.
Central Indiana Power
Regarding your recent commentary on the open book certifications, Albert Einstein said it best: “Never memorize what you can look up.” I fully agree!
Customer Engineer – Data Applications
CCNP CCIP CQS-IPT CIPT
I just opened my June issue of Certification Magazine, and I must say I was thrilled when I saw what NACSE was considering.
I am in the process of obtaining my CCNA, as well as becoming a CCNA instructor. Since before I tested for my A+, I have argued that the certification process, as it is currently implemented, is unrealistic, requiring prospective test-takers to memorize a wealth of information, that soon after the test they will forget and use reference materials should they need to find an answer to a particular problem, and they don’t immediately have the answer.
As a student, and Cisco lab assistant for my Cisco Networking Academy at Glendale Community College, in Glendale, Ariz., I have spent more time on Cisco.com, and using Google’s site search features to diagnose, troubleshoot and repair equipment that students have “blown up,” as well as answer some questions that the Cisco curriculum created by mentioning a topic or feature and not going into too much detail. For example, there is a media activity that asks CCNA1 students to correspond IEEE number designations to their respective technologies, yet in the reading there is no mention of this relationship.
The argument has always been that open-book exams are always easier, but I don’t think that is the case. An individual would still need to know where the information is located.
I believe that there can be a compromise to please both camps, because there is information that truly needs to be memorized, like subnetting and the layers of the OSI and TCP/IP models, and there is information that is memorized for tests, but in the real world is looked up in a reference book, like whether telnet uses TCP or UDP. One possibility would be to set a time limit for each question. A subnetting- or OSI model-related question should require considerably less time than a question asking the user to define port numbers.
After reading your article, I hope that NACSE is able to work with the various institutions that offer their certifications, and bring the certification process to a more realistic approach.
I am so-called certified in CompTIA A+, MCSE NT 4.0 and 2000. I have been slowly working toward a CCNA. In my opinion it’s great to see that some people finally are getting it.
Unless you have a photographic memory, great concentration skills and the desire to spend all your free time studying the latest technologies, there is no humanly way possible to memorize everything needed to pass all the various certification tests.
From the people I know in the IT field, it’s not how much you remember, but where you know to look to find what you need. Example: Google groups.
I just hope that more companies catch on and follow in NACSE’s footsteps.
Richard R. Bedford
I like it, but with a small caveat: If you are going to do that, then incorporate simulations in the exams as well.
For those who argue that it will cheapen the certifications, this: In the real world have you never googled a problem, or used Microsoft’s site? How many of us know everything that there is to know about a particular technology, field etc. Isn’t the ability and knowledge to find the information just as important as actually knowing all of it? (If you know it all, please let the rest of us know how you do it.)
I am currently back in school getting my bachelor’s in information systems at a technical college, and it is interesting to see the different testing styles used by the teachers. Some favor the “old-school” closed-book memorization type, while others are not as concerned that you know all of the material, but are more concerned that you understand the concepts introduced in the subject. Without the concepts, any test given “old-school” style is a memorization exercise, because you really haven’t learned anything. While the open-book contingent realize that you are not going to learn everything in the allotted time, and that in the “real world” you will have references to use whether it is a book or the Internet etc. And those tests, believe it or not, tend to be harder since they are able to be more granular in the questions they ask. Given a time limit, you still have to know enough about the subject matter to know where to look or you will run out of time.
As a comparison, look at the tests given for engineers (EIT & PEs). They are allowed to bring a number books (for reference) and calculators to the exams, and I don’t see anybody proclaiming that it cheapens the tests, nor do I see it devaluing their licenses. Because of the breadth of the tests given, they realize that no candidate is going to remember everything. But again, given the time limit, they need to know how and where to look for the information. Why should technical certifications be any different? I mean, really, how many “paper” MCSEs are out there under the “closed book” memorization testing system? They have already done the damage to the value of certifications.
Electrical Designer/IT manager
I see NACSE’s actions not so much as revanchist as realist. I agree both that non-experts can do well on tests, yet that also that examinations are the metrics we use to evaluate expertise. NACSE is trying to counteract what is, in my opinion, a flaw in reasoning—memorization, boot camps and test preparation seminars. It is this flaw that has detracted from numerous certifications (e.g., “paper” certifications, etc.).
A certification should be a verification of expertise, and expertise is the embodiment of what one knows. Too often a certification is used as a path to promotion or to a new discipline. A whole industry has developed based on going to boot camp so you can become a certified X, and that getting certification Y gets you Z dollars. What has been left behind is knowing it. I didn’t go to boot camps or prep classes to get my certification—I just know it.
Does “open book” invalidate a test? Not necessarily. If the test adequately evaluates one’s knowledge, then the goal is adequately served. An “open book” does not mean it’s “easier” than a traditional certification test. Indeed, it probably should be more exhaustive and detailed.