Certification: Creating the World as It Should Be
Certification programs try to identify individuals with a set of knowledge, skills and abilities that allow them to perform a particular role. Most try to align closely to prevailing practices in the field. But certification programs should aim higher and elevate practice in the field.
Most certification exams begin with a job-task analysis (JTA). In a JTA, people in the role being tested participate in a structured exercise similar to what someone in a role should know and be able to do. The aim of a JTA is to describe the world as it is. This descriptive approach is most effective when applied to roles in which change is gradual. Double-entry bookkeeping has existed for centuries. While practices do evolve and change, accounting practices are relatively static for periods of years. Therefore, it is appropriate for an accounting certification to describe the world as it is.
Rapidly evolving new products and technologies make IT quite different. New technologies might represent a significant change in how certain IT objectives are achieved. New technologies might even change the objectives themselves. If a technology vendor intends to certify people in their use of new technologies, it must describe the world as it will become.
Furthermore, practices can become entrenched and remain commonplace even though best practices evolve that advise against the older ones. For example, many organizations use Telnet for the purpose of connecting to and administering remote UNIX and Linux hosts, even though this approach has long been recognized as a security risk due to vulnerability in its authentication method. Consequently, a suitably representative JTA focus group might identify Telnet as an exam objective for a system administration exam because of its continued use in the field.
By making this a requirement, the certification program effectively advances the bad practice by enshrining it as an exam objective.
Sometimes certifications should be prescriptive and describe the world as it should be.
What does this mean in practice? Consider Red Hat’s flagship certification, Red Hat Certified Engineer (RHCE). Red Hat’s certification team believes it would be doing its customers a serious disservice if it promoted the use of Telnet for remote system administration by making it an exam objective. Instead, for years, Red Hat has expected people taking the RHCE exam to know how to implement and use Secure Shell. Secure Shell provides better functionality more securely. In doing this, Red Hat tests for the world as the company believes it should be and feels confident that this leads to better security practices in the field. Once practices improve, what was once prescriptive becomes descriptive.
This year, Red Hat had a new opportunity to reflect on the notions of descriptive versus prescriptive exam objectives. Red Hat recently launched the JBoss Certified Application Administrator (JBCAA) program in support of its JBoss middleware product line. To explain how descriptive versus prescriptive has come into play, some background on the technology is needed.
At the core of Red Hat’s JBoss product line is the JBoss Enterprise Application Platform. The core of the core is the JBoss Application Server. One can think of an application server as a Web server that provides additional functionality for running particular kinds of code. In the case of the JBoss Application Server, the code is in Java. The application server effectively reduces the amount of new code an application developer must write when creating applications used to buy shoes, books, plane tickets, etc. It also allows the code to run faster and to provide functionality that might be difficult to create in a stand-alone application.
JBoss is an open source software project (actually, many under a single umbrella). Just as Linux arose from the desire to run an inexpensive, customizable UNIX-like operating system, JBoss was created to provide an inexpensive, customizable competitor to the large, expensive application servers on the market. It has made its greatest inroads with the developer community. A developer might work in a shop that uses a competing, proprietary application server for production, but will still often use the less expensive and more configurable JBoss environment for development.
Over time, the effectiveness of JBoss technology has led organizations to move beyond using JBoss for development to using it in production environments. Right now much of the setup, administration and monitoring of JBoss Enterprise Application Platform is being handled by application developers — that is, by the people who write the code that is deployed to the application server.
Application developers might have deep knowledge of Java and application development, but that does not necessarily mean that they understand how best to set up JBoss Enterprise Application Platform, deploy applications in a production environment, monitor the server and in general ensure that everything is working properly. They understand their application better than the platform on which they will run it. Consequently, there are things they probably could or should be doing to ensure successful deployment but might not be.
On the other hand, the production operations groups that manage the underlying systems on which the application server runs often are more conversant in operating systems and networking than in a more narrowly specialized technology such as an application server, so if administration falls to them, there are — again — things they could or should probably be doing but are not.
Right now, the role of a JBoss application administrator is not as sharply defined as, say, the roles of Linux system administrator and network administrator. One would not find many people using this technology whose job title matches the role, even though the technology has a large user community. Rather, what one finds are people in other, related roles who are taking on these responsibilities in addition to others. Consequently, a strictly descriptive approach to defining the JBoss application administrator role would have likely produced a set of exam objectives around just getting things set up to where they are good enough.
Red Hat expects to accomplish more with its programs. It is not enough to measure whether people can get things set up just good enough.
In developing the JBCAA exam, Red Hat conducted a job-task analysis and performed many of the steps necessary to describe the world as it is. However, in that process, and in further development of Red Hat’s work model, the company involved a lot of people in a variety of roles for whom administering an application server is not their primary responsibility. These included support engineers, developers and enterprise architects. Rather than asking questions such as, “What do you as an application administrator do?” Red Hat asked, “What would you expect or require an application administrator to do?”
The result is that Red Hat’s exam objectives not only describe the practices in the field, but also prescribe tasks and practices that ensure better coordination of efforts among the various roles that interact with JBoss Enterprise Application Platform. By describing and prescribing, Red Hat moves the ball and begins to elevate practice. A JBCAA will interact more effectively with developers, architects and support engineers than the ad hoc administrators of today.
Consider the words of one successful pilot JBCAA candidate — someone who has won awards and recognition for his work with JBoss: “That was certainly one tough exam, but I will say, I went back and started implementing some of the things that were on the test that I wasn’t doing but should have been.”
Randy Russell is director of certification programs at Red Hat. He can be reached at editor (at) certmag (dot) com.