Solving Certification Program Management Problems
This article provides information on some typical problems encountered as certification programs are developed. Despite its 12-year history, certification in the information technology industry can still be considered to be in its infancy. This is both good and bad. Bad, because there are no models or blueprints to guide a program manager from start to finish, nor are there established guidelines to ensure success, minimal cost and optimal efficiency. Good, because the successful approaches to certification in IT can still be improved. Success can be made more successful. Incremental advances as well as new paradigms are still waiting to be discovered. Certification in the IT world is an exciting place to be.
The EMC Proven Professional certification program deepens your understanding of EMC products and technology, enables you to develop a core competency in critical elements of EMC E-Infostructure and improves your ability to articulate the added value your expertise in E-Infostructure means to your constituency. This world-class technical certification program incorporates the industry’s best practices for e-learning and certification, as well as our own best practices and technological expertise developed over decades of experience.
The EMC Proven Professional certification program enables you to get maximum value from EMC E-Infostructure. This unique program gives you the knowledge you need to better design, implement, operate and teach others about EMC-based IT infrastructure.
The Starting Point
The EMC executive board approved the business plan for the EMC Proven Professional certification program in September 2000 and set a February 2001 date to launch the program. In order to meet this goal, the program had to be rolled out in phases, and the exam that serves as a prerequisite for all the certification tracks needed to be in place by February 2001.
Because of its experience in helping organizations create successful certification programs, particularly the tests that support them, Galton joined EMC in its efforts to create a high-quality certification program for storage solutions. Galton’s founders began the pioneer IT certification program while at Novell and have helped hundreds of IT (and non-IT) organizations in their certification efforts. With the ability to provide services for job analyses, item authoring, computerized test publication and test results management, Galton was viewed by EMC as the one-stop shop for its IT certification needs. With its innovative Exam Express program, Galton was the portal through which EMC was able to easily access Prometric’s channel of secure testing centers.
Problems and Resolutions
Since creating a program of this magnitude takes on certain problems and issues, the resolutions and workarounds become the key to success. The following text covers problems encountered by EMC and the solutions worked out jointly by EMC and Galton. It is expected that problems similar to these will exist in other programs and perhaps these EMC/Galton solutions can work for them as well.
Problem 1: Maintenance
Many IT certifications are product-based. That is, the certification is based on the features of a new software or hardware technology, or on the tasks a person can complete using the product. This product-based approach limits the scope of the certification, causes more frequent revisions to exams and makes the revision process more difficult.
Solution: EMC decided to make its certifications role-based and categorized the program into four tracks:
- Operator: Designed for the person who runs a data center.
- Builder: Designed for someone implements or builds a data center.
- Architect: Designed for someone who architects and plans the enterprise storage network.
- Instructor: Designed for someone who teaches in the three tracks.
The key to was to break each track into “areas of expertise.” The areas were the primary skills and knowledge one needed in each track. The objectives for each exam will stay valid until there is a shift in the work required. If the job role stays the same, the exam items will be valid longer. That has proved to be true to date.
Problem 2: Time to Market
Time to market is critical in the IT certification world. If it takes too long to develop and publish an exam, a return on investment is not realized. The next three problems present innovative solutions to reduce the time to produce a quality certification exam.
The Galton proposal for development services was detailed and provided timelines for completion of tasks, but it was too lengthy and too complex for everyone to understand. This made it difficult to get quick buy-in and decision making for the projects.
Solution: At EMC’s request, Galton condensed its standard proposal into a two-page table outlining succinctly the tasks, responsibilities, time frame and costs of the project. Using the table, all parties quickly understood exactly what was required and could easily see the interactions and dependencies. A budget was quickly forecasted. In addition, the two-page table enabled EMC and Galton to put project plans in place in a quick and timely manner and speed the budget approval process.
Problem 3: Accounting
With complex development projects, there are many milestones. It is possible and reasonable for Galton to provide invoices for its work following each successfully completed milestone. Because of the compressed time frame, the milestones came often, causing a lot of work to go into the invoicing/payment cycle. This work could be better utilized in managing the development aspects of the project
Solution: Galton agreed to invoice EMC only twice on each test development project, once at the beginning and once at the end of the project.
Problem 4: Exam Shelf Life
The beta exam cycle can be time-consuming and needs to be a short as possible. In that way, the “live” or published exam has more shelf life.
Solution: By reducing the number of beta exams taken to run the psychometric evaluation to shorten the cycle, EMC contracted with Galton to provide an ongoing in-service analysis of its exams. This analysis monitored the quality of existing certification exam periodically, confirming the ongoing quality of the exam and postponing the need for further beta analyses.
Problem 5: Resource Time
The creation of test questions can occur in many ways. One good way that focuses on quick but quality development is the item development workshop. In the workshop, item writing, reviewing and editing occur almost simultaneously and require constant coordination between the EMC and Galton teams. The first EMC exam workshop was slated for mid-October 2000, and a project plan with deadlines was established. Galton had worked with many clients in the past that had good intentions but missed deadlines for their contributions to the project schedule. Therefore, Galton project managers built in extra days in the schedule to accommodate these typical delays. Obviously the existing schedule was longer than it ideally should be. EMC wanted a tighter schedule so a negotiation of dates took place. EMC committed to and had to prove that it could meet the new deadlines on the first project in order to keep the new, streamlined project plan in place for future exams.
For the reasons mentioned above, item development workshops were scheduled for longer periods than actually necessary. Subject matter expert (SME) time was valuable both in actual cost and in opportunity cost.
Solution: Workshops, which used to take four days, were reduced to three days. By streamlining the first-day instruction given to the item writers, the time s