‍

Few people can claim to have played a crucial role in the birth of a new industry, but by establishing a computer-based testing (CBT) solution for Novell over a quarter of a century ago, Clarke Porter and Carolyn Rose were among the early pioneers.

Back in the 1980s, prior to the existence of the internet, the entire testing network consisted of 100 testing centers linked to a mainframe in Minnesota. But the company Clarke founded — now called Pearson VUE — currently employs teams of content developers and psychometricians to help develop sophisticated computer-based tests (CBT) delivered through a network of over 5,000 test centers in 180 countries — over 15,000 if you count its Certiport business.

Alongside other talented and competitive industry pioneers, they took risks and were often met with resistance. And now, after reaching far beyond even their original ambitious vision, their opinion about the future of CBT holds considerable weight.

But in 1988 it was a very different story as Clarke — then working at Control Data Corporation as the business development leader in the commercial training group — created the very first CBT system to solve a problem in the IT industry.

An entire ecosystem of training providers, professional associations, vendors, delivery partners, and credential issuers rapidly formed. It is this rich infrastructure that drives growth in the industry today. But the ecosystem is now being challenged by technology changes, fraud, and a shortage of people interested in technology careers. Now the question is how the industry will evolve.

In the beginning

“It’s amazing that the root problem was almost the same as it is today — we need more qualified talent,” says Clarke. “At the time Novell was the leading networking software provider, with north of 80 percent market share. They recognized that they could not effectively directly employ the number of support personnel needed to keep up with the demand and growth rate. Ultimately that was the impetus behind pioneering certification for IT professionals and the need for a cost-effective global infrastructure to deliver high-stakes testing.

“We were in a technology environment that was mainframe-based at the time, and there was a testing system Control Data had called PLATO. Soon there were around 100 beautiful testing centers in major metros around the United States.

“They were designed to do computer-based training. After a while we realized we could do other things there, and began using this set-up to test stock brokers.”

FINRA, then called the National Association of Securities Dealers (NASD), became the first CBT customer, delivering its stock broker licensing exams by computer.

“In New York, LA, and Chicago stockbrokers could just walk in, sign up and take their test,” says Clarke. “As you might imagine, it was very popular.”

Clarke started off by building the stockbroker testing software, and was in charge of getting testing software to other customers. Then in 1989 he became responsible for commercial sales at Control Data for something new — training.

Teaching through certification

“We thought, when talking to Novell about this problem of rapid growth and a lack of qualified support personnel, that maybe we could do training. They had two training solutions in place, Novell Authorized Education Centers, which were third party rather like our third party testing channel today, and company-owned training centers called Novell Education Centers.

“But when they looked at what was happening in their Novell Authorized Education Centers it was a bit inconsistent and they did not have a way of checking if people had learned what was required. So instead of trying to lead with training, we suggested they might do a certification program where they would actually certify their engineers to support the Novell product. That’s how it started. Novell primed the pump by creating a business requirement that people employ certified personnel.

But the certification industry and indeed Pearson VUE would not be what it is today without Carolyn Rose, who championed computer-based testing and certification from the client side. “I was put in charge of Novell Education in the ’80s, which was a side organization within the company,” she says. “It was seen as a necessary evil because you have to offer training.

“We had a traditional model of training where it was direct. We owned the training centers and hired instructors but that was a huge pinch point for the growth of the company. We had a backlog of at least six months and that was a problem. So we envisioned authorizing third parties to deliver the training.

“The Novell Authorized Education Centers (NAECs), we authorized the third party businesses with instructors that used Novell Education materials to teach the Novell designed courses. The approach paralleled and complemented what we did on the testing side.

“We realized that we had a quality issue. An instructor just went to two classes to get qualified. We decided that’s not good enough so we implemented certification exams for them. The first real strong certification which came about was the Certified NetWare Engineer, CNE along with Certified NetWare Instructor, CNI. The idea there was we would certify individuals as third-party support personnel to address the growing demands for NetWare support.”

Training not required

Carolyn and her team decided that they would not require certification candidates to go through compulsory training, as long as they had the knowledge to pass the exam.

“We decided we wouldn’t force them to go to a class,” she says. “A lot of people thought that was crazy. And that’s when I met Clarke. He was trying to help me with the PLATO centers to get them to commit to building out a global infrastructure and really help us grow our certification programs, but it wasn’t consistent with their business plan. So Clarke said, ‘You know what, I’ll do it.’

“PLATO was on mainframe and the emerging technology of the day was client server computing. So Clarke said we can build out a global testing center infrastructure for you using client server technology. We said let’s do it, knowing we were taking calculated but well managed risks.”

Stick to the plan

Like many pioneers, Clarke and Carolyn decided on core philosophies and stuck to them: exploiting available technology and accelerating growth of the network computing industry through quality education programs. In 1987 over 2,000 IT students had been trained and by 1994 they reached over half a million students per year. In 1986 there were 47 authorized Certified Network Instructors and in 1994, over 2,600. By 1994 they also had 44,000 Certified Network Engineers.

Clarke says: “We launched a third party testing channel strategy using the Novell Authorized Education Centers, community colleges and Radio Shacks. We had to make our own technology because we had decided to move away from the PLATO system. We needed our own solution and we decided we would use Novell Local Area Network (LAN) technology in a hub strategy. We built a completely networked testing solution where we could do centralized registration and testing on PCs and linked to LANs which would dial in to our communication hub to send us back results and get registration data. Over a very short period of time, we shifted all the business from the Control Data PLATO solution onto this LAN solution that Drake owned and we used these third party centers. So the birth of the third party network was really out of necessity.”

But Carolyn remembers that it did feel like they were taking risks when they delivered the first tests.

“We were very scared when they turned the switch on,” she says. “We had paper-and-pencil backup just in case something went wrong, we were really nervous. But it worked.

“Once we felt comfortable with the security, infrastructure and processes we then allowed the Novell Authorized Education Centers to deliver the exams. That was a huge breakthrough because the quality of the test centers was better, it was a much better alignment with where our business was going.”

If you build it, they will come

It was at that time that Drake got out of the training business, deciding to focus instead on assessment, and changed their name to Prometric.

Clarke says: “Ashton-Tate was our second certification client. They pretty promptly went out of business so a lot of people are not aware of it. Shortly after that we brought on Sun Microsystems, Microsoft and IBM. We decided we needed to build a community for the test sponsors and hosted our first annual conference at Santa Cruz Operations in California.

“Carolyn was a great sales support person. She went on sales calls with Drake explaining why Novell had created certifications for IT. There were only six companies at the first conference but we were trying to build interest and enthusiasm in certifying IT personnel. We were ramping up to half a million tests a year by this stage in the early nineties, with test centers in Europe, the Middle East and Australia, along with some smatterings in other countries.

“Around 1990 we began branching out to other sectors. Our two big sectors were the Federal Aviation Administration and then the IT sector with companies such as Novell and Microsoft. We were growing exponentially. Other companies looked at what Novell had done and the lessons learned, and as a result began focusing more on psychometrics because a blueprint had been established. The industry kept maturing and learning from itself.

“Novell required their resellers to hire people, but then companies started hiring away these professionals from the resellers to go and work in their IT departments. Then they saw that taking the Novell curriculum was a way to really get their people up to speed. Of course the resellers had to hire more people and get them trained and certified. So it took on a life of its own and became a chicken and egg situation.”

Building a better exam

As CBT became mainstream, test integrity became more important, says Clarke: “They realized now it’s gotten serious, we’re now giving people career-ready skills, career paths and it’s more important to protect the integrity of the item bank to make sure that the tests are valid with better psychometrics.

“At the beginning of Novell certification, the test was open book. There was proctoring but there was less emphasis on it. Our first evolution was to make the tests closed book and then to make sure we policed our proctor network. We introduced secret shoppers and other measures to make sure the proctors were not helping candidates cheat. We placed more emphasis on proctoring as it matured. Psychometrics in IT testing started getting more prominent around 1992.”

In fact, one of the big steps Novell took was hiring David Foster, a leading psychometrician, now the CEO of Caveon.

Carolyn says: “Before ‘Dr. Foster’, Novell’s exams were not psychometrically sound and the quality was not what it could have been. Dave’s expertise made the exams psychometrically sound, statistically reliable and defensible so we could really stand behind them.”

And they took the unusual step of working with rival companies.

“In the Novell spirit of ‘co-opetition’ — cooperating with competition — Dave shared his knowledge with our competition,” she says. “We would do these dog-and-ponies, sponsored by Drake, and we would educate the whole industry on why this was the way to go. This helped everyone in the industry including us.”

Wanted: IT skills

Clarke explains that the industry began because of the need for qualified staff in the IT industry.

“All the IT companies were using third parties to sell, support and service their products,” he says. “The question is: How do I know I’ve got good people? The real driver for certification was that it provided a total quality management package for IT companies to manage human capital they could rely on who are not their employees.

“They are not selecting these employees or working on their career development or training but they can say: I need you to meet these standards to be able to support and use my products properly. Novell were saying: I need to leverage a lot of third party resources but I don’t want to unleash unqualified personnel out into the marketplace because then my implementations will fail.

“The industry needed so many of these people so quickly that they couldn’t afford either the time or cost of going through a more heavyweight process like that so they said: we can get the job done with training and certification programs. It wasn’t perfect but it’s going to be much better than earlier solutions for the job at hand.

“That theme is still true today. An over-arching business driver is having qualified talent and using certification as a way to standardize things so that people have a certain level of confidence, skill and capability to support corporate users. It’s the cost factor, and even now we can see there are maybe a million open IT jobs so it’s important.”

But Carolyn recalls that not everyone agreed with the model originally. “There was internal resistance initially to using third parties,” she says. “But because we had the testing, we felt that’s a quality measure that we are comfortable with and it actually made more sense anyway. As long as you have the exams you can stand behind, that’s going to make a huge difference.”

Ensuring exam security

Another major concern at the beginning — and one that persists today — was test security.

Clarke says: “There was a misperception that putting tests on computers was not as secure as printing on paper. It was kind of ludicrous because you have to mail the paper or ship it in boxes and people can photocopy it. People were afraid of it so they put up false barriers.

“We had to show that computers were much safer because we put the test papers on the server, locked it in a room, connected the computers to it and used encryption to keep the data safe. Then everything went onto networks and people got a little bit more concerned about secure transmission of the data. But if you look at security, the cornerstone is really the invigilator, or proctor.

“They are at the center and they have to be trusted to have access to the system. But in reality the proctor, being human, can be the Achilles heel of your security strategy. That fear has always been the same. Now we select proctors carefully. We interview them, we have filters about how honest they are, we monitor them and certify them. We also check on them to make sure they are doing their job correctly. In the late ’90s we introduced statistical monitoring, looking for anomalies in the data that might point to a proctor or a group of proctors misbehaving.

“But it is foolish when considering any test, whether paper — or computer-based, to expect 100 percent security. There is no way you can stop people remembering questions, for example. But there is always a continuum of comfort you can have in security, reliability and validity.”

Carolyn adds: “Even today there are still pockets where people believe pencil-and-paper is somehow more secure. But many now see and understand the benefits.”

A new identity

Clarke left Prometric in 1994 and founded VUE.

“I was trying to help Clarke secure investment in VUE and I remember we were at dinner with a potential investor,” says Carolyn. “I turned to Clarke and said that we really need VUE’s services because we had a pinchpoint in the growth and you can address this.

“So Novell invested in VUE and I was on the board to look after our investment. I worked with Clarke to build out the VUE business.

“With Novell’s backing, VUE went into the testing business and brought competition to the industry, competing with Sylvan Prometric. NCS bought VUE in 1997 and Pearson bought NCS in 2000.”

Take your test anywhere?

The industry that Clarke and Carolyn shaped has not changed dramatically, but with the advent of smartphones and tablets, surely it is possible that test centers will disappear and candidates might sit tests online? Clarke disagrees.

“I bet it will not happen,” he says, “because there are going to be areas where you get economies of scale. I don’t think testing one person at a time is very scalable, unless you don’t care about proctoring. Clients could decide that they don’t care about proctoring and just put the test on the internet. Testing centers are cost effective when there is enough scale. I think we’ll always have testing centers because they make sense, but it’s clear that delivery will be augmented with remote delivery solutions — particularly in areas where we lack the scale to make test centers feasible.

“But the future is access everywhere. That’s been my vision since day one and we have been making steady progress towards it. We’ve gone from 100 test centers originally to hundreds, then via the internet thousands, now with remote or online proctoring we could go to infinite where the testing location could be anywhere. It’s all about having the right technology to support it. There are opportunities with smartphones, tablets, observational testing.

“Technology is enabling us. We had mainframes, then local area networks, then the internet. Technology unlocks more convenience, and this is balanced with cost.”

Carolyn believes that test centers may be replaced by remote or test-anywhere models in the future.

“With the proliferation of mobile devices, costs being driven down and security going up with biometrics now, it does seem likely,” she says. “I also think CBT will continue to grow and replace paper-and-pencil.”

Golden age ahead

Carolyn is proud of having been a pioneer. “Yes I love that,” she says, “It’s very gratifying. I travelled the whole world and met with a lot of businesses, instructors delivering the courses and students. Without exception there was a real sense of energy that we were all making a difference, touching people’s lives. When they became certified they could get a better paid job and contribute into their communities. It has an amazing ripple effect.

“Technology has reached out to people, touched them and changed their lives on a global basis. We had some resistance and pushback around the world. But it did become a universal program, and after we built the infrastructure it enabled Cisco, Microsoft and all the IT players that have certification programs to benefit and touch other people’s lives. Likewise, test owners across multiple sectors are now using that testing infrastructure for their professional high-stakes tests and licensures.

“What’s amazing to me is how much that is still in play today and that it hasn’t really changed so much. While Clarke and I get a lot of the credit for doing the work, it was absolutely a team effort and we all felt like this was our baby.

Clarke adds: “I don’t dwell on what we helped achieve, but certainly the certification industry has evolved to meet the needs of all — the vendor, the candidate and the employer. Professionals are more and more at the center of creating their own career paths, and it’s on us to provide them the tools to explore all their options and make the best choices about their jobs and paths.

“With remote proctoring, digital badging, and other new initiatives, we really are about to enter a golden age for professional IT certification.”