It’s a Standards World

Posted on
Share on Google+Share on LinkedInShare on FacebookShare on RedditTweet about this on TwitterEmail this to someone

The other morning I went into my garage to find an inch of water on the floor. A bit of research led me to the cause: a busted sprinkler head. The pressure shot the water high into the air, coming down the wall of the garage and finding a way in. It wasn’t tough to fix. I got a threaded PVC pipe from the hardware store, removed the busted piece and screwed the new one in. It didn’t strike me at the time how difficult the procedure would have been had I not been able to find a ½-inch PVC pipe with the correct threads. Of course, that would never happen, since all ½-inch PVC pipes and fittings, in the United States anyway, have the same threads.

You live in a standards world, if you are lucky. We don’t think twice about it, but what would happen if you couldn’t be sure that threads would be the same? How many separate sets of spare parts would you need to have around? How much more time and money would be needed?

I am currently involved with the American National Standards Institute (ANSI; on a committee that reviews the applications of organizations with certification programs. These organizations apply to have their certification programs accredited by ANSI to demonstrate that they follow national and international standards for such programs. The standards cover a wide variety of aspects of a certification program, from establishing best practices to the development and administration of tests.

While ANSI’s standards for certification programs follow the template set forth by the recently developed ISO Standard 17024 for certification programs, ANSI’s history goes back to 1911, when, in an earlier incarnation, it began an “effort to avoid duplication, waste and conflict.” It was during its early years that it actually developed the standard for pipe threads! And along the way it developed new standards needed to increase efficiency for war production. The standards covered quality control, safety, photographic supplies, equipment components for military and civilian radio, fasteners and other products.

So why is this discussion of standards important? The discussion may not be, but standards certainly are. And standards in test development and delivery are critical for the success of IT certification. When applied, they direct the development of secure, valid and reliable tests, qualities necessary for good certification decisions. Without standards, poor tests would surely result, and it would be impossible to pass or fail individuals, give a meaningful score or certify someone.

Unfortunately, that happens in some certification programs. Smaller ones usually cut corners, wanting to create a good test, but not knowing how or not wanting to take the proper steps (which usually means they do not desire or are unable to spend the money) to make a good test. What happens is predictable: The tests are bad, certification decisions are arbitrary, and no one is pleased with the result.

What are the standards that can be used to create IT certification tests? Well, they are the same for any tests. They are known as the National Standards on Psychological and Educational Testing, jointly produced by the American Psychological Association, the American Educational Research Association and the National Council on Measurement in Education. The latest revision of the standards occurred in 1999. While I have no space to cover any of the standards, I have reprinted the table of contents, which shows the overall range of topics covered.

Like baking a cake, or fixing sprinklers, creating a good test is a matter of following the rules set down over time by experience. These rules, called standards, are helpful, especially in the growing and developing industry known as IT certification.

David Foster, Ph.D., is a member of the International Test Commission and sits on several measurement industry boards.


Table of Contents

Part I: Test Construction, Evaluation and Documentation

      1. Validity


      2. Reliability and Errors of Measurement


      3. Test Development and Revision


      4. Scales, Norms and Score Comparability


      5. Test Administration, Scoring and Reporting


    6. Supporting Documentation for Tests


Part II: Fairness in Testing

      7. Fairness in Testing and Test Use


      8. The Rights and Responsibilities of Test Takers


      9. Testing Individuals of Diverse Linguistic Backgrounds


    10. Testing Individuals with Disabilities


Part III: Testing Applications

      11. The Responsibilities of Test Users


      12. Psychological Testing and Assessment


      13. Educational Testing and Assessment


      14. Testing in Employment and Credentialing


    15. Testing in Program Evaluation and Public Policy



Share on Google+Share on LinkedInShare on FacebookShare on RedditTweet about this on TwitterEmail this to someone


Posted in Archive|