Are accessibility and exam security mutually exclusive aims?
Although certification exam security is a primary concern in credentialing, assessments must also be accessible to individuals with disabilities and non-native English language speakers. Credentialing organizations receive frequent requests to use accommodations on exams. Some credentialing programs have blanket policies. Such a policy might be that “all candidates testing in non-English speaking countries who are taking an English version of an examination receive 30 additional minutes to complete the exam.”
Blanket policies can create legal and security issues, however, as not all candidates require the provided accommodation. Conversely, programs that lack clearly defined accommodation policies risk a constant need for assistance and allegations that the credentialing organization has not provided legally required accommodations.
Exam accessibility is of utmost importance to ensure valid scores; however, there are often questions about whether certain accommodations risk compromising exam security. Two vital components of ensuring exam score validity are exam security and providing accommodations to appropriate individuals.
Exam accessibility is legally required to allow all candidates who possess the required knowledge to earn a credential to demonstrate that knowledge. Accommodations are changes made to “standardized” conditions to enable individuals with needs and barriers to meaningfully access an exam. Accommodations can provide equitable access and ensure a valid measure of exam constructs, which enables individuals with disabilities and English learners to demonstrate their knowledge and skills.
Laws and Professional Standards
The Americans with Disabilities Act (ADA) determined that “reasonable” accommodations must be provided to individuals with disabilities. Technical assistance guidance from the U.S. Department of Justice in 2015 clarified that assessments must be offered in a manner that is accessible to individuals with disabilities. These exams include “exams administered by any private, state, or local government entity related to applications, licensing, certification, or credentialing for secondary or post-secondary education, professional, or trade purposes(.)” The guidance noted that the absence of previous formal testing accommodations does not preclude a person from receiving testing accommodations.
Additionally, recent action by the U.S. Equal Employment Opportunity Commission (EEOC) has affected special populations. The Commission began a public comment period on February 24, 2016, on a proposed rule related to Section 501 of the Rehabilitation Act of 1973 related to affirmative action in employment for individuals with disabilities. The current Section 501 regulations indicate that the federal government should be a “model employer of individuals with disabilities”; however, the guidelines regarding the specifics of this statement are insufficient.
The proposed rule would more clearly define this statement to include the percentage representation of individuals with disabilities across the pay levels (to include higher and lower levels of employment) to be employed by the federal government and contractors working on behalf of the federal government. The proposed rule would also require agencies to include disabilities in anti-harassment policies. This proposed rule could require more individuals with disabilities to seek certification in various areas; thus, it is critical to address this potential increase in the number of candidates requesting accommodations proactively.
Enyart v. National Conference of Bar Examiners (NCBE), 630 F.3d 1153 (9th Cir. 2011) addresses accessibility. Plaintiff Enyart is legally blind and requested specific assistive technology software (i.e., JAWS and ZoomText) from the NCBE. Although the NCBE denied Enyart’s specific accommodation, the NCBE offered other accommodations. NCBE argued that the “reasonable accommodation” principle applied to Enyart’s request; however, the court deemed that the Department of Justice’s higher standard of the “best ensure” principle applied. The ruling in this and similar cases illustrates that some courts are upholding the “best ensure” standard for the provision of accommodations.
In addition to federal laws and court cases, several sets of professional standards address the issue of accessibility. The 2014 revision of the Standards for Educational and Psychological Testing, published jointly by the American Educational Research Association, American Psychological Association, and National Council on Measurement in Education, indicated that comparability in the interpretation of scores from accommodated and non-accommodated exams is essential, and rests on clearly defining the intended construct to be measured.
Two primary sets of standards are specific to credentialing exams: the National Commission for Certifying Agencies (the accreditation division of the Institute for Credentialing Excellence [ICE]) and the American National Standards Institute (ANSI). ICE released a revised version of the Standards for the Accreditation of Certification Programs in 2014 for use beginning in 2016; these standards contain requirements for accommodations and accommodation requests. In 2004, ANSI published the Guidance on Psychometric Requirements (2004), which provides accreditation standards guidance related to ISO/IEC 17024 (Section 4.3.6).
These standards state, “A policy regarding the implications of testing irregularities, and a policy regarding accommodations, modifications, and adaptations, is essential. Guidelines are required for understanding the most likely effects of irregularities, accommodations, or changes upon the interpretation of resultant scores (p. 5).”
Lessons Learned from Educational Assessments
Little research has been conducted specific to licensure and certification exams on how to maintain test security while providing accessibility. Therefore, it is useful to look at the educational assessment policy analysis that the National Center on Educational Outcomes (NCEO) conducted in 2014. (The complete analysis is available here.)
At the K-12 level, exam security policies often address how to administer accommodations without compromising exam security. These policies can provide insights into how certification exam policies might be structured. NCEO found that the most common accommodations addressed in test security policies were braille, transcription and scribing, read aloud by a human reader, and sign language interpretation. Slightly fewer than half of the states included extended time in their policies. The analysis also found that the policies of about half the states addressed training requirements for accommodations providers and administrators of accommodated tests.
The answer to “What should I do to ensure that I have a method to grant accommodation requests appropriately and maintain the security of my exam?” is straight-forward: have policies and processes in place to handle accommodation requests, and ensure the delivery vendor can provide accommodations in a secure environment. All test administrators, accommodations providers, and other parties involved in preparing and administrating accommodated assessments should be required to sign test security and confidentiality agreements to help improve test security.
Additionally, coordinate with the delivery vendor to ensure that common accommodation software is available on machines in accessible locations; this software must maintain appropriate functionality without modifying other settings (i.e., accessing the Internet). In cases where a human access assistant is required, make sure that this assistant is appropriately trained, has signed relevant documents, and understands the effects of compromising item security and modifying items during delivery.
Develop accommodation policies during exam development and refine them throughout the exam lifecycle. Carefully consider the items and structure, types of accommodations likely to be requested/needed, and implementation of accommodations without sacrificing security or the intended meaning of exam scores. Consult content and accommodation/disability experts. Content experts evaluate construct changes as a result of an accommodation. Disability/accommodation experts document best practices for incorporating appropriate accommodations without risking exam security.
Lessons learned from educational assessment test security policies, as well as the literature, can provide some basic guidelines for including accommodations in policies. These lessons help to ensure security in many ways; one of the most overarching ways is a shared understanding of the role of accommodations within assessment. Once the need for accommodations is understood by the test owner, it is easy to apply security policies that work in coordination with the accommodation policies. Some of the lessons include:
Publish requirements regarding accommodations (e.g., dates, required materials, and policies for challenging decisions) in an area available to all eligible candidates.
Standardize the request and review process.
Document the accommodation request review process (e.g., required materials/information, inappropriate/irrelevant information, communication of decisions, standardized application to all requests).
Log all requests and review the logs with experts to refine policies/procedures.
Individually review and process all requests within the defined accommodations policies. All candidates should be treated the same. However, blanket policies should be avoided because blanket policies could include individuals who do not require a particular accommodation.