1993 Report Summary

http://www.education.umn.edu/NCEO/OnlinePubs/SynthesisReport004.pdf

Currently, there is great variability in the accommodations enterprise. States issue reports on student performance. But it is difficult to make valid comparisons among states because of the lack of uniformity regarding the inclusion or exclusion of students with disabilities. Moreover, even when states include students with disabilities in their state assessment programs, there is eligibility for accommodations. Some states separate or identify scores that are achieved under nonstandard administrations while others do not.

On a national level, too, questions about acceptable accommodations of tests abound. How should eligibility for accommodations be determined? What modifications should be allowed? Should scores achieved under nonstandard conditions be flagged? How explicit should the description of nonstandard conditions be? At best, what currently exists is a lot of confusion and widespread variation in practice.

One of the results of the confusion and variation in practice is questions about what data and test scores mean. These questions plague all parts of the assessment and decision making process. The decision to exclude students from testing to boost scores (particularly when the practice is differentiated from one place to another) is a practice that has been identified as a "highly unethical" approach that produces test score pollution (Haladyna, Nolen, & Haas, 1991). Distinguishing this form of exclusion from others (e.g. exclusion to reduce stress to the student) is tricky at best.

Additional questions arise when decisions need to be made about the accommodations that will be allowed and provided, and eligibility for these accommodations. Questions continue to arise as test results are obtained. Do scores from accommodated tests have the same meaning as those from standard administrations? If there are differences in performance levels between groups, are these due to actual differences in the construct being measured or are they artifacts of modified testing procedures? For example, if a test purporting to measure reading comprehension is read aloud to a student who is blind, does the resulting score actually represent the student's reading comprehension ability or does it become a measure of listening comprehension? If it does become the latter, is that acceptable? An even less extreme adaptation, such as reformatting written text for students with learning disabilities so that sentences are not "broken" (i.e., continued from one line to another), raises issues about the construct validity of the resulting test score.

Answers to these questions are needed even more when data are used to make life decisions. What implications do assessment data have for performance in natural environments? For example, if, with limited time, a person with a disability can obtain an equivalent performance on a test to that of a person without disabilities, does this mean that an employer could expect equal job performance from the two employees?

The task confronting us now is to attempt to develop a comprehensive set of guidelines that state and national agencies can use in decision making. These guidelines need to address:

1. Inclusion/exclusion criteria

2. How and when to modify tests or testing procedures

3. How to report scores and summarize data (e.g., should scores obtained under nonstandard conditions be flagged in some way?)

Without some agreement about these guidelines, we can expect one or more of the following scenarios. First, we will continue to have confusion over policies, scores, and interpretation of data. This confusion will not end until practices are more consistent. Second, the trend toward modifying tests and testing procedures may be carried to absurdity. Test developers and administrators may come up with a multitude of modified versions of a test in an attempt to accommodate every individual need. At the extreme, no two people would get the same test. Third, requests for accommodations will get out of hand. In North Carolina, for example, when anyone who wanted a test accommodation could ask for one, they received several thousand requests when they had expected fewer than 100. Not only will the numbers of requests become overwhelming, but the nature of the requests may become absurd. One licensing organization (real estate) recently described one of the extreme requests it had received. A person with a certified fear of heights and water requested that the certification exam be administered on the first floor of a building located in a part of town that did not require the examinee to cross any bridges! Beyond theses scenarios is the inevitable increase in litigation. Already the courts have been forced to become the arbiters of policy on test accommodations. This raises the question of how appropriate or desirable it is for people who are not experts on issues of testing or disabilities to shape policy about them.

As it stands, it is likely that advocacy groups and assessment/measurement personnel will continue to do battle, a battle neither side can win. Parents of students with disabilities want their children included in testing for purposes of accountability. At the same time, however, they do not want their children included in taking tests that are going to be painful for them. What we must do is to develop a set of guidelines that will be fair to persons with disabilities while still maintaining a reasonable degree of integrity of tests and the interpretations drawn from them. This requires a delicate balancing of individual and societal rights.