Site Map

Conditions

Education

Behavior

Miscellaneous

Home

METHOD

In 1999 NCEO gathered, organized, and reported various aspects of state accommodation policies (Thurlow et al., 2000). Five main categories of accommodations were identified: presentation, presentation equipment, scheduling, response, and setting accommodations. State policy information was organized according to each of these main categories, and by each of the more frequently mentioned individual accommodations (read aloud, extended time, dictated response, etc.). The report also included information on states that allowed specific accommodations only under certain conditions (e.g., on certain test content areas, for specific grades of students, etc.). For the current synthesis, the number of states allowing accommodations both with and without such limitations was tallied. Information for the top 12 accommodations that are allowed in state policies is included in Table 1.

Table 1. Most Frequently Allowed Accommodations in State Policies

It is important to note that these are not necessarily the most frequently used accommodations, they are simply the most frequently allowed in state policies. Because two of the first three accommodations listed are not considered to be highly controversial (individual and small group administrations), and may actually be used during standardization procedures, these accommodations were deleted for purposes of this synthesis of accommodation research. For the remaining 10 accommodations, survey and empirical research was collected and summarized.

The Tindal and Fuchs (1999) report provided initial guidance on what research studies to include. Updated searches of ERIC were conducted to identify more recent research. Research on accommodations used for K-12 students in large-scale assessments was the primary focus. However, for several of the accommodations, limited research of this type was identified and so other research (accommodations for college students, intellectual assessments, etc.) was also included. The 10 accommodations are presented here in alphabetical order.

BRAILLE EDITION OF THE TEST

Braille editions of tests are developed to accommodate students with blindness or significant visual impairments. Sixty-three possible dot combinations that can be read tactually form the basis for the Braille code, which is now universally accepted as the graphic symbol for blind readers (Barraga, 1983).

Explanation

Academic tests are not designed to measure students’ sensory abilities. However, if students with blindness are required to take a regular-print version of the test, their performance will be more representative of their visual disability than of their academic abilities. By taking a Braille version of a test, students with blindness are able to demonstrate their true academic abilities. Phillips (1994) pointed out that providing accommodations to students with sensory deficits has traditionally been a common practice.

State Use

The Braille accommodation is allowed by 33 of the 48 states that have statewide assessments (Thurlow et al., 2000). Five additional states allow this accommodation with limitations. Some states allow the Braille accommodation only on certain assessments and others may not allow the accommodation simply because a Braille version of the test has not yet been developed. 

Empirical Research

Despite the fact that the Braille accommodation has been widely accepted, studies have shown that students using a Braille edition of a test may have trouble with certain types of items. Bennett, Rock, and Kaplan (1987) found that math items involving special symbols and tally systems were differentially difficult for students taking a Braille edition of the Stanford Achievement Test (SAT). In addition, Bennett, Rock, and Novatkoski (1989) found that differential item functioning existed for the Braille edition of the SAT math section, specifically among items that included figures in the stimulus and items for which spatial estimation was considered helpful in eliminating two of the options. Similarly, Coleman (1990) found that Braille readers had more difficulty with length measurement tasks than regular-print readers. A statewide stakeholder focus group in Texas suggested that tactual versions of print diagrams and pictures may convey very different information to students than visual diagrams and pictures, thus challenging the validity of the Braille version of the test. Although certain types of test items have been found to be more difficult for Braille readers, it appears that overall test scores are not greatly affected by this.

In addition to having difficulty with certain test items, students taking a Braille assessment may need extra time to complete the test. According to Phillips (1994), reading in Braille is a slower process than reading print materials. Braille shorthand can help speed up the process; however, Barraga (1983) suggests that extensive abbreviation can actually make it more difficult for students to recognize words in an unfamiliar context. Wright and Wendler (1994) found that the majority of students who were given twice the normal amount of time were able to finish the test. 

Controversy

The Braille accommodation is typically regarded as a change that maintains the validity of the test. There is little controversy about whether this accommodation should be allowed. However, even though the accommodation is allowed, the scores from Braille editions of a test often are not aggregated or reported in the same way as other scores. Analyses of test performance at the item level also suggest that certain types of items are differentially difficult for Braille readers.

Recommendations

• Braille assessments should be developed for tests that currently do not have a Braille edition and should be offered to students with severe visual impairments who use Braille.
• Items that are difficult to translate into a Braille version of the test should be avoided during initial item development (e.g., picture items, tally items, etc.).
• Students using a Braille version of the test should be given extended time to complete the test, perhaps as much as double the time.

COMPUTER/MACHINE RESPONSE

Students are using computers more than ever before. Similarly, computerized assessments are becoming more widely used. Because computerized assessments often involve both computer presentation and computer response, it is often difficult to disentangle the impact of the computer response mode on test administration.

Explanation

Many students complete writing assignments via computer. Students who are more familiar with completing assignments on computer may be more comfortable with completing a test in this mode. Because tests are intended to measure the outcomes of instruction, rationale for using this accommodation when students use computers in the classroom is clear. Also, many students with physical impairments that limit their abilities to respond with paper-and-pencil may not be able to demonstrate their true knowledge and skills unless a test is administered in an alternate format, such as via computer. Thus, computer/machine response has been studied for both writing assessments and assessments in other subject areas.

State Use

Of the 48 states with statewide assessment programs, 28 allow the computer/machine response accommodation. Six additional states allow it in some situations and prohibit it in others. States that limit its use commonly only allow this accommodation on certain assessments, or only allow it if all students in the classroom have the opportunity to take the test via computer (Thurlow et al., 2000).

Survey Research

Jayanthi, Epstein, Polloway, and Bursuck (1996) sent surveys to 708 general education teachers to obtain information on educators’ perceptions and use of a variety of accommodations. Results of the survey (401 respondents) indicated that of the 24 accommodations studied, allowing a word processor was somewhat difficult for elementary and middle school teachers, but a bit easier for high school teachers to provide. Overall, 8.3% of teachers surveyed reported using this accommodation, and teachers reported finding this accommodation somewhat helpful.

Empirical Research

Several studies have examined the use of computerized assessments for students in grades K-12. For this analysis, our primary focus was computer and word-processor response accommodations, rather than general computer administrations. Studies that examined student preferences related to the computer/machine response accommodations also are discussed.

Writing assessments. Eight studies were identified that examined the effects of having students complete writing tasks via computer. The majority of these studies involved middle-school students, and some included students with disabilities.

One of these studies showed students performing better under a computer response mode. Russell and Haney (1997) had two groups of middle school students complete an open-ended assessment on paper that involved questions in a variety of subject areas. Then, one group completed a performance writing assessment (which required extended written response) on paper, and one group completed it on computer. Paper-and-pencil test responses for the second test were then converted into computer responses. Results indicated that groups scored similarly on the open ended assessment, but that students using a computer for the performance writing assessment scored significantly better than those responding with paper-and-pencil (effect size = .94).

Other studies have suggested that positive effects of the computer response mode are mediated by student efficiency in keyboard use. In a later study, Russell (1999) again had middle school students answer open-ended test questions in different subject areas, some students using a paper-and-pencil format and others responding in the computer format. Results indicated that for students who keyboard at a speed of 20 words or more per minute, the paper-and-pencil format underestimated achievement level. For slower keyboarders, the computer response format underestimated achievement level. Similarly, Russell and Plati (2001) found that paper-and-pencil results severely underestimated the achievement of students in grades 8-10 who were accustomed to writing using a computer. This study used open-ended questions from the language arts section of the Massachusetts Comprehensive Assessment System (MCAS).

Hollenbeck, Tindal, Harniss, and Almond (1999) found no differences between the scores for seventh and eighth grade students composing stories for a statewide writing test in computer mode versus handwritten mode. In fact, students with disabilities performed significantly poorer when composing with computer than when handwriting their stories. Significant effects were found, however, for students who were allowed to use spell-checkers. It was not clear whether handwritten work was converted into word-processed work prior to evaluation in this study.

Several studies have looked at process related differences between handwritten and computer formats for writing. In a study by Vacc (1987), four eighth grade students (all certified as having mild mental handicaps) wrote letters in both computer and handwritten formats. Vacc found that writing letters on a microcomputer took more time, produced noticeably longer letters, and involved more revision than handwritten letters. The mean number of words written per unit of time was substantially higher for participants’ handwritten letters. Judges evaluations did not differ significantly between the two production modes.

MacArthur and Graham (1987) had fifth and sixth graders with learning disabilities compose and revise stories under three modes: handwritten, word processed, and dictation. Although the dictation condition resulted in significant differences, the handwritten and word-processed stories did not differ in terms of length, quality, story structure, mechanical or grammatical errors, or vocabulary. Word-processing was less than half as fast as handwriting. The overall amount of revision was similar across the handwritten and word-processed modes.

Another study looked at differences in how judges rated handwritten and word-processed essays for a statewide assessment (Hollenbeck, Tindal, Stieber, & Harniss, 1999). Middle school students completed the assessment in handwritten mode. Next, essays were word-processed, without changes in spelling or grammar. Original handwritten compositions were rated significantly higher than the typed composition on three of the six traits for the total group. Five of the six mean trait scores favored the handwritten essays. This study suggested that the two modes should not be used in the same evaluation system. Similarly, factor analysis studies of writing assignments corresponding to the Oregon statewide writing assessment found that when handwritten and word-processed essays were analyzed separately, all traits loaded on a single factor; however, when data from the two modes were analyzed together, two factors emerged: one for the handwritten traits, and one for the word-processed traits (Helwig, Stieber, Tindal, Hollenbeck, Heath, & Almond, 2000). Furthermore, low correlations were found for individual students’ ratings in computer and handwritten essays in this study.

Other computerized response assessments. Nine studies examining the use of computerized assessments for K-12 students in non-writing assessments were identified. Three of these studies involved assessment of writing-related skills (capitalization and spelling), and the others examined other skill areas (math, reading, vocabulary, etc.). Several of these studies demonstrated that students performed better under the paper-and-pencil format.

Watkins and Kush (1988) conducted a study in which elementary students with learning disabilities received a capitalization test under both a paper-and-pencil method and via computer (with conditions counterbalanced). During the computer administration, the students were presented items that measured particular capitalization objectives until the student reached a “mastery,” “non-mastery,” or “review” criteria for each objective. Analysis of the paper-and-pencil tests also involved coding student progress into one of the three categories. Results indicated that students scored higher on the paper-and-pencil test; however, the computer and paper-and-pencil tests did not significantly differ in their assignment of students to instructional interventions.

Varnhagan (1984) studied one group of students in special education and one group of third-grade regular education students who were both administered a spelling test in standard written format, and via computer. Results indicated that the written mode elicited higher scores than the computer format for both groups. Students generally took more time in completing the computer test. Contrary to this, Hasselbring and Crossland (1982) found the computer response format to be favored over the paper-and-pencil format on a spelling test for 9- to 14-year-old students with learning disabilities. The computer administration required less teacher administration time, and difficulties associated with deciphering student handwriting were avoided in the computer response format.

In a study by Swain (1997), third grade students (some with disabilities in math, some without) completed two math tests (Keymath and a computer-administered math test). Participants scored higher on all subtests of the paper-and-pencil test than on the computer-administered test. No interaction effects (mode of administration by disability status) were found.

Russell (1999) also found limited support for computer response accommodations on a math test. He found that regardless of keyboarding speed, middle school students’ performance on math assessments administered on a computer underestimated their level of achievement. He suggested that students may require scratch paper for many math items, which is difficult to provide in a computer administration.

Other studies have shown no differences according to assessment mode. Miller (1990) administered a receptive vocabulary screening test to students with cerebral palsy and students without disabilities. Students completed different forms of the test in both standard response mode and via computer (counterbalanced design). Results indicated that the two modes were equivalent for the groups of students. In the study by Russell and Haney (1997), in addition to completing a writing performance assessment, the two groups of students completed a multiple choice test in differing formats (computer and paper-and-pencil). While students who were administered the writing test via computer performed better on a writing performance task, the two groups did not significantly differ in performance on the multiple choice test.

Horton and Lovitt (1994) found mixed results in the area of reading. Their study involved having middle and high school students (some with and some without learning disabilities) complete reading inventories under two conditions: computer presentation and response, and paper-and-pencil administration. Results favored the computer mode on factual questions for the students overall; however, it appeared that middle school students with learning disabilities scored markedly better on social studies items in the paper-and-pencil format than in the computer format. On interpretive test items, the results of the group analysis showed no significant difference between the two assessment methods.

In addition to the Hasselbring and Crossland (1982) and Horton and Lovitt (1994) studies which provided some support for the computer response accommodation, one other study was identified that supported the use of this accommodation for students with disabilities. Burk (1999) studied the effects of a computer format versus a paper-and-pencil format for a variety of populations on multiple choice tests in multiple subject areas. Results indicated that students with learning disabilities performed significantly better on tests delivered via computer.

Student preferences. A number of the studies on the effects of computer accommodations also surveyed students’ perceptions of the computer administration. Hollenbeck et al. (1998) found that both students with and without disabilities tended to prefer the handwritten format for the writing assessment. Varnhagan (1984), Watkins and Kush (1988), and Horton and Lovitt (1984) all found students to prefer the computer format, despite their general findings that students performed better under the handwritten condition.

Overall, it appears that the computer/word-processor response accommodation has limited empirical support, especially for students with disabilities. The research review identified studies that both supported and discouraged its use, as well as studies that demonstrated no mode effect.

Controversy

Research findings indicate limited agreement about whether computer/machine response is a valid accommodation for students with disabilities. Teachers find it somewhat difficult to implement as an accommodation, and states are not unanimous in terms of allowing it. Also, it is clear that without necessary computer knowledge and typing skills, this accommodation may hinder student performance. However, with increasing use and access to computers, this accommodation may become more helpful and necessary for students. Furthermore, computer administrations can additionally offer easy access to other accommodations (e.g. computer read aloud, large print, etc.). Although this synthesis did not focus on the multiple accommodation possibilities available via computer, research studies have begun to analyze how several accommodations can be incorporated by using computers. Finally, if students cannot physically use a pencil to complete a test, it may make more sense for them to use a computer response format.

The fact that judges often rate word-processed essays lower than handwritten essays suggests that this accommodation may put students at a disadvantage unless there is appropriate training of scorers. Greater attention to these scoring implications will be needed if computer use in assessments increases as expected.

Recommendations

• Students must be provided the necessary typing and computer skills so that they are accustomed to using the computer/machine response accommodation. Without this support, caution should be taken in allowing a student to use the computer/machine response accommodation.
• This accommodation may also be relevant for students who physically cannot manipulate a pencil, and can more easily respond using a computer or machine. Of course, instruction in its use, and previous experience with computers are essential.
• Because studies have demonstrated the tendency to use different evaluation criteria for handwritten and computer versions, it is essential that evaluators first demonstrate the ability to use the same criteria before scoring assessments.
• When students use this accommodation, it is important to know that equipment is working properly. Also, students should frequently save their responses during testing to ensure that work is not inadvertently erased (CEC, 2000).

Site Map

Conditions

Education

Behavior

Miscellaneous

Home

Tourette Syndrome "Plus" © Copyright 1998 - 2005 Leslie E. Packer, PhD. except as noted.
All rights reserved 
This page last updated January 7, 2005. Reprint Policy. Privacy Policy. Contact.