Vanderbilt Researchers Seek To Make Standardized Tests Accessible

"This tool, the TAMI, should help all test developers systematically apply principles of universal design to advance the accessibility of tests for all students, not just students identified with disabilities. TAMI is helping test developers achieve their dual goals of better tests and better testing practices," Elliott, Dunn Family Professor of Education, director of the Learning Sciences Institute and director of the Interdisciplinary Program in Educational Psychology, said. "Teachers and test developers alike have told us that the TAMI advances their goals of better and more inclusive assessment for students with disabilities. These educators eagerly tell us that these students have learned significantly more than they had previously been able to show on less accessible tests."

“We define accessibility as ‘the extent to which an environment, product or service eliminates barriers and permits equal access to all components and services for all individuals’,” Beddow, a research assistant in special education and a member of the Learning Sciences Institute, said. “In the case of standardized testing, this means developing assessment tools that do not place students at a disadvantage because of difficulties with reading, comprehension or other problems when it comes to being able to understand the question posed and its corresponding answer set.”

In the seven years since No Child Left Behind began issuing mandates to public schools, high-stakes testing has taken on an increasingly larger role in students’ academic careers.  Nationally administered tests, like the SAT or ACT, are constantly evolving to meet the needs of both students and educational institutions and are still one of the best single indicators of future academic success. High-stakes tests being administered at the state level, however, are not always subjected to the same amount of scrutiny when it comes to addressing the needs of students with learning disabilities. Yet they are being used more and more for making important decisions regarding grade-level promotions, teacher performance bonuses and whether or not students graduate high school.

The researchers emphasize that they are not using TAMI to water down standardized tests, but instead to make the tests more appropriate to the students taking it, while ensuring that the knowledge being assessed and demonstrated is of the same depth. 

“We anticipated that there would be a need for a tool that could be used to analyze and modify test items for this purpose because of our involvement with a similar project, the Consortium for Alternate Assessment Validity and Experimental Studies (CAAVES),” Beddow said.

Recent regulations under NCLB permit states to create separate tests for students in special education who have demonstrated past difficulties with standardized assessments. These students must all have Individual Education Plans and need to have been identified as having a disability that is believed to be the cause of their failure to perform at expected grade level.

The researchers began by developing an item modification guide that was used to modify a set of questions given to students with and without identified disabilities, and in both their original and modified forms. They partnered with testing boards in Indiana, Idaho, Arizona and Hawaii to carry out their initial research.

“The findings largely confirmed our hypothesis,” Beddow said. “We were able to close the achievement gap with the regularly performing students by modifying the questions that might have been problematic for students with special needs.”

TAMI is the result of a thorough revision of this initial guide. It consists of two parts: an item analysis, which uses multiple categories with detailed rubrics to judge the actual questions, and a computer-based test analysis that can be used to assess the accessibility of a specific computer-based test delivery system. The item analysis takes into account factors like the clarity of the question’s wording, whether or not necessary visuals are included and the choice of wrong answers, which are the parts of standardized tests that can be problematic for students with special needs. Revising questions with an aim towards streamlining and simplifying unnecessarily complex questions is the focus of the item analysis.

“Modifying these types of test is not just a matter of ensuring computer literacy,” Beddow said. “It is also about making sure that the screen is legible, that answer selection is simple and intuitive, and that audio is available to those who need it. The goal of this part of TAMI is to make sure that the computers are not further complicating the test or altering the validity of the students’ responses.”

The instrument is currently available to be freely used by the public at http://peabody.vanderbilt.edu/TAMI.xml. It is being distributed to state assessment departments and testing companies in all 50 states. 

Research is continuing into how TAMI might be used to influence future assessments for all students.

Kettler is a research assistant professor of special education and a member of the Learning Sciences Institute.

The project was funded by an Enhanced Assessment Grant from the U.S. Department of Education in the Consortium for Alternate Assessment Validity and Experimental Studies.