Concussion tests' marketing outpaces scientific evidence, new review says

Jacob Resch, director of the Brain Injury Laboratory at The University of Texas at Arlington, is lead author on the review, which updates a 2005 look at the available research on computerized neurocognitive testing. In 2005, researchers said not enough evidence existed to support clinical use of the then relatively new assessments.

The more recent work acknowledges that computerized tests, such as those marketed under the name ImPACT, HeadMinder, CogState, and ANAM, have become extremely commonplace across the sports world. But, the authors still urge caution with their use and point out a need for more peer-reviewed studies.

"Limited data has been published since 2005 to assist clinicians in determining the clinical value of this form of testing," Resch said. "While these products are an important component of concussion management, their development, marketing and sales seem to have outpaced the evidence. So, some caution is needed."

Co-authors are Michael McCrea, an author of the 2005 study and director of brain injury research at the Medical College of Wisconsin, and C. Munro Cullum, professor and head of the neuropsychology program at UT Southwestern Medical Center.

"Given the attention that concussion in sport has gained in recent years, it is surprising there has not been more research into the some of the newer computer-based methods used to evaluate post-concussion symptoms," Cullum said. "Since there is no single brain-test or biomarker for concussion at this point, the diagnosis of concussion remains a challenge in many cases, as it relies upon reported and observed symptoms."

Resch, McCrea and Cullum found 29 peer-reviewed articles since 2005 have addressed the characteristics of commercially available computerized neurocognitive tests. After a detailed analysis, they concluded that evidence on reliability and validity of the tests wasn't consistent.

For example, in a May 2013 study published by the Journal of Athletic Training, Resch and other researchers found that the ImPACT test misclassified healthy study participants as impaired as much as 46 percent of the time for some evaluation factors. ImPACT stands for Immediate Post Concussion Assessment and Cognitive Testing and is by far the most used computerized neurocognitive test for concussion management.

An Institute of Medicine report early this year said that the number of people 19 and under treated in U.S. emergency rooms for concussions and other non-fatal, sports and recreation-related traumatic brain injuries increased from 150,000 in 2001 to 250,000 in 2009. With the recent settlement of a landmark lawsuit filed by former NFL players, concussions remain in the headlines and on the minds of athletes, parents, coaches and others in the sports world.

Recently, nearly 40 percent of athletic trainers reported using a computerized neurocognitive tests as part of their response to a sports-related concussion, according to the new paper. The convenient computerized tests are typically used, just as pen and paper versions were in the past, to establish a baseline to use as a point of comparison after an athlete is injured. But, because concussion symptoms can vary widely across individuals and may be subtle, clear documentation of mild brain injury can be difficult.

The findings of the new survey should serve as a caution to those utilizing and interpreting computerized cognitive test results, the researchers said.

"Neurocognitive testing is an important component of the concussion assessment, but should not be used as a stand alone method to diagnose injury or determine an athlete's level of recovery and fitness to return to play," said McCrea. "A multi-dimensional approach is supported by the evidence as best practice."

Until more research is done, the new paper from Resch, McCrea and Cullum recommends healthcare professionals who use the tests take some precautions, including:

• Making an informed decision when choosing a test and investigating its limitations.
• Making sure the test is part of a multi-faceted concussion management approach.
• Incorporating a clinical neuropsychologist into the sports-related concussion management team to assist in interpretation of test results.
• Ensure proper training of those administering the test.
• Conducting quality control reviews of baseline testing.