Making it harder to 'outsmart' concussion tests

The study details a promising approach for pinpointing more athletes who play "impaired" on the Immediate Post-Concussion Assessment and Cognitive Testing, or ImPACT, a computerized tool consisting of eight subtests that gauge neurocognitive performance.

Administering ImPACT in the preseason helps establish a cognitive baseline that can be compared against the results of a post-concussion test, informing decisions about whether and when an athlete returns to action.

Concussions result from the brain slamming against the skull, usually causing short-term issues that some research suggests may evolve into long-term problems such as memory loss and depression when the brain is subjected to repeated trauma. To mitigate the risk of reinjury, athletes diagnosed with concussions take the ImPACT or a similar test to help determine when they have fully recovered.

But some athletes have undertaken the practice of sandbagging: giving lackadaisical effort on the baseline test to record a lower score in the hope of playing sooner after a concussion. Sandbagging can ruin the validity of the test and, because a recovering brain is more susceptible to further trauma, ultimately increase the likelihood of another concussion."At this point, people (administering) ImPACT may not have very much training in neuropsychological testing or standardized test administration or data interpretation," said lead author Kathryn Higgins, a postdoctoral researcher with the Center for Brain, Biology and Behavior at Nebraska. "If the baseline is the standard for when an athlete is recovered, there are all sorts of issues with returning someone to play based on poor baseline data."

So Higgins conducted an experiment to determine whether a statistical approach could identify more of the athletes who sandbagged on the baseline test. The experiment asked 54 athletes from rural Midwestern high schools to take the test twice, once while giving their best effort and once while subtly sandbagging. After analyzing the results, Higgins identified four subtests that created the largest disparity in scores. She then developed an equation that yielded a composite score from those subtests.

Establishing a threshold for the composite score allowed her to correctly find 100 percent of sandbagging cases while identifying the best-effort cases more than 90 percent of the time. Prior research suggests that ImPACT's existing system of validity checks, which flag suspicious scores on five individual subtests, detect just 65 to 70 percent of sandbaggers.

"Obviously, my flags are going to be better (in this case) because I built them and tested them on the same sample," said Higgins, who conducted the study as part of her dissertation. "But I thought it was worth pointing out that this equation has strong potential as another way to detect poor effort on baseline testing."

Higgins said she hopes further research will independently evaluate her approach and others that might improve the assessment of high school athletes, who suffer an estimated 300,000 sports-related concussions per year in the United States alone.

"There's so much room for work to be done," Higgins said. "We've come so far in the last 10 years -- we know so much more than we did -- but there are still a lot of things that we don't know."