Officials leading hurricane response need ‘risk literacy’

New research suggests that emergency management officials often do not have the numeracy skills needed to make the best decisions based on data they receive about which residents to evacuate during a hurricane and when to make the decision.

The study, published online Aug. 30 in the Bulletin of the American Meteorological Society, showed that the most numerate officials were almost twice as likely as less numerate ones to provide additional evacuation times to their coastal communities. Less numerate ones, on the other hand, gave their communities less advance warning, and when they finally did issue evacuations, over-evacuated tens of thousands more people.

And, the study found, federal agencies need to supply emergency management officials with the most complete information available in order for them to make the best choices for people who live in a disaster's path.

This is not to say emergency management officials are not good at math, said Noah Dormady, lead author of the study and an associate professor of public policy at The Ohio State University.

"It's more they haven't received the right training in probability and risk to effectively understand scientific forecasts that contain probability information," Dormady said.

The study was based on a pair of experiments designed to study and evaluate officials' decision-making during a natural disaster. The study is among the first to evaluate evacuation decision-making by emergency management officials, rather than by individuals deciding whether or not to leave an area because of a storm.

The study involved 81 emergency managers and other public safety officials predominantly from coastal states that are affected by hurricanes, as well as 227 Ohio State graduate and upper-division students from related fields of study. The researchers began their experiments by testing each subject's ability to make decisions using probability and statistics -- a sort of baseline math test that allowed them to evaluate each participant's numeracy. In other words, how well they understood the ways probability and statistics might play out in the real world.

Researchers then provided the study participants with a scenario based on a real storm -- Hurricane Rita in 2005. Rita was among the strongest hurricanes ever recorded in the Gulf of Mexico, hitting land near the Louisiana-Texas border, killing 120 people and causing an estimated $18.5 billion in damages. Participants were not told the scenario was based on Rita, so they could not use any knowledge of the storm to guide their decisions.

Participants were randomly assigned to treatments with varying amounts of information about the approaching storm. Some subjects received a great deal of information, including the forecasted track and potential alternative tracks; others received limited information.

The researchers asked study participants to determine whether to evacuate, when to evacuate and whom to evacuate. Decisions were structured to coincide with the release of advisories from the National Hurricane Center (NHC).

Basing the experiments on a real disaster allowed researchers to determine which choices were "good." Because the outcome was known to the researchers, they could evaluate the areas affected -- those that either under-evacuated or over-evacuated regions.

The researchers found that people who scored well on the baseline math test made better choices on behalf of the community. People with higher numeracy issued evacuation orders that gave people more time to leave the storm's path. People who have the practical ability to evaluate and understand risk ordered evacuations in the simulated disaster about nine hours earlier than people who did not have that skillset, the study found.

"This tells us that public trust in evacuation orders, which has been waning for years, could be improved if the population had greater confidence in their emergency management leaders' ability to make sound decisions when they are given probabilistic forecasts," Dormady said. "People need to trust that their neighborhood is not being over-evacuated out of an abundance of caution simply because public officials didn't understand the risk."

The study also found that officials who had the most complete set of forecast data were more likely to issue evacuation orders earlier, giving people in the hurricane's path more time to get out of town than they would have had otherwise. That additional time was significant, the study found, adding between 16.6 and 22.8 hours to a community's evacuation time.

Those hours, Dormady said, could be the difference between life and death for people in a storm's path, and could also make evacuations, which are often a safety risk themselves, more safe. In fact, in some major disasters, more people die from poorly administered evacuations than from the catastrophic event itself.

"Extra hours are crucial -- they give people more time to pack and prepare, they give emergency management officials more time to communicate the importance of evacuating, and they make evacuations, overall, more safe," said Dormady, "And what we saw is that more complete forecast information gave emergency management officials the tools they needed to make better choices for their communities."

Co-authors of this study include Anthony Fasano, Drew Flanagan and William Welch at Ohio State, Alfredo Roa-Henriquez at Johns Hopkins University, and Dylan Wood at the University of Notre Dame.

This work was funded by the National Science Foundation.