Bad weather warnings most effective if probability included, new research suggests

In their new paper, "The Cry Wolf Effect and Weather-Related Decision Making," Jared LeClerc and Susan Joslyn of the University of Washington examine the impact of false alarms on the participants' willingness to take precautions when authorities issue warnings and urge action. The authors note that excellent U.S. National Weather Service forecasts with substantial lead times have not prevented "massive loss of life" because the public too often has failed to take warnings seriously. Most researchers now recognize that psychological factors are a big part of the problem of public noncompliance with warnings. The LeClerc and Joslyn paper recently appeared in the online version of Risk Analysis, a publication of the Society for Risk Analysis.

The researchers point out that complicated factors govern people's reluctance to heed advice. When urged to evacuate their homes, for example, residents have cited concerns about travel expenses, dangers on the highway, inconvenience, and property losses from looting. However, "lack of trust in the warning" also may limit the public's willingness to comply with orders. Although the impact of false alarms -- or the "cry wolf" effect -- has been widely discussed, evidence remains unclear about how people's experience of false alarms affects their trust in weather warnings.

To understand better the psychology of this increasingly critical topic of the public's trust in and compliance with warnings, LeClerc and Joslyn designed a controlled laboratory experiment in which they could "systematically manipulate the rate and degree of prior exposure to false alarms" to observe the effects on participants' trust in the warnings and decisions about taking recommended actions. A key question was whether increasing the number of false alarms would decrease the willingness of experiment participants to take precautionary actions.

The experiment involved 354 University of Washington psychology students who participated for course credit and an opportunity to earn prize money. Participants played the role of a road maintenance company under contract to apply salt brine to roads in a U.S. town to prevent icing. Applying salt brine cost $1,000 a day, but failure to salt when freezing temperatures occurred resulted in a $5,000 penalty for the company. The experiment used historical forecast data from the cities of Spokane and Yakima in Washington State on the ranges of temperature, probabilities of freezing and forecast error. Participants were given a virtual monthly budget of $35,000 and were instructed to maximize profits by minimizing salting expenses and avoiding penalties. Salting had to occur before freezing temperatures were reached. Some participants received the advice of a decision support aid that recommended for or against salting based on whether the probability of freezing was above or below a certain threshold. Participants registered their trust in nighttime low temperature forecasts on a six-point scale, ranging from "not at all" to "completely."

The results provided "the first evidence of which we are aware for a significant cry wolf effect in weather-related decision making using a controlled experimental approach," even if the effect size was moderate, the authors conclude. Overall, when nine more false alarms were issued (38 out of 56 total salting recommendations, or 68 percent) experiment participants were "less likely to follow advice." The nine additional false alarms were compared with normative false alarm levels (29 out of 45 total salting recommendations, or 64 percent). The participants also trusted warnings less and "made economically inferior decisions" that reduced their virtual earnings, according to the researchers.

"The bottom line here is that false alarms may indeed be a subtle contributing factor to noncompliance with weather warnings," the authors state. Most significantly, according to the authors, adding a probability of freezing to the forecast "led to greater compliance with the advice and greater increase in decision quality than did lowering the false alarm level" and the implications of that finding are important for warning situations.