Machine learning model helps forecasters improve confidence in storm prediction

Over the last several years, Russ Schumacher, professor in the Department of Atmospheric Science and Colorado State Climatologist, has led a team developing a sophisticated machine learning model for advancing skillful prediction of hazardous weather across the continental United States. First trained on historical records of excessive rainfall, the model is now smart enough to make accurate predictions of events like tornadoes and hail four to eight days in advance -- the crucial sweet spot for forecasters to get information out to the public so they can prepare. The model is called CSU-MLP, or Colorado State University-Machine Learning Probabilities.

Led by research scientist Aaron Hill, who has worked on refining the model for the last two-plus years, the team recently published their medium-range (four to eight days) forecasting ability in the American Meteorological Society journal Weather and Forecasting.

Working with Storm Prediction Center forecasters

The researchers have now teamed with forecasters at the national Storm Prediction Center in Norman, Oklahoma, to test the model and refine it based on practical considerations from actual weather forecasters. The tool is not a stand-in for the invaluable skill of human forecasters, but rather provides an agnostic, confidence-boosting measure to help forecasters decide whether to issue public warnings about potential weather.

"Our statistical models can benefit operational forecasters as a guidance product, not as a replacement," Hill said.

Israel Jirak, M.S. '02, Ph.D. '05, is science and operations officer at the Storm Prediction Center and co-author of the paper. He called the collaboration with the CSU team "a very successful research-to-operations project."

"They have developed probabilistic machine learning-based severe weather guidance that is statistically reliable and skillful while also being practically useful for forecasters," Jirak said. The forecasters in Oklahoma are using the CSU guidance product daily, particularly when they need to issue medium-range severe weather outlooks.

Nine years of historical weather data

The model is trained on a very large dataset containing about nine years of detailed historical weather observations over the continental U.S. These data are combined with meteorological retrospective forecasts, which are model "re-forecasts" created from outcomes of past weather events. The CSU researchers pulled the environmental factors from those model forecasts and associated them with past events of severe weather like tornadoes and hail. The result is a model that can run in real time with current weather events and produce a probability of those types of hazards with a four- to eight-day lead time, based on current environmental factors like temperature and wind.

Ph.D. student Allie Mazurek is working on the project and is seeking to understand which atmospheric data inputs are the most important to the model's predictive capabilities. "If we can better decompose how the model is making its predictions, we can hopefully better diagnose why the model's predictions are good or bad during certain weather setups," she said.

Hill and Mazurek are working to make the model not only more accurate, but also more understandable and transparent for the forecasters using it.

For Hill, it's most gratifying to know that years of work refining the machine learning tool are now making a difference in a public, operational setting.

"I love fundamental research. I love understanding new things about our atmosphere. But having a system that is providing improved warnings and improved messaging around the threat of severe weather is extremely rewarding," Hill said.