Climate scientists put predictions to the test

Published in the Journal of Geophysical Research-Atmospheres, the study is one of the first to systematically address a longstanding, fundamental question asked not only by climate scientists and weather forecasters, but the public as well: How good are Earth system models at predicting the surface air temperature trend at different geographical and time scales?

Xubin Zeng, a professor in the University of Arizona department of atmospheric sciences who leads a research group evaluating and developing climate models, said the goal of the study was to bridge the communities of climate scientists and weather forecasters, who sometimes disagree with respect to climate change.

According to Zeng, who directs the UA Climate Dynamics and Hydrometeorology Center, the weather forecasting community has demonstrated skill and progress in predicting the weather up to about two weeks into the future, whereas the track record has remained less clear in the climate science community tasked with identifying long-term trends for the global climate.

"Without such a track record, how can the community trust the climate projections we make for the future?" said Zeng, who serves on the Board on Atmospheric Sciences and Climate of the National Academies and the Executive Committee of the American Meteorological Society. "Our results show that actually both sides' arguments are valid to a certain degree."

"Climate scientists are correct because we do show that on the continental scale, and for time scales of three decades or more, climate models indeed show predictive skills. But when it comes to predicting the climate for a certain area over the next 10 or 20 years, our models can't do it."

To test how accurately various computer-based climate prediction models can turn data into predictions, Zeng's group used the "hindcast" approach.

"Ideally, you would use the models to make predictions now, and then come back in say, 40 years and see how the predictions compare to the actual climate at that time," said Zeng. "But obviously we can't wait that long. Policymakers need information to make decisions now, which in turn will affect the climate 40 years from now."

Zeng's group evaluated seven computer simulation models used to compile the reports that the Intergovernmental Panel on Climate Change, or IPCC, issues every six years. The researchers fed them historical climate records and compared their results to the actual climate change observed between then and now.

"We wanted to know at what scales are the climate models the IPCC uses reliable," said Koichi Sakaguchi, a doctoral student in Zeng's group who led the study. "These models considered the interactions between the Earth's surface and atmosphere in both hemispheres, across all continents and oceans and how they are coupled."

Zeng said the study should help the community establish a track record whose accuracy in predicting future climate trends can be assessed as more comprehensive climate data become available.

"Our goal was to provide climate modeling centers across the world with a baseline they can use every year as they go forward," Zeng added. "It is important to keep in mind that we talk about climate hindcast starting from 1880. Today, we have much more observational data. If you start your prediction from today for the next 30 years, you might have a higher prediction skill, even though that hasn't been proven yet."

The skill of a climate model depends on three criteria at a minimum, Zeng explained. The model has to use reliable data, its prediction must be better than a prediction based on chance, and its prediction must be closer to reality than a prediction that only considers the internal climate variability of the Earth system and ignores processes such as variations in solar activity, volcanic eruptions, greenhouse gas emissions from fossil fuel burning and land-use change, for example urbanization and deforestation.

"If a model doesn't meet those three criteria, it can still predict something but it cannot claim to have skill," Zeng said.

According to Zeng, global temperatures have increased in the past century by about 1.4 degrees Fahrenheit or 0.8 degrees Celsius on average. Barring any efforts to curb global warming from greenhouse gas emissions, the temperatures could further increase by about 4.5 degrees Fahrenheit (2.5 degrees Celsius) or more by the end of the 21st century based on these climate models.

"The scientific community is pushing policymakers to avoid the increase of temperatures by more than 2 degrees Celsius because we feel that once this threshold is crossed, global warming could be damaging to many regions," he said.

Zeng said that climate models represent the current understanding of the factors influencing climate, and then translate those factors into computer code and integrate their interactions into the future.

"The models include most of the things we know," he explained, "such as wind, solar radiation, turbulence mixing in the atmosphere, clouds, precipitation and aerosols, which are tiny particles suspended in the air, surface moisture and ocean currents."

Zeng described how the group did the analysis: "With any given model, we evaluated climate predictions from 1900 into the future -- 10 years, 20 years, 30 years, 40 years, 50 years. Then we did the same starting in 1901, then 1902 and so forth, and applied statistics to the results."

Climate models divide the Earth into grid boxes whose size determines its spatial resolution. According to Zeng, state of the art is about one degree, equaling about 60 miles (100 kilometers).

"There has to be a simplification because if you look outside the window, you realize you don't typically have a cloud cover that measures 60 miles by 60 miles. The models cannot reflect that kind of resolution. That's why we have all those uncertainties in climate prediction."

"Our analysis confirmed what we expected from last IPCC report in 2007," said Sakaguchi. "Those climate models are believed to be of good skill on large scales, for example predicting temperature trends over several decades, and we confirmed that by showing that the models work well for time spans longer than 30 years and across geographical scales spanning 30 degrees or more."

The scientists pointed out that although the IPCC issues a new report every six years, they didn't see much change with regard to the prediction skill of the different models.

"The IPCC process is driven by international agreements and politics," Zeng said. "But in science, we are not expected to make major progress in just six years. We have made a lot of progress in understanding certain processes, for example airborne dust and other small particles emitted from surface, either through human activity or through natural sources into the air. But climate and the Earth system still are extremely complex. Better understanding doesn't necessarily translate into better skill in a short time."

"Once you go into details, you realize that for some decades, models are doing a much better job than for some other decades. That is because our models are only as good as our understanding of the natural processes, and there is a lot we don't understand."

Michael Brunke, a graduate student in Zeng's group who focused on ocean-atmosphere interactions, co-authored the study, which is titled "The Hindcast Skill of the CMIP Ensembles for the Surface Air Temperature Trend."