Google Flu Trends estimates off, study finds

The findings were reported at the ATS 2010 International Conference in New Orleans.

"We knew from the Google Flu Trends validation study that it is highly correlated with surveillance for the non-specific syndrome of influenza-like illness," said Justin Ortiz, M.D., clinical fellow at the University of Washington who led the study. "However, it has never been evaluated against a gold standard of actual laboratory tests positive for influenza virus infection. When we compared Google Flu Trends data to CDC's national surveillance for influenza laboratory tests positive, we found that Google Flu Trends was 25 percent less accurate at estimating rates of laboratory confirmed influenza virus infection."

Google Flu Trends uses the popularity of certain Google search queries in real time to estimate nationwide rates of influenza-like illness activity, a non-specific combination of symptoms including a fever with either a cough or a sore throat without any confirmatory laboratory testing. While some traditional flu surveillance systems may take days or weeks to collect and release data, Google search queries can be counted almost instantaneously.

The problem is that studies have shown that influenza-like illnesses are actually caused by the influenza virus in only 20 percent to 70 percent of cases during the influenza season.

"Many respiratory virus infections other than influenza can result in influenza-like illness. Furthermore, there is a wide and unpredictable proportion of influenza-like illness that is due to actual influenza virus," said Dr. Ortiz. "Because Google Flu Trends estimates of influenza-like illness may not necessarily correlate with actual influenza virus infections, we undertook this study to evaluate the validity of Google Flu Trends influenza surveillance by comparing it to a gold standard of CDC's national surveillance for influenza laboratory tests positive."

The researchers analyzed the incidence of flu outbreaks in the United States between 2003 to 2008. They found that Google Flu Trends deviated greatest from CDC surveillance for influenza virus tests positive during the 2003-04 influenza season, a year notable for early and intense influenza activity, with a high number of pediatric influenza deaths and substantial media attention to influenza.

"Internet search behavior is likely different during anomalous seasons such as during 2003-4," explained Dr. Ortiz. "We hypothesize that during periods of intense media interest or unexpected influenza activity such as the 2009 H1N1 influenza pandemic, Google Flu Trends may be least accurate at estimating influenza activity."

"Still, Google Flu Trends influenza surveillance provides an excellent public health service, because it provides nationwide influenza activity data in a cheap and timely manner," said Dr. Ortiz. "Nevertheless, our study demonstrates that its data should be interpreted with caution and that other surveillance systems more accurately reflect influenza activity in the United States."