Patient beware: Researchers diagnose crowdsourced hospital ratings

Researchers Victoria Perez and Seth Freedman of IU's School of Public and Environmental Affairs compared social media ratings offered by patients with the extensive data available through the federal government's "Hospital Compare" website.

Their key findings:

• On patient experience -- food, friendliness, amenities -- the Google, Yelp and Facebook ratings most often aligned with hospitals that are highly ranked by Hospital Compare for patient experience based on surveys.
• On quality of care and safety as measured on Hospital Compare, the Google, Yelp and Facebook ratings were not as accurate. In fact, 20 percent of the hospitals rated "best" within a local market on social media were rated "worst" in that market by Hospital Compare on patient health outcomes.

"Our results indicate that crowdsourced ratings reflect measures of quality most easily observed, which is not all that matters in health care," Perez said. "While crowdsourced sites may provide similar information to the government's patient experience surveys, they are not a substitute for measures of clinical quality or patient safety."

The researchers acknowledge that finding that information can be a challenge. Their research highlights shortcomings with the Hospital Compare scores and a need to communicate clinical quality more clearly to patients. Of the 57 Hospital Compare metrics, patients must wade through 46 to determine clinical quality and safety. Many may not apply to the specific condition for which they are seeking care.

"For patients broadly interested in the dimensions of clinical quality and safety, our research shows the need for better tools to help patients search for hospitals that meet their clinical needs," Freedman said.

While the Hospital Compare ratings have been available since 2005, only recently has it been possible to make a comparison with crowdsourcing sites. Because of the rapid growth of social media, 90 percent of hospitals now show up on Hospital Compare and on the crowdsourced sites.

"Our study establishes how the 'best' and 'worst' rankings in a hospital market depend on where a patient goes for information," Perez said. "While 5-star ratings are easy to understand, our research shows patients should think twice before using it as a single source of information for a life-changing decision about hospital care."

An in-depth article about the research, "Do Crowdsourced Hospital Ratings Coincide with Hospital Compare Measures of Clinical and Nonclinical Quality?" was published in the peer-reviewed journal Health Services Research.