How good is your hospital?

In a paper published July 2 in the American Journal of Medical Quality, the authors first cited research showing that despite almost two decades of public reporting of quality metrics, consumers have found hospital rating systems "to be limited and lacking in personalization or relevance for individual consumers." This lack of consumer engagement, the authors suggest, is likely driven by the substantial variability that exists between the ranking of top performing hospitals in different ranking systems: The U.S. News & World Report Best Hospitals List, the Vizient Quality and Accountability Study, the Centers for Medicare & Medicaid Services (CMS) Star Rating, the Leapfrog Hospital Safety Grade, and the Truven Top 100 Hospitals list.

Lead author Dr. Bala Hota, the Medical Center's chief analytics officer, noted that while each of the rating organizations provides valuable data and insight that help drive hospital quality improvement efforts, their complexity and variability have made them difficult for consumers to use.

"The science behind each rating systems is very complex and measures different outcomes, domains and even time periods," Hota said. "And while this wealth of data supporting the ratings is vital to hospitals, consumers are confused when the ratings disagree."

Thus nearly two years ago, Hota and his Rush colleagues began gathering the data and assembling an objective framework to needed assess the overall similarity of rating systems to one another. The paper, "Disagreement Between Hospital Rating Systems: Measuring the Correlation of Multiple Benchmarks and Developing a Quality Composite Rank" details how they aggregated scoring data from multiple hospital ranking systems to generate a single measure, the Quality Composite Rank (QCR).

For the study, the scores for 70 high-performing hospitals ranked by the various raking systems were combined into a core data set of ten performance measures. Using a series of statistical correlation approaches that accounted for differences and similarities in what each rating organization measured, researchers were able to better identify variations and ultimately generate a single digit composite score that rewards hospitals for consistency across ratings systems.

"Standardizing what is measured more objectively identifies hospitals that do well in multiple measurement systems. Hospitals with the best QCR scores had higher quality scores across more areas and measured by more scoring systems. We believe that suggests a more sustained and institutional commitment to quality care," Hota said.

More importantly, the authors believe a single-digit QCR composite score built from the various ratings systems will benefit patients.

"The most important metrics are those that help patients navigate the health system. But publicly-reported quality measures that the public does not understand defeats their purpose," said Omar Lateef, DO, Rush University Medical Center chief executive officer and paper co-author.

"When patients see conflicting ratings, they must then reconcile that information in their mind. What we've done is to develop a measure that quantitatively does that reconciliation."

The paper's other co-authors are Thomas Webb, manager of quality improvement, Avanthi Chatrathi, MPH, and Elizabeth McAninch, MD