Measuring the cost of quality measurement

So how should quality measures be prioritized? Many factors are currently considered, including a measure's expected effect on patients and health care, potential for promoting improvement, scientific underpinnings, usability, and feasibility. But there is a major omission from this list: the cost of each measure. The cost of specific measures has received limited attention in discussions about global costs of quality measurement and is not formally considered when evaluating and selecting measures, in no small part because that cost is usually unknown. Without understanding the cost of a specific measure, assessing its value cannot be fully determined.

Major organizations and individual hospitals are not including costs in decisions about which measures to use. For example, the National Quality Forum, which vets many measures used by health care organizations, does not require those who develop measures to report cost data. Estimated costs are generally unavailable when choosing among measures, and processes for analyzing the burden of measures are inconsistently documented and rarely transparent. Limited information is available about whether and how the Centers for Medicare & Medicaid Services (CMS) factors the cost of measures into Hospital Value-Based Purchasing; if it does, these cost estimates are not publicly reported. The Office of Management and Budget estimates costs for measurement initiatives, but these estimates focus on the annual burden of entire measurement and reporting programs (eg, Physician Quality Reporting System) rather than the burden of individual measures or the burden for individual institutions. When selecting measures, hospitals and clinicians have even less access to cost information than these larger institutions. Negotiations between hospitals and insurers often happen without either side having information about costs of measures included in their risk-based contracts.

Although collective costs appear to be substantial, in reality, little is known about the cost of collecting and analyzing data and interpreting results for particular measures. Cost estimates are needed for individual measures, as well as standards for the units, timeframe, and other variables needed for valid cost comparisons across measures. Organizations endorsing measures should include cost estimates in measure descriptions. To start, these organizations could set a deadline after which measure submissions must incorporate cost information. They might create a pilot program to devise and test standard specifications for cost information and to develop methodologies for collecting cost data and estimating costs. Even general estimates could inform measure selection, and the science of cost estimation would likely improve rapidly if measuring cost became a routine component of measure development.

Measurement costs are likely not trivial. They include both fixed costs associated with implementing a quality measurement infrastructure and measure-specific costs, which can vary substantially across measures and often depend on local measurement capacity and simultaneous use of other measures. Costs can be borne at multiple levels, all of which should be considered. For example, the official cost of a claims-based measure may fall to CMS or state officials, but because government reports are often released with a lag of a year or more, hospitals or practices might need to implement the measure to track improvement in near real time.

Although claims-based measures might be presumed to be relatively inexpensive because data are collected routinely for other purposes, even they require several steps, the costs of which are borne largely by those conducting the analyses. These include start-up costs of learning to use a measure and ongoing costs of using it regularly, such as combining and homogenizing data from different sources, cleaning and preparing data for analysis, applying analytic programs to specific data sets, and packaging results for reporting. Even for measures that use routinely collected data (eg, readmissions, cervical cancer screening), linking performance to financial or other incentives may have unintended consequences related to practice changes driven by incentives rather than patient outcomes.4

Measures requiring dedicated data collection are likely even more costly. Patient and family experience surveys, although sources of valuable information, are expensive. These costs are usually borne by hospitals and practices, typically using third-party vendors, but may be passed on to payers, consumers, and taxpayers. Medical record abstraction is particularly expensive because of substantial labor costs, especially when performed by clinician reviewers. Automated abstraction from electronic health records (EHRs) has been heralded as a means of reducing costs, but automation currently involves immense fixed costs and nontrivial ongoing costs.5 Even 7 years after passage of the Health Information Technology for Economic and Clinical Health (HITECH) Act, which mandated automated quality measures as part of the "meaningful use" criteria, such measures are infrequently available and rarely used. Despite slow progress, costs may decrease as EHRs increasingly incorporate and use automated measures.

Assessing cost explicitly and transparently and comparing the costs of similar (and dissimilar) measures could have several benefits. First, measuring costs could help payers, hospitals, practices, clinics, and other health care organizations prioritize measures. Some useful measures may be worthwhile even if expensive; those with limited clinical value should be retired, especially if expensive. If measures have moderate clinical value, cost may become a critical factor in deciding whether to use them. Cost should not be the only driver for how quality measures are selected. Entities that endorse or select measures should also consider the effect of the measures on health outcomes and costs of care, which in some cases might outweigh any direct costs associated with using the measure. This is an empirical question, ideally addressed using techniques such as cost-effectiveness analysis.6

Second, cost estimation could lead to better understanding of the magnitude of the cost of quality measurement. Policy makers and payers could use this information to create more realistic and effective incentive structures, thus encouraging providers to measure quality and report their performance.

Third, making measurement costs explicit could spur innovation in developing more cost-effective data collection. If the cost of a measure is made publicly available and included in decisions about its use, developers might be inclined to make their measure less expensive. For example, reporting the cost of a chart-abstracted measure (eg, several Hospital Inpatient Quality Reporting Program measures) might encourage developers to explore structured data or natural language processing. Without estimating costs or including them in decisions about using measures, there is limited incentive to develop cost-effective measurement strategies.

Measuring quality of care is essential to improving it. However, the current, cost-uninformed approach has created a proliferation of measures, many of which are needlessly burdensome for health care organizations. Better understanding the cost of measures would not only inform decisions about which measures to use, but also guide future development of high-value measures that maximize benefit while optimizing use of finite quality measurement resources.