Post-operative blood clot rate a flawed measure of hospital quality

The study examined whether surveillance bias (i.e., the greater the intensity of a search for a condition the greater likelihood it will be found) influences the reported rate of venous thromboembolism (VTE; blood clot). Venous thromboembolism, which includes deep vein thrombosis (DVT) and pulmonary embolism (PE), is a common postoperative complication that remains a leading potentially preventable cause of postoperative illness and death. The Agency for Healthcare Research and Quality developed a risk-adjusted postoperative VTE rate measure, Patient Safety Indicator 12 (PSI-12), that has been incorporated into numerous quality improvement programs and public reporting initiatives.

"However, measuring VTE rates may be flawed because of surveillance bias, in which variation in outcomes reflects variation in screening and detection, or 'the more you look, the more you find' phenomenon. This can occur in a number of ways: hospitals may use screening protocols, in which asymptomatic patients routinely undergo VTE imaging studies on a certain postoperative day, or clinicians have a lower threshold to order a VTE imaging study for patients with minimal or equivocal signs or symptoms (e.g., any leg swelling prompts a venous duplex [an imaging procedure]). Hospitals that are more vigilant and perform more imaging studies for VTE may identify more VTE events, thus resulting in paradoxically worse performance on the VTE outcome measure," according to background information in the article.

To examine the effect of surveillance bias on the validity of VTE as a quality measure, Karl Y. Bilimoria, M.D., M.S., of Northwestern University and Northwestern Memorial Hospital, Chicago, and colleagues conducted a study that merged 2010 Hospital Compare and American Hospital Association data from 2,838 hospitals. Next, 2009-2010 Medicare claims data for 954,926 surgical patient discharges from 2,786 hospitals that were undergoing 1 of 11 major operations were used to calculate VTE imaging and VTE event rates.

Instead of finding the expected relationship between adherence to VTE prevention and lower VTE rates, the researchers found that VTE prevention rates were positively correlated with VTE event rates. "A paradoxical relationship was also found between a measure of hospital structural characteristics reflecting quality [hospital characteristics that examine health care quality and which reflect a hospital's resources and focus on programs intended to provide higher-quality care] and VTE event rates: hospitals with higher structural quality scores had better VTE [prevention] adherence rates, but they had unexpectedly higher risk-adjusted VTE rates. Most important, hospital VTE rates were associated with the intensity of detecting VTE with imaging studies."

Hospitals in the lowest imaging rate quartile diagnosed 5.0 VTEs per 1,000 discharges, whereas the highest imaging rate quartile hospitals found 13.5 VTEs per 1,000 discharges.

"Our study calls into question the merit of the PSI-12 VTE outcome measure as a quality measure and its use in public reporting and performance-based payments. Hospitals reported to have the highest risk-adjusted VTE rates may in fact be providing vigilant care by ordering imaging studies to ensure that VTE events are not missed. Patients selecting hospitals according to publicly available metrics may be misled by currently reported VTE performance. The measure could be counterproductive if a hospital performs poorly on the VTE outcome metric, expends efforts to improve VTE prophylaxis resulting in increased awareness and vigilance in looking for VTE, and then finds more VTEs and becomes an even worse performer on the VTE measure," the authors write.