Cardiovascular devices often approved by FDA without high-quality studies, study suggests

Cardiovascular devices are increasing in number and usage. "In 2008, at least 350,000 pacemakers, 140,000 implantable cardioverter-defibrillators, and 1,230,000 stents were implanted. Although there has been recent scrutiny of evidence used in the U.S. Food and Drug Administration (FDA) drug approval process, less attention has been paid to the approval process for medical devices," the authors write. They add that the study data on which FDA approval is based should be of high quality. "Ideally, this evidence should consist of randomized, double-blinded studies with adequate controls, sufficient duration, and thorough follow-up on prespecified primary end points without bias."

Sanket S. Dhruva, M.D., of the University of California, San Francisco, and colleagues analyzed the type and quality of study evidence used by the FDA for the premarket approval (PMA) of cardiovascular devices. These types of devices were included in the study because it was expected they would undergo the most stringent approval process, given their increasing usage and potential impact on illness and risk of death. The authors conducted a systematic review of 123 summaries of safety and effectiveness data (SSEDs) for 78 PMAs for high-risk cardiovascular devices that received PMA between January 2000 and December 2007, examining the methodological characteristics and primary end points. SSEDs are intended to present a reasoned, objective, and balanced critique of the scientific evidence which served as the basis of the decision to approve or deny the PMA.

The researchers found that of the 78 PMAs, 51 (65 percent) were supported by a single study. Of the 123 studies, only 98 SSEDs (80 percent) reported the number of participants enrolled. Of 123 studies in SSEDs, 27 percent were randomized and 14 percent were blinded. Fourteen percent of the studies reported did not have a primary end point stated. Of the 213 primary end points, 52 percent were compared with controls.

"In the SSEDs, there were 157 primary end points for which both the number enrolled and analyzed were stated. Of these, 122 (78 percent) had a discrepancy between the number enrolled and those analyzed," the authors write. One hundred thirteen discrepancies (93 percent) were that more patients were enrolled than analyzed. They add that the discrepancies between the number of enrolled patients and the number analyzed for primary end points may introduce bias because patients with less favorable outcomes may be lost to follow-up and safety concerns may underlie this missing data.

The researchers also found that of the 213 primary end points reported in the SSEDs, the results of 15 percent were noninterpretable. The most common reason was that no target goal for device performance was stated in 25 end points (78 percent), and in one instance the results were not stated. The authors state, "In some instances, end points were interpreted to meet their targets when they may have met only a part of them."

The authors write that there are several possible reasons why the criteria on which FDA device approval is based appear to be less rigorous than those for drug approvals. "First, device approvals are a more recent activity for the FDA, having begun in 1976 with the FDA Device Amendment, so the agency has less experience with devices than it does with drugs. Further, the last decade has brought a significant increase in the number and complexity of devices." They add that new surgical operations do not require FDA approval, and new devices, which are nearly always implanted, are between surgical operations and drugs on the FDA approval continuum.

The authors note that their study was based on the information presented in the SSEDs and some study information may be missing. However, they argue that SSEDs "should be a thorough and accurate compilation of the FDA's critique of evidence."

"The emphasis at the FDA in the last 17 years since the Prescription Drug User Fee Act has been rapid approval of new drugs. This study suggests that the emphasis for the FDA in 2009 and beyond must be approvals based on research that meets rigorous scientific standards for evidence of benefit and lack of harm to patients. To uphold the FDA's mission of ensuring 'safe and effective' medical devices, it is essential that high-quality studies and data are available."