Publication Bias Found Among Trials Submitted To FDA: New Study

Among those trials published, unexplained discrepancies between the FDA submissions and their corresponding publications—the addition or deletion of outcomes, changes in the statistical significance of reported outcomes, and changes in overall trial conclusions—tended to lead to more favorable presentations of the drugs in the medical literature available to health care professionals.

Lisa Bero and colleagues from the University of California San Francisco reviewed the publication status of all 164 efficacy trials carried out in support of the 33 new drug applications (NDA) for new molecular entities approved by the FDA in 2001-2002, and compared information from the FDA reviews with published journal articles. Seventy-eight percent of the trials were published. Trials with favorable outcomes for the drugs were more likely to be published as those without favorable outcomes. Of a total of 179 primary outcomes included in the NDAs, 41 were omitted from the papers. The papers included 138 outcomes that were also in the NDAs (77%), plus 15 additional outcomes that favored the test drug, and two other neutral or unknown additional outcomes. Thus, the papers included more outcomes favoring the test drug than did the NDAs, report the authors.

The research also found additional discrepancies between the FDA reviews and the published papers. Of the 43 primary outcomes reported in the NDAs that showed no statistically significant benefit for the test drug, only half were included in the papers; for five of the reported primary outcomes, the statistical significance differed between the NDA and the paper and generally favored the test drug in the papers. Nine out of 99 conclusions differed between the NDAs and the papers; each time, the published conclusion favored the test drug. The authors did not investigate why the discrepancies existed, nor whether the changes were prompted by the drug sponsor, authors, or journals.

Because of their findings of publication bias and selective reporting, the authors conclude that "the information that is readily available in the scientific literature to health care professionals is incomplete and potentially biased."

In a commentary on the research, An-Wen Chan from the Mayo Clinic in Rochester (uninvolved in the study) says this new research makes an important contribution to the growing body of evidence that the trial literature is skewed towards reporting favorable results. "Biased reporting of results from NDA trials is particularly concerning because these journal articles are the only peer reviewed source of information on recently approved drugs for health care providers, who will have had limited clinical experience with these new treatments," Dr Chan says. "There are also substantial cost implications if the efficacy is overestimated and the drugs overused."

Before a new drug is approved for the treatment of a specific disease in the United States and becomes available for doctors to prescribe, the drug's sponsors must submit a "New Drug Application" (NDA) to the FDA, which provides details of the drug's development from laboratory and animal studies through to clinical trials. FDA reviewers use this evidence to decide whether to approve a drug.