Evaluating medical research: New treatments better than standard ones just over half the time

Now, in a paper published Aug. 22 in the top journal Nature, Dr. Djulbegovic and colleagues report that on average new treatments work better than existing ones just over half the time. On scientific and ethical grounds, they say, the randomized controlled trial (RCT) system's little more than 50-50 success rate over the past half century is evidence that the system is working as intended.

The researchers analyzed 860 phase III published and unpublished RCTs performed by academic institutions or pharmaceutical companies. These trials collectively involved more than 350,000 patients.

"Our retrospective review of more than 50 years of randomized trials shows that they remain the 'indispensable ordeals' through which biomedical researchers' responsibility to patients and the public is manifested," the researchers conclude. "These trials may need tweak and polish, but they're not broken."

People who consent to participate RCTs are willing to be randomly allocated to new or existing treatments. While RCTs are considered the gold standard for comparing the effects of one treatment to another, the gradual progress they yield can seem frustratingly slow -- particularly for patients with poor standard treatment options.

Yet, the genuine uncertainty associated with individual RCTs has been vital to the gains in therapeutics, said Dr. Djulbegovic, professor of medicine and oncology at the USF Health Morsani College of Medicine and Moffitt Cancer Center. If there was significant likelihood that one treatment in a comparison was better than the other, it would be unethical to deny some patients the superior treatment, and well-informed patients would probably refuse to participate in the study, he said.

Incremental advances in treatment generated by RCTs over time -- such as childhood leukemia cure rates moving from zero to 80 percent even though only 2 to 5 percent of new treatments provided a breakthrough -- have translated into important improvements in health and lifespan, the authors say. However, they suggest trials could still benefit from more rigorous design, implementation and reporting -with widespread publication of trial results, including negative findings.

The Nature paper is one of a series of publications that Dr. Djulbegovic and colleagues have authored this year (25 so far), including a clinical therapeutics case study in the New England Journal of Medicine.