Large-scale analysis finds majority of clinical trials don't provide meaningful evidence

The analysis, published May 1 in the Journal of the American Medical Association, found the majority of clinical trials is small, and there are significant differences among methodical approaches, including randomizing, blinding and the use of data monitoring committees.

"Our analysis raises questions about the best methods for generating evidence, as well as the capacity of the clinical trials enterprise to supply sufficient amounts of high quality evidence to ensure confidence in guideline recommendations," said Robert Califf, MD, first author of the paper, vice chancellor for clinical research at Duke University Medical Center, and director of the Duke Translational Medicine Institute.

The analysis was conducted by the Clinical Trials Transformation Initiative (CTTI), a public-private partnership founded by the Food and Drug Administration (FDA) and Duke. It extends the usability of the data in Clinicaltrials.gov for research by placing the data through September 27, 2010 into a database structured to facilitate aggregate analysis. This publically accessible database facilitates the assessment of the clinical trials enterprise in a more comprehensive manner than ever before and enables the identification of trends by study type.

The National Library of Medicine (NLM), a part of the National Institutes of Health, developed and manages Clinicaltrials.gov. This site maintains a registry of past, current, and planned clinical research studies.

"Since 2007, the Food and Drug Administration Amendment Act has required registration of clinical trials, and the expanded scope and rigor of trial registration policies internationally is producing more complete data from around the world," stated Deborah Zarin, MD, director, Clinicaltrials.gov, and assistant director for clinical research projects, NLM. "We have amassed over 120,000 registered clinical trials. This rich repository of data has a lot to say about the national and international research portfolio."

This CTTI project was a collaborative effort by informaticians, statisticians, and project managers from NLM, FDA and Duke. CTTI comprises more than 60 member organizations with the goal of identifying practices that will improve the quality and efficiency of clinical trials.

"Since the Clinicaltrials.gov registry contains studies sponsored by multiple entities, including government, industry, foundations and universities, CTTI leaders recognized that it might be a valuable source for benchmarking the state of the clinical trials enterprise," stated Judith Kramer, MD, executive director of CTTI.

The project goal was to produce an easily accessible database incorporating advances in informatics to permit a detailed characterization of the body of clinical research and facilitate analysis of groups of studies by therapeutic areas, by type of sponsor, by number of participants and by many other parameters.

"Analysis of the entire portfolio will enable the many entities in the clinical trials enterprise to examine their practices in comparison with others," says Califf. "For example, 96 percent of clinical trials have ≤1000 participants, and 62 percent have ≤ 100. While there are many excellent small clinical trials, these studies will not be able to inform patients, doctors, and consumers about the choices they must make to prevent and treat disease."

The analysis showed heterogeneity in median trial size, with cardiovascular trials tending to be twice as large as those in oncology and trials in mental health falling in the middle. It also showed major differences in the use of randomization, blinding, and data monitoring committees, critical issues often used to judge the quality of evidence for medical decisions in clinical practice guidelines and systematic overviews.

"These results reinforce the importance of exploration, analysis and inspection of our clinical trials enterprise," said Rachel Behrman Sherman, MD, associate director for the Office of Medical Policy at the FDA's Center for Drug Evaluation and Research. "Generation of this evidence will contribute to our understanding of the number of studies in different phases of research, the therapeutic areas, and ways we can improve data collection about clinical trials, eventually improving the quality of clinical trials."

An analysis-ready copy of the ClinicalTrials.gov database is now available at www.ctti-clinicaltrials.org. Specialists from numerous therapeutic areas are now scrutinizing the contents to better understand how the number and characteristics of clinical trials match the perceived needs of the research communities.

This dataset will be useful for academic institutions and also for pharmaceutical and device companies to produce reports showing the completeness of their data entry compared to other institutions. Advocacy groups can chronicle the number and types of trials in their area of interest.

Data quality is likely to improve as a function of the accountability fostered by this transparency.