Majority of first-wave COVID-19 clinical trials have significant design shortcomings, study finds

The researchers, whose findings appear June 9 in the journal BMJ Open, analyzed the 201 clinical trials for drugs or plasma that, as of March 26, had been registered in the U.S. under ClinicalTrials.gov and in the international clinical trials registry maintained by the World Health Organization (WHO). Of the trials analyzed in this early snapshot, 100 (49.8 percent) were registered in China and 78 (37.8 percent) in the U.S. The majority of trials in the U.S. clinical trials registry were registered from international researchers. Among the 201 trials analyzed, 126 were recruiting participants from China, 31 from Europe, 14 from the U.S.

The researchers found that many of the trials lacked key features needed to optimize their scientific value such as the use of control groups and patient and clinician blinding.

The analysis found that one-third of trials lacked clinical endpoints to clearly define success or failure. Nearly one-half were designed to enroll fewer than 100 patients, limiting their usefulness to assess modestly-sized treatment benefits. Two-thirds were "open label," meaning that patients and doctors were aware who did and didn't receive the treatment -- in principle, allowing their unconscious expectations to influence the results.

"Because of these weaknesses, many of these studies are likely to yield only preliminary evidence," says study first author Hemalkumar Mehta, PhD, an assistant professor in the Department of Epidemiology at the Bloomberg School. "Given the urgency of identifying definitive evidence on potential COVID-19 treatments, this is an instance where we wish we did not have to say 'further research is needed' because of basic trial design shortcomings and small trials."

The rapid initiation of so many clinical trials reflects the lack of any effective treatment for acute COVID-19 disease, whose global toll exceeds 400,857 deaths worldwide as of June 8, 2020, according to the World Health Organization's situation report. Of the 201 trials the researchers analyzed in their paper, the vast majority involved treatments already used for other diseases -- treatments that could potentially be repurposed for COVID-19 relatively quickly because they have existing safety profiles and have already been evaluated by the U.S. Food and Drug Administration or other regulators around the globe.

U.S. doctors generally have wide discretion to prescribe FDA-approved drugs for other, "off-label" conditions. The authors note that off-label use can carry risks, and it is essential that therapies, including those used off-label, are studied for safety and effectiveness for COVID-19.

In all, the 201 trials involved 92 distinct drugs as well as antibody-containing blood plasma.

Mehta and colleagues found that most of these trials demonstrated design weaknesses. For example, about a third had no defined clinical endpoint, such as hospital discharge or survival, by which success could be measured. About a quarter lacked the standard random assignment of patients to a candidate treatment or control/comparator drug. Of the 152 trials that did randomize patients to a treatment or comparator, only 55 involved the usual practice of 'blinding' -- a bias-reducing strategy in which patients as well as doctors and others who direct care and assess outcomes are kept from knowing who received the treatment or placebo.

"We understand the urgency of clinical research on COVID-19, but this is a time when we need rigorous science to inform policy and clinical decision-making," says study senior author G. Caleb Alexander, MD, professor in the Department of Epidemiology at the Bloomberg School. "Any treatment that is ultimately deemed safe and effective via robust trials could potentially be used by millions of people."

The researchers noted that the number of U.S.- or WHO-registered clinical trials of potential COVID-19 treatments tripled from the beginning of March to March 26 when they did their snapshot survey, and since March 26 has risen to more than 2,000 registered trials as of June 8, 2020, as new trials are registered on a daily basis.

"As the safety and effectiveness of new treatments are evaluated, it's vital that we use the best science to do so," Mehta says. "It's especially important at this juncture with many lives in the balance."

"Characteristics of registered clinical trials assessing treatments for COVID-19: a cross-sectional analysis" was written by Hemalkumar B. Mehta, Stephan Ehrhardt, Thomas J. Moore, Jodi Segal, and G. Caleb Alexander.

Disclosure: Alexander is past chair of the FDA's Peripheral and Central Nervous System Advisory Committee; has served as a paid advisor to IQVIA; is a co-founding principal and equity holder in Monument Analytics, a health care consultancy whose clients include the life sciences industry as well as plaintiffs in opioid litigation; and is a member of OptumRx's National P&T Committee.