Diet trials often amend their outcome measures as they go

Moreover, a study published today in JAMA Network Open concludes that 86 percent of diet trials amended their desired study outcomes along the way. That raises the possibility of bias, say researchers David Ludwig, MD, PhD and Cara Ebbeling, PhD, of Boston Children's Hospital, and Steven Heymsfield, MD, of the Pennington Biomedical Research Center (Baton Rouge, Louisiana).

In a concurrent op-ed in today's New York Times, Ludwig and Heymsfield call for more investment in rigorous diet research. They note that nutrition research is not only under-funded, but also more challenging to conduct than drug studies.

"High-quality trials are hard to do because diets, and the behavior of humans who consume them, are so complicated," they write. But because few big companies stand to profit, "typical diet trials must get by on shoestring budgets, rarely exceeding a few hundred thousand dollars, compared with drug trials that may cost several hundred million dollars."

Diet trials: What was planned, versus what was reported

For the new study, the team conducted a literature search of drug trials for obesity published in top journals (The New England Journal of Medicine, JAMA, The BMJ, The Lancet, Annals of Internal Medicine, and The American Journal of Clinical Nutrition) from 2009-2019. They identified 343 diet studies, and, for comparison, 148 drug trials for obesity.

All studies were registered in advance on ClinicalTrials.gov, a measure intended to keep investigators accountable to what they originally planned to test, ensuring that they don't report findings selectively. But when the researchers compared the original registry description with the final published papers, 18 diet trials (86 percent) versus only two of the drug trials (22 percent) showed substantive discrepancies.

Typically, these involved a change in the time frame of the primary outcome or the number of co-primary outcomes. One diet trial, for example, initially listed "weight at five years" as the primary outcome, but later amended this to "change in body fat at one year." Other trials initially planned several primary outcomes or measurements at several time points, but later pared these down to a single outcome or single time point in the published study.

A 'Manhattan project' for diet trials?

The investigators propose several immediate remedies, such as creating specialized registries for diet trials to reflect their special challenges. In the meantime, in the Times, Ludwig and Heymsfield call for "a sort of Manhattan Project" to pin down the effects of diet changes once and for all.

While this would require a substantial investment, they write, "the amounts involved would total a fraction of a cent for every dollar spent treating diet-related conditions like obesity, Type 2 diabetes, and cardiovascular disease."