New findings may fix the replicability crisis in microbiome research

In new work published in Nature Metabolism, the scientists report that daily fluctuations in the gut alter the microbiome so significantly that different bugs populate it in the morning and in the evening. That means that a researcher analyzing a stool sample collected at breakfast will reach radically different conclusions from a researcher analyzing a stool sample collected right before dinner. The UC San Diego scientists propose that this variability is keeping gut microbiome researchers from being able to replicate each other's experiments.

"Unexplained variability and lack of replicability may be due to the fact that the microbiome oscillates throughout the day, with different populations of microbes dominating at different times," said Amir Zarrinpar, M.D., Ph.D., gastroenterologist and associate professor of medicine at UC San Diego School of Medicine and senior author on the study. "We found that when a sample is taken can dramatically affect which microbes were present and the conclusions the scientists drew about the disease they were studying."

Scientists conduct experiments for many reasons. The traditional reason is to answer a specific question, but another reason is to make a discovery or arrive at a scientific truth that others can replicate with their own experiments. In gut microbiome research, scientists collect stool samples to discover which microorganisms are present, and in what amount. Then, they link those changes to disease processes.

For this study, the team compared computer analyses of previously published studies, including their own. They discovered that changes in the microbiome were so pronounced over time that they affected the results as much as diet did. "We found that in as little as four hours after a mouse eats breakfast, nearly 80 percent of its microbiome is different," said Zarrinpar. Analyzing the conclusions drawn in the studies, Zarrinpar and his team found that results and conclusions depended heavily on when the researchers collected the samples.

Zarrinpar was inspired to conduct this study by a conversation he had with a colleague. "He told me that a postdoc in his lab took over an experiment that had been started by someone else. The postdoc couldn't replicate any of the previous trainee's findings. That made him question his predecessor's research," said Zarrinpar. "Then the postdoc realized that a bacteria that was incredibly pervasive in his findings was one that appears late in the day. He went back to his lab and saw that the previous trainee liked to collect samples in the morning, while he himself collected samples before going home. That's why he couldn't replicate the first trainee's findings."

Being able to reproduce the results of a previous experiment -- replicability -- is a key element in knowing whether a finding reliably represents new knowledge about reality or is simply an artifact of the experiment. Microbiome research is currently experiencing a replicability crisis, in part because of the interdisciplinary nature of the field, the complicated relationship between microorganisms and their hosts, and the difficulty of controlling so many variables.

Zarrinpar believes that his team's newest findings about the significance of timing can help fix the replicability crisis in microbiome research. He explains, "If we're ever going to be able to communicate to each other about our science and what we think is going on in an effective way, then we need to understand that if you got different results than I did, maybe that could be due to the time that we're collecting samples or not. Right now, you can't even tell."

According to Zarrinpar, scientists in other fields, such as circadian biologists, have also been lobbying the NIH to be stricter about the need to report timing of sample collections. Zarrinpar is hoping that publication of this paper will convince more scientists -- and the people who fund and publish their research -- of the significance of timing and its possible effect on other fields as well, such as metabolism research.

Zarrinpar's next steps involve advocating for standardized guidelines that ensure consistency in microbiome sample collection times and methodology. "This will likely involve collaboration with other researchers, funding agencies and journal editors to promote the adoption of such standards," he said. His next paper focuses on understanding the impact of timing in humans -- a variable that's much more difficult to control.

This research was funded in part by the National Institutes of Health, the Soros Foundation, and the American Hospital Association.

Co-authors on the study include: Celeste Allaband, Amulya Lingaraju, Stephany Flores Ramos, Tanya Kumar, Haniyeh Javaheri, Maria D. Tiu, Ana Carolina Dantas Machado, Roland A. Richter, Emmanuel Elijah, Gabriel G. Haddad, Pieter C. Dorrestein, Rob Knight of University of California San Diego, and Vanessa A. Leone, University of Wisconsin-Madison.