Science on salt is polarized, study finds

The new article in the International Journal of Epidemiology is co-authored by Ludovic Trinquart, Columbia University Epidemiology Merit Fellow at the Mailman School of Public Health; David Johns, a doctoral student in Sociomedical Sciences at the Mailman School of Public Health and an affiliate of the Data & Society Research Institute; and Sandro Galea, dean of the Boston University School of Public Health and adjunct professor of Epidemiology at the Mailman School.

The researchers systematically reviewed 269 academic reports published between 1979 and 2014, including primary studies, meta-analyses, clinical guidelines, consensus statements, comments, letters, and narrative reviews. Each was classified according to whether it supported or refuted the link between reduced sodium intake and lower rates of heart disease, stroke, and death. More than half of the reports were published since 2011--suggesting an increasing level of interest in the question, even if there was no emerging consensus.

A citation analysis found papers on either side of the hypothesis were more likely to cite reports that drew a similar conclusion than to cite reports drawing a different conclusion. Dominating the literature were a small number of influential papers that presented strong evidence for and against.

"There are two almost distinct bodies of scholarship--one supporting and one opposing the claim that salt reduction in populations will improve clinical outcomes," says Johns. "Each is driven by a few prolific authors who tend to cite other researchers who share their point of view, with little apparent collaboration between the two 'sides.'"

"We pay quite a bit of attention to financial bias in our work," says Galea. "We seldom pay attention, however, to how long-held beliefs bias the questions we ask and the results we publish, even as new data become available."

An analysis of systemic reviews revealed very little consistency in the selection of primary studies. If a primary study was selected by a review, the chance that another review would select the same study was less than a third. The finding points to uncertainty and disagreement about what should count as evidence, the authors argue. Moreover, choices about which studies to cite as primary evidence directly influenced the conclusions of systematic reviews.

Even while the scientific debate over salt continues, public health officials, from the local to the global level, have enacted policies to lower consumption. World Health Organization guidelines recommend limiting salt intake. In December 2015, New York City became the first U.S. city to require chain restaurants to label foods high in sodium.

"Decision-makers often must choose a course of action in the face of conflicting, uncertain evidence," says Trinquart. "Both the misuse of uncertainty and the exaggeration of certainty can shape the outcomes of public health decision-making processes."

The authors say the citation analysis method used in the study represents a new way of understanding the relationships between academic research papers and authors. In the future, the same method could be applied to other topics, including controversial issues such as e-cigarettes as well as topics on which there is greater agreement.