Badges acknowledging open practices significantly increase reported data

The badges, visual icons placed on publications, certify that an open practice was followed and signal to readers that an author has shared the corresponding research evidence. Journals can opt into the initiative, awarding badges to articles that meet the specified sharing criteria. In January 2014, the journal Psychological Science announced they had adopted badges to acknowledge open practices. Approximately a year and a half after implementation, a research team developed a protocol to assess the impact of the open data and open materials badges.

"When Psychological Science started offering badges to encourage open practices, there was no evidence to support whether scientists would be influenced by them," says Mallory Kidwell, the study's lead author and project coordinator at the Center for Open Science. "Many researchers thought that the badges would not be significant enough to change standard research practices."

The study, to be published on 12th May in the open access journal PLOS Biology, suggests that awarding badges to publications not only increases reported data and materials sharing but also increases how accessible, correct, usable, and complete the data and materials are.

Researchers at institutions around the world coded all 2,478 empirical articles published from January 2012 to May 2015 in Psychological Science and four other highly impactful journals in the discipline. Authors observed a large increase in the rate of reported data sharing in Psychological Science after 2014, with little change in all four comparison journals' very low sharing rates. Psychological Science also showed an increase in reported materials sharing, although to a weaker degree and with much more variability between the comparison journals.

"We noticed that sharing materials can be a bit more complicated than sharing data, and can be more difficult for some sub-disciplines of psychology than others" says co-author Erica Baranski. "Social and cognitive psychology oriented journals, whose materials often consist of surveys and stimuli, had much higher rates of reported materials sharing than other comparison journals."

However, reporting the availability of data and materials does not guarantee that they are actually available. "If an article stated that data or materials were openly available, we investigated whether their contents could be easily accessed and interpreted by an outside researcher," explained co-investigator Ljiljana Lazarevi. Again, the authors found that badges were associated with a dramatic increase in the accessibility, correctness, usability, and completeness of reportedly available data and materials.

While the badges may seem like a simple intervention, there is now empirical evidence supporting their effectiveness. "Relative to their ability to promote transparent practices, the cost and risk for journals to implement badges is minimal," says Kidwell. "The adoption of badges across a broader range of journals could have a large effect on the field, and may lead to a cultural shift among scientists toward increased transparency and sharing of research data and materials."