Negative citations important to scientific progress should be tracked, says new study

But what happens if a researcher makes a citation in order to point out flaws or weaknesses in a previous study?

Negative citations are not necessarily a bad thing, says Nicola Lacetera, an associate professor of strategic management at the University of Toronto Mississauga who is also cross appointed to UofT's Rotman School of Management. Tracking those citations can reveal where there is particular "vitality" in a research area, especially when there is controversy among scientists active in it.

"On the one hand, it might be that a particular study is just of poor quality and reliability," says Prof. Lacetera, whose findings have been published in the Proceedings of the National Academy of Sciences. "On the other hand, maybe it is good science that can be improved and so criticisms are more constructive. That's the way science evolves and becomes better."

The effect and frequency of citations have been studied before but rarely has research looked at them in detail or attempted to classify them.

In this study, advancements in the computer science field of natural language processing allowed the researchers to target negative citations in more than 15,000 articles published over a decade in the Journal of Immunology, including the incidence of those citations and details about their patterns.

The researchers, including Prof. Christian Catalini of MIT's Sloan School of Management and Prof. Alexander Oettl of Georgia Tech, who are both graduates of the Rotman PhD program, found that negatively-cited studies are in the minority, are usually high quality -- cited both positively and negatively -- and receive more attention than the average, showing only a slight drop-off later in the life of the paper.

Their negative citations tend to come in the first few years after the original study has been published. They are often from other scientists working in a similar research area who may even have worked with the cited author on other studies. However they don't tend to come from colleagues who may be in the same building or down the hall, suggesting that perhaps those people share their criticisms informally with the original author rather than risking the social awkwardness of putting their critique in print.

Besides revealing patterns behind knowledge production, the method the researchers developed for their study holds promise for further exploration into the ways scientists build on each other's work and the improvement of scientific paper search algorithms. Meanwhile, the study should be consolation to those researchers targeted for criticism -- it's the stuff great science is sometimes built on, says Prof. Lacetera.

"No paper is perfect, including ours," he says.