The review of scientific studies in journals is subjective and quality is variable

In the sphere of scientific publications, the process known as 'peer reviews' consists of evaluating manuscripts that are sent to scientific journals to be published, by two or more qualified independent reviewers, who are usually experts on the subject. Catarina Ferreira, researcher at Trent University (Canada), the Institute of Research in Game Resources and CIBIO-UP in Portugal, is leading a study reviewing the history of this review process. In the study, published in 'Biological Reviews', Ferreira highlights the resistance of this technique to changes in publishing practices, which has further exacerbated the vulnerabilities of this system.

"The main weaknesses are currently related to three aspects: the voluntary nature of the peer review, since 'peers' only participate in the process if they wish, the disparity of review criteria or guidelines produced by scientific journals, and a lack of tangible recognition of the reviewers for their service to the scientific community. All of the above makes the peer review process slow, highly subjective, and results in reviews of greatly varying quality," the researcher said.

This is a serious problem, not only for the scientific community, but also for the journals, which in the last few years have openly recognised that it is increasingly difficult to secure the participation of scientists as reviewers -in ecology, the non-acceptance rate for requests to review articles is 49%- and obtain high quality exams.

"Cases of scientific fraud and retractions of articles -adds Ferreira- are increasing, and they reflect the ineffective examination on which a weak peer review is based."

The steps for publishing in a scientific review

In the specific case of this study, all of the authors, who are trained as biologists, focussed on the field of ecology and evolution. From the hundreds of scientific journals that are published in these fields, they chose to contact the 38 considered to be the top journals, as classified by the standings produced by 'Google Scholar Metric'.

"We did it in this manner simply because we assumed that the journals that have a higher 'traffic' of articles received also have more pressure to improve the peer review system," argued the expert. Peer review was defined as "not consistent" in all of the 38 journals selected on ecology and evolution, and scientists ensure that it is something that is a cross-cutting issue in practically all scientific fields.

"The definition of an 'ideal' peer review is somewhat complex and, currently, even the journals that have the most organised systems struggle with weak reviews, which demonstrates that standardising review criteria and guidelines does not at all resolve all of the weaknesses in the system," the researcher added.

How can this system be improved?

Some measures have been suggested as potential mitigators of weaknesses such as the privatisation of peer reviews and making participation mandatory.

"Applying correction factors to the h-index -the highest number of articles that an author has published and been cited at least the same number of times-, paying fees to the reviewers or offering them royalties (such as discounts on subscription fees or acknowledgement notes) are some of the proposals. Moreover, some editors are making efforts to homogenise review criteria between them, such as the British Ecological Society, and some review guidelines exist, although there is no agreed criteria on which is the most relevant," highlights Ferreira.

In the 38 reviews analysed, the disparity of methods used by the journals to instruct their reviewers on peer reviews was evident, from the complete absence of guidelines and unclear criteria, to more formal systems with forms and defined criteria.

"None of the measures proposed to date has the potential to resolve the problems in the long term, because they are partial and not holistic. In our opinion, a contemporary peer review process, in which the current needs of the scientific community are addressed, should be centralised in a platform -independent of journals, whose interests are above all financial- with clear review criteria and guidelines, adjusted according to the scientific field," the researcher concludes.

They also propose that this centralisation be led by scientists, since this would facilitate the standardisation of the process, as well as increasing its transparency and reliability.