Quality On Peer Review Must Be Raised With Co-operation, Says Report

The report “Peer review: its present and future states”, which draws on ideas from an international conference in Prague in October 2006,  reflects some concern on the shortcomings of peer review while outlines some possible measures to cope with them.

The conference has “brought together key representatives from scientific publishing, national and international funding bodies and research institutes,” commented Professor Josef Syka, President of Czech Science Foundation. The event was organised by the Czech Science Foundation, the European Heads of Research Councils (EuroHORCS) and the ESF.  The conference was “a unique opportunity to discuss common concerns and exchange information on innovative approaches and methodologies,” he added.

No reward without risk

A central theme of the report is that the current peer review system might not adequately assess the most pioneering research proposals, as they may be viewed as too risky. John O’Reilly, former Chief Executive of the U.K.’s engineering and Physical Sciences Research Council (EPSRC), now Vice Chancellor of Cranfield University, said traditional peer review might be too risk averse. He suggested the need to encourage pioneering research that is high risk in the proposal, but high impact if successful.

The conference called for new approaches, enabling the assessment of innovative research, to be embedded in the peer review system. An example of a new approach to overcome the perceived risk-averse funding culture was given by Dr. He Minghong from the National Natural Science Foundation China.  His Foundation encourages reviewers and programme managers to spot risky project proposals which are then funded under stricter conditions. Their duration is shorter, their budget smaller and they are more closely monitored.

European harmony and cooperation

The challenge to increase the efficiency of the peer review process is shared by all research organisations. Participants agreed that the increasing importance of competitive research funding has also added on the pressure on referees and on research funding agencies.

Representatives from organisations throughout Europe, Asia and the U.S. explained how they are working to make their own review cycle more effective. One of the examples is the implementation of remote evaluation, encompassing virtual review panels in teleconferences and online tools to remotely access applications.

Dr. Toni Scarpa, from the U.S. National Institute of Health (NIH) described the use of knowledge management software tools to identify reviewers and assign applications to them.

The European Commission representative, Dr Graham Stroud spoke of the European Commission’s (EC’s) plan to use remote evaluation within the 7th Framework Programme, to minimise the length of time busy scientists have to spend in Brussels. This will also help efforts to further develop the pool of high-quality referees that the EC is currently managing.

A good reviewer is hard to find

For funding agencies finding and retaining good reviewers have becoming increasingly difficult.. The report stresses the need for better European cooperation in peer review process

Professor Raimo Väyrynen, former President of the Academy of Finland referred to the successful co-operation between Nordic countries with disparate research systems as an example of international harmonisation without agreeing on a common system. “It’s all about identifying common concerns and being flexible in what you want to achieve together,” he said.

Participants called for co-operation beyond jointly developed guidelines and the exchange of best practices. With increasing cross-border research cooperation and more trans-national research programmes, using fully harmonised peer review systems, where appropriate, was also discussed.

“They are many problems relating to the Peer-review and ESF has created a platform for its Member Organisations to jointly develop new ideas and news ways on Peer Review,” commented Dr. John Marks, the Chief Executive of the ESF. The forum will also discuss the sharing of resources and adequate models for international peer review.

All contributors to the conference report agreed that peer review is an essential part of research and that no other credible mechanism exists to replace it. They agreed further that a continuous monitoring and exchange of information, exemplified by this conference, will be an effective way to work together to develop and improve it.