ecms_neu_mini.png

Digital Library

of the European Council for Modelling and Simulation

 

Title:

When Competition Is Pushed Too Hard. An Agent-Based Model Of Strategic Behaviour Of Referees In Peer Review

Authors:

Juan Bautista Cabota, Francisco Grimaldo, Flaminio Squazzoni

Published in:

 

(2013).ECMS 2013 Proceedings edited by: W. Rekdalsbakken, R. T. Bye, H. Zhang  European Council for Modeling and Simulation. doi:10.7148/2013

 

ISBN: 978-0-9564944-6-7

 

27th European Conference on Modelling and Simulation,

Aalesund, Norway, May 27th – 30th, 2013

 

Citation format:

Juan Bautista Cabota, Francisco Grimaldo, Flaminio Squazzoni (2013). When Competition Is Pushed Too Hard. An Agent-Based Model Of Strategic Behaviour Of Referees In Peer Review, ECMS 2013 Proceedings edited by: W. Rekdalsbakken, R. T. Bye, H. Zhang, European Council for Modeling and Simulation. doi:10.7148/2013-0881

 

DOI:

http://dx.doi.org/10.7148/2013-0881

Abstract:

This paper examines the impact of strategic behaviour of referees on the quality and efficiency of peer review. We modelled peer review as a process based on knowledge asymmetry and subject to evaluation bias. We built two simulation scenarios to investigate largescale implications of referee behaviour and judgment bias. The first one was inspired by “the luck of the reviewer draw” idea. In this case, we assumed that referees randomly fell into Type I and Type II errors, i.e., recommending submissions of low quality to be published or recommending against the publishing of submissions which should have been published. In the second scenario, we assumed that certain referees tried intentionally to outperform potential competitors by underrating the value of their submissions. We found that when publication selection increased, the presence of a minority of cheaters may dramatically undermine the quality and efficiency of peer review even compared with a scenario purely dominated by “the luck of the reviewer draw”. We also found that peer review outcomes are significantly influenced by differences in the way scientists identify potential competitors in the system.

Full text: