|
Digital Library of the
European Council for Modelling and Simulation |
Title: |
When Competition Is Pushed Too
Hard. An Agent-Based Model Of Strategic Behaviour Of Referees In Peer Review |
Authors: |
Juan
Bautista Cabota, Francisco Grimaldo, Flaminio Squazzoni |
Published in: |
(2013).ECMS 2013 Proceedings edited
by: W. Rekdalsbakken, R. T. Bye, H.
Zhang European Council for Modeling and Simulation. doi:10.7148/2013 ISBN:
978-0-9564944-6-7 27th
European Conference on Modelling and Simulation, Aalesund,
Norway, May 27th – 30th,
2013 |
Citation
format: |
Juan Bautista Cabota,
Francisco Grimaldo, Flaminio Squazzoni (2013). When Competition Is Pushed Too
Hard. An Agent-Based Model Of Strategic Behaviour Of Referees In Peer Review, ECMS 2013 Proceedings edited by: W. Rekdalsbakken, R. T. Bye, H. Zhang, European Council for Modeling
and Simulation. doi:10.7148/2013-0881 |
DOI: |
http://dx.doi.org/10.7148/2013-0881 |
Abstract: |
This paper
examines the impact of strategic behaviour of referees on the quality and
efficiency of peer review. We modelled peer review as a process based on knowledge
asymmetry and subject to evaluation bias. We built two simulation scenarios
to investigate largescale implications of referee behaviour and judgment bias.
The first one was inspired by “the luck of the reviewer draw” idea. In this
case, we assumed that referees randomly fell into Type I and Type II errors, i.e.,
recommending submissions of low quality to be published or recommending
against the publishing of submissions which should have
been published. In the second scenario, we assumed that certain referees
tried intentionally to outperform potential competitors by underrating the
value of their submissions. We found that when publication selection
increased, the presence of a minority of cheaters may dramatically undermine the
quality and efficiency of peer review even compared with a scenario purely
dominated by “the luck of the reviewer draw”. We also found that peer review outcomes
are significantly influenced by differences in the way scientists identify
potential competitors in the system. |
Full
text: |