eLife is trying out a new model of reviewing, that includes obligatory preprints, obligatory open data and code, as well as open reviews. You can read more about it here:
I was lucky enough to review an article under this new model. The openness of reviews is not yet obligatory, but I opted for it. Here are my thoughts on the process.
Firstly, I always sign my reviews, so having the authors find out my identity was a familiar thing. Why sign? Mostly because I do a better job when my name is on the review.
Sometimes people list fear of retribution as a reason not to. Another reason is that I might not be taken seriously as a female postdoc. Perhaps I’ve been lucky, but this hasn’t been my experience. So, I sign.
In the eLife review process, however, it is not just the authors and editor who will know my identity, but the review will become part of the public record – and that’s a whole other ballgame.
It was with a heavy heart that I wrote a negative review, knowing that it will be available to others. Given how high profile a journal eLife is, this is going to be a common scenario.
I’m actually not completely certain where my review will appear, or how people might come to find out about it. I do know that if the manuscript gets rejected at eLife, my review will only be made public once the paper gets accepted elsewhere. This ensures that the authors will address my main points even if they submit to another journal.
What’s not clear to me is whether they will have a way of responding if I made mistakes in my review (and I do make mistakes). In a later submission elsewhere they might ignore my advice for all the right reasons but they should get to say so somewhere other than in the final version of their paper.
There is also a part of the review that is intended only for the author: minor points and concrete suggestions for how to wrangle the data or perform further experiments. I like that there is some space for this nitpickery behind the scenes. That’s the part where the reviewer can go, like, if this was my experiment, I would have done it this way and written this whole other paper, and the authors can say sod off, this is not your experiment.
Another thing that I liked about eLife’s review process is that the decision about the manuscript happens in a conversation between the editor and reviewers. Have I mentioned that I make mistakes? This helps.
You might wonder who will ever want to read reviews of a version of an article that will in all likelihood be different from the accepted version. Honestly, I don’t think that’s the point. The point is that transparency makes for a better process. In some rare cases the entire sequence of editorial events will need to be scrutinised. All other cases are going to be equivalent to putting on a seatbelt every time you didn’t end up in a car crash. It’s worth it.
However, I see another advantage to open reviewing. Seeing how someone performs a review offers a great deal of insight into how they work. Are they a deep thinker? Are they kind? Constructive? Do they communicate with clarity? Is their aim better science, or are they just trying to find something to knock the article down in this particular journal? Do they recognise that reviewing is a position of power, and do they treat the process with care? Are they someone I would want to work with? I think these open reviews could be of use both for hiring decisions and for the decision whether to apply to someone’s lab.
To wrap up, I’m certain this new system is better and I’m glad that eLife is spearheading it. I hope that the inevitable wrinkles will be ironed out in time with plenty of good will.