I was recently invited to review a manuscript for a journal I follow regularly. The content was right along the lines of my kind of research, and I was happy to accept. I was, of course, Reviewer Number Two. I always have been, in each of my fifteen-ish reviewing experiences. But this was the first time that the drop-down menu actually encouraged me to be your stereotypical Number Two:
I laughed about this fluke, but in all honesty, my laughter was nervous.
I was in the second year of my PhD when that first reviewing assignment landed in my lap. There I was, sifting through my inbox, deleting the “Dear Dr. Todorovic” flattery of predatory publishers. But then I hesitated at this one e-mail, because apart from the heading, it lacked the usual telltale signs of spam.
An invitation to review.
I forwarded it to my supervisor. “Oh, that’s a good journal – don’t you know it? I’d accept if I were you.”
Wow. Pride and chagrin, all rolled into one.
I kept re-reading that manuscript, re-wording my review, postponing the submission. Should I tell the editor I’m a clueless student, and not Dr. Todorovic? Should I say I’ve never done this before, that I don’t know what I’m doing or why they picked me? That it’s all a big mistake? In the end I said nothing. I pressed the submit button; the world didn’t implode. Two days later, an e-mail arrived. The other reviewer didn’t do their job on time, the editor thought my concerns alone were enough to request a major revision. I was mortified.
It got easier over time and with some experience, but it never really got easy. I still open the report of the other reviewer, the one that knows what they’re doing, with trepidation. If they caught something I should have, I feel ashamed. If their misgivings align with mine, I’m flooded with relief. If I mention something they didn’t, I worry that I was nitpicking.
As Reviewer #2, I get to see plenty of weak papers in low-impact journals, written in broken English, with poorly described experimental procedures and inconclusive results. It’s very annoying when these come from good labs that write up their other papers, the ones I don’t get to review, with care. I never know how much to judge and how much to help. I never know if helping will be seen as asking them to write the paper I would have written, the thing Reviewer #2 is notorious for. I never know what to do when I need just a few extra pieces of information to understand the design, before I can decide about the rest. I get frustrated when I don’t understand things, and I worry that this frustration will spill over into my review as pointless vitriol. Another feature of #2.
And then they’ll smile through clenched teeth, thank me for my input.
It’s worse when the journals are good. I can judge whether a design is creative and elegant, and can lead to the claimed conclusions. I can judge whether the analyses are sound. In some cases I will even check whether the numbers in the reported statistics all match up (you’re welcome). But when I have to judge novelty? And whether the wow effect matches the scope of the journal? Good grief, how should I know? Ask Reviewer #1, I can barely keep up with my own narrow topic.
Most of the learning from that first review onward was (and still is) a lonely process, with only the other reviewer’s comments as any substantial form of feedback. So every time I hear a gripe about Reviewer #2, I cringe a little on the inside. It’s me, it’s me, and I’m trying to be invisible.
I don’t think we should stop grumbling about Reviewer #2, I’m a big fan of complaining. But maybe, just maybe, a little bit of structured guidance would help? Someone to show us how to be kind but decisive. To tell us to always list strong points, then voice our misgivings as suggestions for improvement. To consider whether the experiment is something others would care to know about before we rip it apart.
I had a supervisor who showed me the ropes, but this shouldn’t be left to individual group leaders. We’re all in this together, both causing the damage and taking it. Instead of throwing young researchers into it head-first, maybe we can teach them, and make reviewing a more user-friendly experience.
“Hi, my name is Ana and I will be your reviewer tonight.”