This article by Rebecca Schumann is really, really bad. Peer review works really well, actually. Of course, anyone can collect horror stories, but even these must be taken with a grain of salt. Some stories amount to: my article was rejected. Curiously, the comments to her article just skewer it, pointing to its multiple inaccuracies. There are 271 comments and most are critical of her argument. She is losing a lot of credibility because academics know she is full of shit. One comment compared her perspective to that of the typical grad student gripe session. How true.
I've probably done an average of 6 reviews a year since 1990 or so. I've also been on the other side of the process for a quarter century. I know I'm good at it because the editors who use me keep asking me to do it again.
In my view, what is more important than peer reviewers is a good editor. This editor will
*Redact or simply not use reviews that are unhelpful.
*Stop using reviewers who are dilatory or too hostile in tone.
Problem solved!
Rebecca's suggestions make no sense because they are addressed to a misconceived notion of what the problem is. She states that someone should earn the right to submit to a journal by doing peer reviews. But a good editor will only ask established scholars to do reviews. Her notion that most reviews are done by grad students or recently minted PhDs is simply not true.
I haven't agreed with all my peer reviews. I have had a few articles rejected. Usually a grown-up will just learn from those experiences and move on. It is probable that a reviewer has saved me from the embarrassment of publishing something that was not yet ready to be published.
My recent experience suggests that articles being submitted are worse than ever. This is likely to increase the number of horror stories, since someone clueless enough to submit utter crap might be clueless enough not to know that a reviewer is saving him / her from embarrassment.
***
One question is whether it is legit to say that an article should cite me. I have mixed feelings. Sometimes I have felt that an article should cite me, but haven't said so. Instead, I'll give a list of other scholars that also should have been cited. Sometimes, I'll include myself in a list of names that the article should have cited. Sometimes, I feel that if have written on the topic, in a way relevant to the article, and my name is on the editorial board of the journal, the writer should have known that the article would be sent to me.
***
I have had people cite me and disagree. That makes it very hard for me to reject an article, because I would never want to reject someone because they disagree with me.
7 comments:
You're exactly right about the important role of editors. Also, that if scholars generally had grown-up reactions to rejection things would be a lot better. No one really seems to be willing to the work of finding their readers, represented by editors and peer reviewers, and learning from rejection how well their search is going.
"and learn from going where we have to go."
as a poet sd.
I don't read RS anymore. But just now I got curious.
"I’m sure you’ve heard the adage 'publish or perish.' In order to get tenure … a scholar must …"
That's how it opens. So, basically, it's written for people NOT in academia. So, basically, "here's how" people NOT in academia can "fix" the way academics peer-review each other.
I think quit-lit is over. At least for me.
I have only had a chance to review 2 articles for two different journals so far and I can say that the decision to ask me to be one of the reviewers was absolutely brilliant. With one of the articles, I had read absolutely every single one of the texts in the bibliography list. With the second one, I'd read most of the bibliography. I was also able to give what I believe are very useful suggestions to the authors as to what else they could read and where the holes in the argument were. And that was only possible because I know this stuff so well.
Of course, I used to believe the peer review was broken when I couldn't get published. But now I'm seeing that it works very well.
I think that if one is in Spanish or English where there are very many journals and reviewers one has v. little to complain about. In some smaller fields it may in fact be quite a trick to get published -- or get published in journals known to your dean -- if you have certain views (or do criticism and analysis rather than honorific reading and biographical report).
Also, or and-but:
I do not consider myself thick skinned and I have had pieces rejected or reviewed negatively. I would not have said these reviews were "kicking down" as RS claims most reviews do. Some people have seriously disagreed with my approaches to things and I once had a couple of people point out in unequivocal terms that a piece was poorly researched. It was: I had rushed it on advice of mentors, wanted to put more time into it. So these allegedly "harsh" reviews (enough so editor was apologetic about their tone) were actually affirming. This is all of course just anecdotal and just my view on my experience but maybe one could consider whether or not there is actually something to the review in question?
As reviewer I am not particularly harsh -- if I learn something from the piece, which I often do, I tend to say publish it. However, I have received some pieces to review that were so poorly put together I hardly knew where to begin.
These things having been said I would also note that peer review is often not blind, or cannot be if one is in field and is familiar with what is being written. Blind if the writer has not published a lot that you have read, yes. If they have, it is nearly inevitable that they will be identified.
I think the system is at least partially broken. I can think of a number of articles in recent years that should never have been published or should have undergone significant revision before publication. As people have said, it's all about the editor and the reviewer. A specific feature I've seen repeatedly is inadequate bibliography: not just the list at the end, but the fact that major studies with relevant material are just not taken into account. One can fault the author, but it's the reviewer who is supposed to catch this. I attribute this to the steady growth of the secondary literature in the field, and a certain degree of laziness/feeling overwhelmed. Also to editors not seeking the services of the right reviewers. But clearly reviewers, perhaps out of their own lack of familiarity with key contributions, are all too often nodding essays through that should have been turned back.
I have benefitted from a number of readers' reports, but at the risk of turning this into an anecdote-fest, there are also occasions where the report is cursory or just wrong. There are a couple of instances, one years ago and one recently, where I was flatly turned down by a journal and I then submitted the piece, unaltered, to another journal of equal or perhaps higher reputation, where it was then accepted without any or with minimal revision. What does that say about the system?
Post a Comment