Featured Post

BFRC

I am posting this as a benchmark, not because I think I'm playing very well yet.  The idea would be post a video every month for a ye...

Wednesday, January 26, 2011

Is Peer Review Oppressive?

I hear people complain about peer review. Here is my perspective.

First, the complaints:

*It stifles creativity and innovation, because really innovative work might get rejected if it breaks with the consensus of the field. It promotes "group-think."

*Some peer reviewers are slow, nasty, or not very good readers.

*It is too competitive. Not everyone can get into the top journals.

Now, my experience. In the cases of journals that use peer reviews and give the reports to the authors, I have had only one reviewer in my entire career who was unfair in an arbitrary way. I disagreed with some opinions of reviewers, but that's just life. Not everyone is going to agree with me all the time; how boring would that be! I never felt I was penalized for being too innovative or original. I do consider myself to be original, and have had no problems publishing tons of articles.

Journals that reject articles without giving any substantive comments might be using peer review, but they are using it wrong. Hispanic Review used to just send your article back to you in 10 days, with no comment. I've been accepted and rejected by them throughout the years.

I've had articles accepted as is with no changes requested. This is also not a good use of peer review, because no matter how good an article might be, there is always something that can be improved. In some cases, these were invited submissions. If your work is half-way decent, you will get invitations, and most of those articles are accepted and count almost the same as refereed articles, if the journal is one that is normally refereed. That's another way of getting around the oppression, but at the cost of not having peer reviewers save you from yourself.

In my experience on the other side of the ledger, I am very fast, I try to give my best impartial judgment and avoid all ethical conflicts of interest. The hardest articles to review are those closest to one's own work, where you have a particular stake in one side of a debate. I have been told by particular authors and editors that my comments are useful in revision. I recommend a fair number of "revise and resubmits" and have seen quite a few articles successfully into print.

If an article really makes me see a canonical author in a new light, I am tremendously grateful.

Rejected articles are all unhappy in the same way, to cite Tolstoy in reverse. Excellent articles make you see something distinctive in the material they treat. They tell you why Machado is different from Jiménez. Bad articles are generically bad; they go through the motions of presenting information, but they don't have a strong thesis and a convincing argument. Prose is often an issue.

As a reader of excellent scholarly journals, peer review also works for me, in that well-edited journals are actually better than internet sites masquerading as journals that publish just about everything they get. I may not love or be interested in every article, but I can usually see why they were accepted.

So peer review works for me on all three sides of the process. (The fourth side I don't know about first-hand, because I am not the editor of a journal, but my spouse is, and she makes it work for her by choosing reviewers who do a good job. Most authors do not complain.) I could have had an article wrongly rejected, and I could have wrongly rejected an article, but I don't think that has happened in more than 5% of cases. I don't think I've every mistakenly accepted an article. I've never said to myself, "I shouldn't have accepted that one."

I don't know who these original scholars are who have been oppressed by peer review. Aren't the top scholars in my field influential because of their originality? I guess there could be scholarly geniuses languishing in obscurity who are even more original and just haven't been able to get their ideas out there.

No. I don't think so.

***

In my experience, scholars who work hard enough, even without being particularly brilliant or original, are able to get published. It might not be PMLA, but it will be a legit publication.

***

The notion that peer review stifles originality is curious, in that we don't really want complete originality, but work that pushes an existing debate in a new direction. The system rewards meaningful advances based on previous knowledge. I think, in fact, that the system is right to do so. But what do I know? I'm just a part of this system myself.

5 comments:

Thomas said...

When my peers disparage peer review I feel a bit like when academics disparage "merely academic" debates.

Jonathan said...

Right. It is like when Swann speaks in a condescending way about his own life-style, and the narrator Marcel asks himself: "Pour quelle autre vie reservait-il d'être serieux...." Or something like that. Pardon my French.

Jonathan said...

"Pour quelle autre vie reservait-il de dire enfin sérieusement ce qu'il pensait des choses, de formuler des jugements qu'il pût me pas mettre entre guillemets.."

Clarissa said...

I'm a little late to the discussion but I still wanted to say that peer review has transformed in the past 10 years (which is when I first started publishing) and not in a good way. Detailed responses from blind reviewers are mostly a thing of the past. For the most part, you get to wait for 6+ months to get a couple of lines that are confusing. Recently I was told the following, "One of our reviewers really liked the article but another one didn't. Since we require a unanimous approval from both our reviewers, we cannot accept the article." It would be extremely helpful to get actual feedback from both of those reviewers, but no such luck.

Another journal accepted the article and published it but forgot to inform me of that. yet again, there was no feedback from anybody.

If peer review were, indeed, what it is supposed to be, I would support it wholeheartedly. What it is today, however, is deeply flawed.

Jonathan said...

Well, I'm sorry to hear that things aren't working as they are supposed to. That's an argument in favor of peer review as it's supposed to be.