Is peer review as great as it's cracked up to be?

Nobody has stepped forward to defend postmodern literary criticism or any field of that type. No here on the Straight Dope and not anywhere else that I’ve ever encountered outside the hallowed halls of academia. Those departments, at the very least, have become a laughing-stock across most of society. You just have to string together some pompous-sounding jargon and people anywhere know that you’re satirizing the intellectual elite. (See, for instance, the wizards of Unseen University in Partchett’s Discworld novels. The works of Lewis Carroll show that it’s not just a recent phenomenon.)

There’s no common scale for measuring nonsense. I find, however, that we generally underestimate how much nonsense flows from the intellectual elite. Consider that my grandmother, within her lifetime, has seen any number of trends sweep through, take hold for a short time, and then be dumped: social Darwinism, Freudianism, Marxism, the collective unconscious, postmodernism, existentialism, social constructivism, and who knows how many others. Yet each trend, once it goes down the tube, vanishes completely. The people who believed in it are not called to account for how they believed in something so silly. We should teach students about this history of bad ideas so that they’ll be better prepared to tackle future generations of bad ideas.

Then, surely, this is general agreement that people think a lot of nonsense comes from postmodern literary criticism, and not the “academic world” in general? I feel a bit like a broken record, but you appear to have taken criticism of a specific and applied it to the general, again.

And behind every intellectual who thought up those things, there were dozens, hundreds, even thousands of non-intellectuals perfectly willing to accept those things. I think you’re underestimating how willing people are to believe nonsense flowing from intellectual elite, or indeed any source. You overestimate common sense.

And out of interest what makes you think that students aren’t taught these things? I was certainly taught about theories of the collective unconscious, social Darwinism and Freudianism during my degree course.

Also I wouldn’t necessarily call social constructivism nonsense. While that is of course my personal opinion, i’m afraid it poses a problem for you; either it isn’t nonsense, in which case you can’t use it as an example, or it is nonsense, in which case my acceptance of some of it is yet more evidence that us non intellectual types can accept nonsense quite easily.

Your last point is one that Stephen J. Gould promoted throughout the life of his essays in Science. I certainly can agree with it.

On the other hand, I note that all those movements are scattered across philosophy, the arts, and pseudo science (leaving Freud to some less recognizable category). However, not one of those movements involved a hard science with peer-reviewed papers. I also note that few of them originated among any “elite.” They mostly arose as protests by persons outside the elite circles against earlier beliefs. I suspect that the thrust of your OP would have been better served by a genuine example of protracted silliness within hard science disciplines, but pointing to the fact that people can come up with odd ideas in various realms of thought does absolutely nothing to promote the idea that peer review is generally less effective than some other unspecified source of wisdom or analysis.

Perhaps that’s because there isn’t really any such “field” as postmodern literary criticism. There are literary critics (and historians and philosophers, etc. etc.) who are influenced by postmodern philosophy–a trend that peaked in the 1990s, but there are no fields as such. You won’t in other words, find a Department of Postmodern Literary Criticism.

Well I’m not sure if I count but I would defend postmodern philosophy as having its merits if not taken to excess. I’d also point out that you’ll find plenty of criticism of postmodernism (including intellectually enaged criticism published in peer reviewed journals in the humanities) in “the hallowed halls of academia.”

Perhaps. But I wonder if you’ve actually read any of the philosophers who have been influential to postmodernists. Have you ever read Jacques Derrida or Michel Foucault? Both are actually saying very interesting things about the world. And they are no more difficult to understand than Kant or Hegel even though these philosophers belong to the Enlightenment. It’s just possible, in other words, that you simply lack an interest in philosophy of any sort. And it’s entirely possible than any specialized area of inquiry–certainly fields such as physics and computer science for example–will sound like “pompous-sounding jargon” to someone outside that field.

So are you now dismissing the whole of nineteenth century scholarship as well as everything that’s been done since, say, 1968?

Honestly? I think it’s much more the case that we generally ignore most things that are said by the so-called intellectual elite in the sense that most Americans hardly know them.

In any case, your thread was about the value of peer review as a means of vetting research. To write off all such publications as belonging to a nonsensical intellectual elite is, as others have pointed out, painting with a pretty broad brush. Is the idea that simply because you’ve read some not terribly well-informed criticisms of humanities departments that you are ready to dismiss the whole of peer-reviewed research as nonsense? Would that include publications in, say, energy conservation, cancer research, genetics, or the impact of a policy such as No Child Left Behind?

Say what? I’m sorry ITR Champion but you are terribly misinformed if you think that any of these important thinkers and several of these bodies of thought has gone down the tube, vanishing completely. Darwin has gone down the tube? According to whom? And the idea that human beings have an unconscious (Freud’s main innovation) has disappeared??? So too with the other modes of philosophy or theory you mentioned. Marxism may not be taken very seriously as a roadmap for future economies but Marx’s writings offer brilliant analyses of capitalism that, IMO, any intellegent person ought to read.

Philosophy and other disciplines in which philosophy offers a critical framework don’t simply fade away–like a flavor of ice cream that ceases to be available. (Ironically, it’s you who is taking the position of the vulgar postmodernist with this oddly relativist point of view.) Successive developments in philosophy provide a kind of cumulative influence so that, say, today’s neo-Kantian political philosophers think differently than they would if they had not read postmodern philosophy–even though they are not postmodernists per se.

I’m hard pressed to see why any one should be called to account for having taking an interest in Darwin, Marx, Freud, Jean Paul Sartre, Derrida, Foucault, Judith Butler, etc. etc. etc.

Well, I’m sorry if you don’t strike me as especially open to the idea of intellectual history. To the contrary, you seem to think that there’s a single set of good ideas out there and that everything else should be treated dismissively and with contempt.

Can you really be so certain that you already know everything there is no know about everything in the world?

Nitpick. ITR Champion claimed that social Darwinism has gone “down the tube.” Social Darwinism has nothing to do with the scientific theory of Evolution. Social Darwinism was a philosophical/political/societal engineering movement that misapplied misunderstood lessons from Darwin’s Theory of Natural Selection through descent with modification to make bold (and crass) claims rationalizing society’s failure to assist the disadvantaged.

It was, in fact, the point that ITR Champion did not include a single hard science in the passage each of us has quoted that prompted my reply, above.

No, not a nitpick tomndebb–just my mistake. By the time I wrote my response I didn’t remember the “social” before the Darwinism and so failed to see it there as I replied.

Let me add though that the reason was almost certainly because I was flummoxed by seeing Freudianism, Marxism, existentialism and, yes, even postmodernism dismissed as forgettable and supposedly embarrassing trends. Now that I realize my mistake I’ll add that the idea of putting these influential bodies of knowledge alongside the the discredited pseudo-scientific “social” application of Darwinism illustrates what I was trying to say: Social Darwinism is, indeed, an entirely discredited pseudo-science, of interest mainly to historians of racism and imperialism, but the others are not. They remain studied and important for all sorts of good reasons in a number of humanities disciplines (as in some other disciplines too).

Sorry for the confusion, ITR Champion.

I saw your post but thanks for directing my attention to it nevertheless.

My point, as will perhaps be clearer now, is that ITR Champion is not only conflating the hard sciences and the humanities, but also being too dismissive of the developments within the latter despite what appears to be fairly limited knowledge of the humanities from either an academic or lay standpoint.

This bears repeating. As I pointed out on the previous page, **ITR ** clearly has no idea how peer review functions in the humanities. Then, as you say, with his misunderstanding of peer review in the humanities, he then tries to paint all of scholarship (including the hard sciences) with the same broad brush. Even if the humanities were all crap (which they are not), I don’t see how that would have any bearing on hard sciences, which is what **ITR ** really has it out for since they do nasty things like argue for evolution.

Short contributions with less lead time go back decades, at least. Unless the reviewers for this journal are a lot faster than the ones I’m familiar with, there is still the review lag - and a re-review lag if that is part of the process. (Conference papers get a yes/no, which speeds things up.)

I see this example, and your other one, a failure of editing, not peer review. If a reviewer turns in an invalid review (and I’d consider “I’d do it this way” invalid) he or she should send the paper out to some more reviewers or back to the original reviewer with some instructions. Reviewers are notably imperfect. I’ve even instructed an author to ignore a particular reviewer who clearly didn’t understand the point of the paper, though I did recommend making some things clearer.

As an editor, I’ve never extensively worked with an author before review, and no editor I’ve ever worked with tried to structure a paper before review. As a reviewer I have suggested ways of restructuring a paper to make the point clearer.

In my experience professors in state schools have less contact with undergrads than those in private schools. The only professor I’m aware of with little undergrad contact taught in the Business School, and even he was happy to have an undergrad take his class. When I was an undergrad all my classes were taught by professors, and in fact the big lecture classes were usually taught by lower level profs than the smaller classes with more undergrad contact.

I’ve reviewed for a conference with double blind reviewing, though most journals and conferences have single blind reviewing. The problem I have with double blind reviewing is, first, it is often easy to tell who the author is, second, the author cannot reference their previous work and show how the current work is an advance on it, and third, sometimes you can accept a paper (especially for a conference without a re-review) if you trust an author with a known track record to make changes.

For grants, the track record of the proposer is so significant, and usually a formal part of the evaluation process, that I can’t see how double blind reviewing would ever work.