PDA

View Full Version : How Does Peer Review Work for Books in the Academic Discipline of History?


Frylock
08-03-2014, 10:32 AM
I know how peer review works for journal articles in Philosophy--author sends in a paper, editor sends it out to a relevant professional philosopher, that reviewer says whether to publish or not (or revise etc) and that's the decision.

I don't really know how peer review works for books, though. If someone has written a book in History (i.e. the academic discipline) and the book is said to be "peer reviewed," does that mean one reviewer took a look at it, or several, or what?

Frylock
08-03-2014, 10:33 AM
Specifically, I'm wondering what it amounts to that this book (http://www.sheffieldphoenix.com/showbook.asp?bkid=264) is peer reviewed.

Ruken
08-03-2014, 11:13 AM
I don't know that particular book, but it depends. Books like this may have a varying degree of fact checking depending on the preferences of the author and editor. An anonymous peer review such as what one expects in an academic journal would be unusual IME.
Edited, book-sized collections of separately-written chapters are in between. I have a chapter in one such book. The editor said he would send chapters out for peer review if he felt it warranted. AFAIK he did not with mine. It was a review article, so no new science was presented.

Ruken
08-03-2014, 11:32 AM
Please pardon the reading comprehension fail. You specifically asked about history, of which I have little knowledge.

Exapno Mapcase
08-03-2014, 11:52 AM
I was under the impression that books are never peer-reviewed. Only journal articles are.

Peer review is not mentioned in that link. Why do you think it was?

Frylock
08-03-2014, 11:58 AM
I was under the impression that books are never peer-reviewed. Only journal articles are.

Peer review is not mentioned in that link. Why do you think it was?

The author advertises it as such (http://freethoughtblogs.com/carrier/) (ctrl-F it), and this NT historian (http://www.patheos.com/blogs/exploringourmatrix/2013/07/finally-there-will-be-a-peer-reviewed-case-for-mythicism.html) anticipated it as such about a year ago.

Czarcasm
08-03-2014, 12:09 PM
The author advertises it as such (http://freethoughtblogs.com/carrier/) (ctrl-F it), and this NT historian (http://www.patheos.com/blogs/exploringourmatrix/2013/07/finally-there-will-be-a-peer-reviewed-case-for-mythicism.html) anticipated it as such about a year ago.And digging just a bit deeper, from this site: (http://freethoughtblogs.com/carrier/archives/4090)My new book, On the Historicity of Jesus, has passed peer review and is now under contract to be published by a major academic press specializing in biblical studies: Sheffield-Phoenix, the publishing house of the University of Sheffield (UK). I sought four peer review reports from major professors of New Testament or Early Christianity, and two have returned their reports, approving with revisions, and those revisions have been made. Since two peers is the standard number for academic publications, we can proceed. Two others missed the assigned deadline, but Iím still hoping to get their reports and Iíll do my best to meet any revisions they require as well.

Frylock
08-03-2014, 12:13 PM
And digging just a bit deeper, from this site: (http://freethoughtblogs.com/carrier/archives/4090)

Oh dammit, I have read that before, and completely forgot about.

My apologies everyone.

kayaker
08-03-2014, 12:25 PM
I think we first need to agree on a definition of "peer reviewed". If I submit a paper to a physics journal, they send the paper to physicists of their choosing. The Jesus text was read/reviewed by people the author chose. Big difference.

Ruken
08-03-2014, 01:06 PM
True, although I always recommend reviewers. And the chemistry journal editors I know often stick to the submitted list.

wolfpup
08-03-2014, 01:08 PM
I think we first need to agree on a definition of "peer reviewed". If I submit a paper to a physics journal, they send the paper to physicists of their choosing. The Jesus text was read/reviewed by people the author chose. Big difference.

Indeed. And it seems to me the author did so mainly in order to be able to make that claim.

I think what the OP is really asking is, and certainly the question that would be foremost in my mind, is how do we know a book like this is credible. And the answer, in brief, is that we don't. Ruken and Exapno Mapcase both raise relevant points in that there is no predetermined peer review process for books as there is for papers submitted to journals, which usually means none at all. Publication by a reputable house like a major university press generally indicates that the author meets some reasonable threshold of credentials and competence, but no more than that (especially if the author -- as sometimes happens -- ventures outside his field of primary competence).

I was curious about this book only because it's an area of casual interest to me, having read Reza Aslan's "Zealot", an impressive work which is diametrically at odds with Carrier's book. What I think can be fairly said is that Aslan is a qualified Biblical scholar while Carrier is not, and that Aslan wrote a scholarly book based on 20 years of research while Carrier seems motivated by his background as a crusading atheist. There are some comments on the book here (http://www.patheos.com/blogs/exploringourmatrix/2014/06/review-of-two-mythicist-books.html).

Exapno Mapcase
08-03-2014, 01:20 PM
I did some Googling and I can't find any evidence that Sheffield-Phoenix ever requires peer review for any of its books. It does for a journal that it publishes and those are the only hits that come up outside of Carrier's book.

My supposition is that Carrier made a special effort because of the nature of the book, solely to be able to preempt the expected criticism.

Frylock
08-03-2014, 01:56 PM
True, although I always recommend reviewers. And the chemistry journal editors I know often stick to the submitted list.

In the comments section to the linked blog entry, Carrier is intimating (I think, if I'm reading it correctly) that what occured with this book is like that--he recommended reviewers to the publisher, and the publisher took at least some of those recommendations.

Carrier won't say who the reviewers are, but characterizes them as "major NT scholars."

But it's not clear if there was any decision made by the reviewers and publisher together, or whether all the substantive conversation was between Carrier and the reviewers.

wolfpup
08-03-2014, 02:53 PM
Carrier won't say who the reviewers are...
The whole thing is fishy as hell. I think it's fair to say that something as outlandishly contentious as the claims that Carrier is making, if submitted as a journal paper, would receive far more scrutiny than a couple of reviewers of the author's own choosing. Garbage papers that expose peer review failures are not beloved by editorial boards.

I should mention, BTW, that Reza Aslan is hardly taking a conventional Christian stance on the subject, either, arguing as he does that Jesus was much more a political revolutionary than a religious figure, and has been much criticized for it by religious factions. The preposterousness of Carrier's thesis is that he never existed at all.

kayaker
08-03-2014, 03:30 PM
I could write a paper touting my ear candling techniques and submit it along with reviewers of my choosing. Just sayin.

njtt
08-03-2014, 03:32 PM
I have done peer reviewing of philosophy/cognitive science books (monographs) for publishers, sometimes on a detailed book proposal/outline, and, in one case, on a complete manuscript (which I strongly recommended should be rejected, incidentally). Unlike reviewing for journals (which I have also done) I was paid a small fee.

I think when we are talking about edited volumes of essays, rather than monographs, the publishers rely on the volume's editors to ensure academic quality. The essays will generally be by authors specifically invited by the editors. It is not as rigorous a process as the peer review used by journals, but there is still quality control by the editors (who will themselves have to have convinced the publishers that they have a sufficiently good reputation to be able to do the job and produce a salable book). Probably book editors do sometimes send out contributions for external peer review also.

I expect it works the same way, more or less, for academic history, although the fact that many historical monographs are written for a relatively wide audience, not just academics, may make a difference, such that more ordinary considerations of marketability to a popular audience come into play. I would not be at all surprised if quite a bit of academically unsound history gets published, because the publisher think it will sell, perhaps because its takes an extreme position, or flatters certain political prejudices. This is not going to happen so much in most other academic fields, where the demarcation between the professional and the popular literature is usually much clearer.

Frylock
08-03-2014, 03:34 PM
I emailed the publisher, as follows:

Can you briefly describe the peer review process that Richard Carrier's _On the Historicity of Jesus_ went through? Was that peer review process part of the decision as to whether to publish?

and got this reply:

We can assure anyone who asks that all our books are peer reviewed before being accepted. But we cannot undertake to describe the process just to any person who asks us to do so—life is too short.

kayaker
08-03-2014, 03:43 PM
and got this reply:

I prefer the frank, honest, "fuck off" reply.

wolfpup
08-03-2014, 03:45 PM
So let's summarize:


Richard Carrier has written a book that contradicts virtually all existing scholarship on the subject

He claims it was peer reviewed by two reviewers that he selected, but he refuses to say who they are

The publisher assures everyone that the book was peer reviewed, but refuses to describe the process or provide any further information

Does that about cover it?

Frylock
08-03-2014, 03:52 PM
Yes. But Carrier makes good arguments though. ;)

Ruken
08-03-2014, 04:16 PM
I think what the OP is really asking is, and certainly the question that would be foremost in my mind, is how do we know a book like this is credible. And the answer, in brief, is that we don't.If the OP is asking that, then the nature of the peer review of this book isn't really going to answer that question. Peer review has many purposes, but it doesn't make a book credible or not.

I write this as someone who has published some incredible work that passed peer review.

wolfpup
08-03-2014, 04:36 PM
Basically I agree, but I'd prefer to say that peer review wouldn't establish whether the book's conclusions were correct or not. Credibility is a subjective concept. It was really my delicate way of saying that the real question is whether or not the book was an utter pile of crap. Peer review basically validates some minimal threshold of methodological competence.

Exapno Mapcase
08-03-2014, 04:58 PM
I think it's fair to say that something as outlandishly contentious as the claims that Carrier is making, if submitted as a journal paper, would receive far more scrutiny than a couple of reviewers of the author's own choosing.

I don't understand why this is so outlandish. The contention that Jesus either did not exist, or was a personage used as a name to give life to beliefs, or was a composite character, has a long history. It is not prima facie foolish and in fact would be the default assumption were it not for Christianity's place in western society. The consensus that Jesus was real is based on nothing very concrete and mostly relies on supposition and assertion. I haven't taken part in the many, many discussions about it because I have no expertise but, to be frank, I can think of no other subject here that would be accepted on the flimsy arguments made for the affirmative. It's probably true that there is not sufficient evidence of any kind to prove the negative either, but that doesn't make the attempt outlandish.

wolfpup
08-03-2014, 05:19 PM
I'm certainly no expert, and no doubt the real story, if we could somehow ever know it, would be full of amazing surprises, and that's why the whole subject of something so fundamental to modern religion is so fascinating. And it's by no means certain that Aslan got it right, either, even in the most rudimentary outlines. But to me his main argument -- that Jesus was a political rebel in a time that we know was politically turbulent and had many such movements -- is a persuasive one, and that he was subsequently deified by a combination of historical distortion and historical revisionism, notably by the Gospel writers in the process of institutionalizing Christianity, is typical of how this sort of mythologizing works. But to claim that Jesus never existed at all just runs contrary to most secular Biblical scholarship. The claims of "peer review" and the secretiveness surrounding it is suggestive of shady and shoddy work.

Emiliana
08-03-2014, 05:35 PM
Authors typically submit suggestions for reviewers, but it's the publisher's job to select them, as well as to orchestrate the communication between author and reviewers which is theoretically anonymous.

One can't send a book to a couple of buddies, hash out a few comments and suggestions, and submit it to a credible publisher as, voila!, peer reviewed.

In my case (history), my book was reviewed by two people, as were most articles. One was reviewed by four, but that was an unusual practice (and it took fricking forever).

Practices vary a bit by discipline, but I imagine every discipline has its less than credible publishers. I don't know anything about the case at hand though.

Frylock
08-03-2014, 05:42 PM
Authors typically submit suggestions for reviewers, but it's the publisher's job to select them, as well as to orchestrate the communication between author and reviewers which is theoretically anonymous.

One can't send a book to a couple of buddies, hash out a few comments and suggestions, and submit it to a credible publisher as, voila!, peer reviewed.

In my case (history), my book was reviewed by two people, as were most articles. One was reviewed by four, but that was an unusual practice (and it took fricking forever).

Practices vary a bit by discipline, but I imagine every discipline has its less than credible publishers. I don't know anything about the case at hand though.

It looks like in the present case the reviewers weren't anonymous to the author. I don't know that that's fatal to the publisher's peer review credibility, but it looks like it's a little less credible than the standard methodology.

To me though, it seems like it's saying quite a bit that there are two (albeit unnamed) "major NT scholars" who'd say "yes, with revision, the field needs a book like this." As to doubts about how major the NT scholars in question are, I guess we can't know for sure but the publishers themselves seem to stand behind the claim and I'd imagine they feel _some_ pressure to be credible here...

Emiliana
08-03-2014, 05:53 PM
It looks like in the present case the reviewers weren't anonymous to the author. I don't know that that's fatal to the publisher's peer review credibility, but it looks like it's a little less credible than the standard methodology.

To me though, it seems like it's saying quite a bit that there are two (albeit unnamed) "major NT scholars" who'd say "yes, with revision, the field needs a book like this." As to doubts about how major the NT scholars in question are, I guess we can't know for sure but the publishers themselves seem to stand behind the claim and I'd imagine they feel _some_ pressure to be credible here...

It can be difficult to maintain anonymity. In small fields, people tend to all know each other's work. It's standard practice not to name the peer reviewers publicly though. Reviewers can, but aren't expected, to reveal their identities to either the author or the public (via a book blurb or the like). One would not recommend against publishing crappy work by a powerful leading scholar who decided to start phoning it in (it happens) if the review wasn't anonymous.

A lot rests on the prestige and credibility of the publishing house. Again, field dependent. But a publisher who published crap (as opposed to controversial but scholarly) work would not attract good historians. I'm speaking of the academic, not the popular, market here. I should have thought that a reputable scholarly publisher would have been willing to explain the per review process (although not reveal the names of the reviewer).

CalMeacham
08-03-2014, 06:58 PM
Before Oxford agreed to publish my book, they ran it by two academic reviewers. Only after they got their approval did they agree to publish. I'm not sure if it constitutes "peer review", though. I think they just wanted to see if I tried to publish some howlers.*

Oddly, when I submitted the Optics book, they didn't do anything of the sort. I guess they figured I knew enough about optics.












*Besides, I don't have any peers. That's not a brag -- it's just that there aren't a lot of physicists writing about mythology. The follow-up to the book that came out a couple of months ago, however, did get "peer reviewed" by classicists, something which made me realize the difference between physicists and classical scholars (one thing, for your edification -- classical scholars live and die by the Word. Make absolutely certain your etymologies and translations are absolutely perfect)

kayaker
08-03-2014, 10:31 PM
A lot rests on the prestige and credibility of the publishing house.

Ahh, a publishing house that replied:

We can assure anyone who asks that all our books are peer reviewed before being accepted. But we cannot undertake to describe the process just to any person who asks us to do soólife is too short.


Rather than just sending their editorial policies which, if they existed and sounded ok, would have been simpler than typing the quip.

njtt
08-03-2014, 11:33 PM
Ahh, a publishing house that replied:

Rather than just sending their editorial policies which, if they existed and sounded ok, would have been simpler than typing the quip.

This is not a big corporation we are talking about here. Although it is associated with a decent university, and needs to avoid publishing anything that might damage the university's reputation, the publishing "house" is probably a very small organization (and I think probably a fairly new one), with only a handful of staff. It very likely does not have a pre-written policy document about reviewing, and dealing with this enquiry may well have been handed off to an unpaid and inexperienced intern.

FWIW, from the outline linked in post #2 (http://www.sheffieldphoenix.com/showbook.asp?bkid=264), the book looks pretty legit to me (which is not to say it is right, of course), but it is not my area.

APB
08-04-2014, 06:45 AM
The section for potential authors on Sheffield Phoenix Press's website says, 'Manuscripts offered by the author will always be sent for evaluation to a series editor or a reader for the Press.'

http://www.sheffieldphoenix.com/authors.asp

That's absolutely standard for history books published by UK academic presses, although frequently it's two readers, often one of series editors and an outside reader.

Series editors will be major experts in the field covered by the series. That they are the editors of a particular series is not secret. Typically the one who will read it will be whichever of them is most qualified. If a second reader is used, this will usually be an outside expert. The series editors then take a collective decision on whether to accept the book. Obviously the views of the one who has read the book tends to carry the most weight. Given that one of the readers' reports will probably be by a series editor, it may not be too difficult for the author to work out who wrote that particular report. But the author will know that it's the series editors who have the final say anyway. The author may well also find out who the other reader was, as this is where the publishers get their blurbs from. Or the reader may just tell the author.

The process for collections of essays in history is usually much the same. Most academic presses will send them out to one outside reader. That's because the editor(s) of the collection will usually have invited the contributors to contribute and so cannot be considered independent.

Academic publishers will sometimes ask authors for recommendations for possible readers. But of course they do so in the full knowledge that authors will recommend names they think will be sympathetic. This can be used as a way of working out who not to send the book to.

Trade publishers are rather different. Some do retain prominent academics to read submissions. But this may involve them reading submissions on subjects on which they're not really an expert. Indeed, what the publisher may want from them is not a thorough check so much as a general sense of its quality. Others trade publishers do not use academic readers at all.

In any case, saying that a history book has been peer-reviewed does not mean that what the book says is correct. Criticisms that a reader might make are not necessarily taken on board. An author is entitled to try to convince the editors that those criticisms are wrong, irrelevant or simply a matter of opinion. Yet those criticisms could still prove to be well-founded. It is also not unknown for readers to recommend publication while saying that they personally disagree with the author's conclusions. Nor is there any requirement for a particular series or a publisher's wider list to be consistent. I'm pretty sure that many of the other authors published by Sheffield Phoenix Press will strongly disagree with Carrier's views. The only person involved in the whole publication process who must agree with Carrier's views is Carrier himself.

Finally, the real process of peer-review of history books happens after publication, when the book gets reviewed in academic journals. The glowingness of the readers' reports counts for little if the journal reviews are damning.

njtt
08-04-2014, 07:21 AM
. . . It is also not unknown for readers to recommend publication while saying that they personally disagree with the author's conclusions. . . .

I should hope not! As someone who has served both as a publisher's reader and a journal referee, I would consider myself to be acting seriously unethically if I recommended against a work solely because I disagreed with the author's conclusions. (I have, however, recommended rejection of an article because I regarded the conclusions to be so obviously true, and widely recognized as such, that there was no justification for arguing them at such length.)

Exapno Mapcase
08-04-2014, 07:22 AM
That's absolutely standard for history books published by UK academic presses, although frequently it's two readers, often one of series editors and an outside reader.
A series editor review is somewhat different from peer review. The series editor determines whether the book meets the standards of the press; the peer review determines whether the book meets the standards of the profession. You see the same distinction in academic journals where the journal editor accepts the paper and then sends it out for peer review. A bit oversimplified, of course.

Trade publishers are rather different. Some do retain prominent academics to read submissions. But this may involve them reading submissions on subjects on which they're not really an expert. Indeed, what the publisher may want from them is not a thorough check so much as a general sense of its quality. Others trade publishers do not use academic readers at all.
Trade publishers operate strictly on a CYA basis. I learned that with my first book. Since I didn't have a doctorate they sent the book out for an expert review, a nightmare line-by-line critique. After that I recruited doctors to sign their names to a foreward I would write. Presto, no review.

Voyager
08-04-2014, 02:50 PM
Trade publishers operate strictly on a CYA basis. I learned that with my first book. Since I didn't have a doctorate they sent the book out for an expert review, a nightmare line-by-line critique. After that I recruited doctors to sign their names to a foreward I would write. Presto, no review.

Not always. I've reviewed a book proposal from someone who not only had a PhD but who was very well known in the field. I think publishers often want second opinions.

However these kind of reviews, though paid, are usually not done anywhere near as completely as paper reviews (though it sounds like you drew an eager beaver.) If for no other reason papers go in when they are ready, but books have publication schedules.

Exapno Mapcase
08-04-2014, 08:20 PM
YMMV. No generalization about publishing goes very far. It's a profession in which idiosyncrasies are pushed to dottiness.