Reply
 
Thread Tools Display Modes
  #1  
Old 08-03-2014, 10:32 AM
Frylock is offline
Guest
 
Join Date: Jun 2001
Posts: 19,731

How Does Peer Review Work for Books in the Academic Discipline of History?


I know how peer review works for journal articles in Philosophy--author sends in a paper, editor sends it out to a relevant professional philosopher, that reviewer says whether to publish or not (or revise etc) and that's the decision.

I don't really know how peer review works for books, though. If someone has written a book in History (i.e. the academic discipline) and the book is said to be "peer reviewed," does that mean one reviewer took a look at it, or several, or what?
  #2  
Old 08-03-2014, 10:33 AM
Frylock is offline
Guest
 
Join Date: Jun 2001
Posts: 19,731
Specifically, I'm wondering what it amounts to that this book is peer reviewed.
  #3  
Old 08-03-2014, 11:13 AM
Ruken is offline
Charter Member
 
Join Date: May 2003
Location: DC
Posts: 7,462
I don't know that particular book, but it depends. Books like this may have a varying degree of fact checking depending on the preferences of the author and editor. An anonymous peer review such as what one expects in an academic journal would be unusual IME.
Edited, book-sized collections of separately-written chapters are in between. I have a chapter in one such book. The editor said he would send chapters out for peer review if he felt it warranted. AFAIK he did not with mine. It was a review article, so no new science was presented.
  #4  
Old 08-03-2014, 11:32 AM
Ruken is offline
Charter Member
 
Join Date: May 2003
Location: DC
Posts: 7,462
Please pardon the reading comprehension fail. You specifically asked about history, of which I have little knowledge.
  #5  
Old 08-03-2014, 11:52 AM
Exapno Mapcase is offline
Charter Member
 
Join Date: Mar 2002
Location: NY but not NYC
Posts: 31,454
I was under the impression that books are never peer-reviewed. Only journal articles are.

Peer review is not mentioned in that link. Why do you think it was?
  #6  
Old 08-03-2014, 11:58 AM
Frylock is offline
Guest
 
Join Date: Jun 2001
Posts: 19,731
Quote:
Originally Posted by Exapno Mapcase View Post
I was under the impression that books are never peer-reviewed. Only journal articles are.

Peer review is not mentioned in that link. Why do you think it was?
The author advertises it as such (ctrl-F it), and this NT historian anticipated it as such about a year ago.
  #7  
Old 08-03-2014, 12:09 PM
Czarcasm's Avatar
Czarcasm is online now
Charter Member
Charter Member
 
Join Date: Apr 1999
Location: Portland, OR
Posts: 61,686
Quote:
Originally Posted by Frylock View Post
The author advertises it as such (ctrl-F it), and this NT historian anticipated it as such about a year ago.
And digging just a bit deeper, from this site:
Quote:
My new book, On the Historicity of Jesus, has passed peer review and is now under contract to be published by a major academic press specializing in biblical studies: Sheffield-Phoenix, the publishing house of the University of Sheffield (UK). I sought four peer review reports from major professors of New Testament or Early Christianity, and two have returned their reports, approving with revisions, and those revisions have been made. Since two peers is the standard number for academic publications, we can proceed. Two others missed the assigned deadline, but Iím still hoping to get their reports and Iíll do my best to meet any revisions they require as well.
  #8  
Old 08-03-2014, 12:13 PM
Frylock is offline
Guest
 
Join Date: Jun 2001
Posts: 19,731
Quote:
Originally Posted by Czarcasm View Post
And digging just a bit deeper, from this site:
Oh dammit, I have read that before, and completely forgot about.

My apologies everyone.
  #9  
Old 08-03-2014, 12:25 PM
kayaker's Avatar
kayaker is offline
Member
 
Join Date: Jul 2009
Location: Western Pennsylvania
Posts: 32,325
I think we first need to agree on a definition of "peer reviewed". If I submit a paper to a physics journal, they send the paper to physicists of their choosing. The Jesus text was read/reviewed by people the author chose. Big difference.
  #10  
Old 08-03-2014, 01:06 PM
Ruken is offline
Charter Member
 
Join Date: May 2003
Location: DC
Posts: 7,462
True, although I always recommend reviewers. And the chemistry journal editors I know often stick to the submitted list.
  #11  
Old 08-03-2014, 01:08 PM
wolfpup's Avatar
wolfpup is online now
Guest
 
Join Date: Jan 2014
Posts: 10,840
Quote:
Originally Posted by kayaker View Post
I think we first need to agree on a definition of "peer reviewed". If I submit a paper to a physics journal, they send the paper to physicists of their choosing. The Jesus text was read/reviewed by people the author chose. Big difference.
Indeed. And it seems to me the author did so mainly in order to be able to make that claim.

I think what the OP is really asking is, and certainly the question that would be foremost in my mind, is how do we know a book like this is credible. And the answer, in brief, is that we don't. Ruken and Exapno Mapcase both raise relevant points in that there is no predetermined peer review process for books as there is for papers submitted to journals, which usually means none at all. Publication by a reputable house like a major university press generally indicates that the author meets some reasonable threshold of credentials and competence, but no more than that (especially if the author -- as sometimes happens -- ventures outside his field of primary competence).

I was curious about this book only because it's an area of casual interest to me, having read Reza Aslan's "Zealot", an impressive work which is diametrically at odds with Carrier's book. What I think can be fairly said is that Aslan is a qualified Biblical scholar while Carrier is not, and that Aslan wrote a scholarly book based on 20 years of research while Carrier seems motivated by his background as a crusading atheist. There are some comments on the book here.
  #12  
Old 08-03-2014, 01:20 PM
Exapno Mapcase is offline
Charter Member
 
Join Date: Mar 2002
Location: NY but not NYC
Posts: 31,454
I did some Googling and I can't find any evidence that Sheffield-Phoenix ever requires peer review for any of its books. It does for a journal that it publishes and those are the only hits that come up outside of Carrier's book.

My supposition is that Carrier made a special effort because of the nature of the book, solely to be able to preempt the expected criticism.
  #13  
Old 08-03-2014, 01:56 PM
Frylock is offline
Guest
 
Join Date: Jun 2001
Posts: 19,731
Quote:
Originally Posted by Ruken View Post
True, although I always recommend reviewers. And the chemistry journal editors I know often stick to the submitted list.
In the comments section to the linked blog entry, Carrier is intimating (I think, if I'm reading it correctly) that what occured with this book is like that--he recommended reviewers to the publisher, and the publisher took at least some of those recommendations.

Carrier won't say who the reviewers are, but characterizes them as "major NT scholars."

But it's not clear if there was any decision made by the reviewers and publisher together, or whether all the substantive conversation was between Carrier and the reviewers.
  #14  
Old 08-03-2014, 02:53 PM
wolfpup's Avatar
wolfpup is online now
Guest
 
Join Date: Jan 2014
Posts: 10,840
Quote:
Originally Posted by Frylock View Post
Carrier won't say who the reviewers are...
The whole thing is fishy as hell. I think it's fair to say that something as outlandishly contentious as the claims that Carrier is making, if submitted as a journal paper, would receive far more scrutiny than a couple of reviewers of the author's own choosing. Garbage papers that expose peer review failures are not beloved by editorial boards.

I should mention, BTW, that Reza Aslan is hardly taking a conventional Christian stance on the subject, either, arguing as he does that Jesus was much more a political revolutionary than a religious figure, and has been much criticized for it by religious factions. The preposterousness of Carrier's thesis is that he never existed at all.
  #15  
Old 08-03-2014, 03:30 PM
kayaker's Avatar
kayaker is offline
Member
 
Join Date: Jul 2009
Location: Western Pennsylvania
Posts: 32,325
I could write a paper touting my ear candling techniques and submit it along with reviewers of my choosing. Just sayin.
  #16  
Old 08-03-2014, 03:32 PM
njtt is offline
Guest
 
Join Date: Jul 2004
Location: U.K.
Posts: 12,068
I have done peer reviewing of philosophy/cognitive science books (monographs) for publishers, sometimes on a detailed book proposal/outline, and, in one case, on a complete manuscript (which I strongly recommended should be rejected, incidentally). Unlike reviewing for journals (which I have also done) I was paid a small fee.

I think when we are talking about edited volumes of essays, rather than monographs, the publishers rely on the volume's editors to ensure academic quality. The essays will generally be by authors specifically invited by the editors. It is not as rigorous a process as the peer review used by journals, but there is still quality control by the editors (who will themselves have to have convinced the publishers that they have a sufficiently good reputation to be able to do the job and produce a salable book). Probably book editors do sometimes send out contributions for external peer review also.

I expect it works the same way, more or less, for academic history, although the fact that many historical monographs are written for a relatively wide audience, not just academics, may make a difference, such that more ordinary considerations of marketability to a popular audience come into play. I would not be at all surprised if quite a bit of academically unsound history gets published, because the publisher think it will sell, perhaps because its takes an extreme position, or flatters certain political prejudices. This is not going to happen so much in most other academic fields, where the demarcation between the professional and the popular literature is usually much clearer.
  #17  
Old 08-03-2014, 03:34 PM
Frylock is offline
Guest
 
Join Date: Jun 2001
Posts: 19,731
I emailed the publisher, as follows:

Quote:
Can you briefly describe the peer review process that Richard Carrier's _On the Historicity of Jesus_ went through? Was that peer review process part of the decision as to whether to publish?
and got this reply:

Quote:
We can assure anyone who asks that all our books are peer reviewed before being accepted. But we cannot undertake to describe the process just to any person who asks us to do so—life is too short.
  #18  
Old 08-03-2014, 03:43 PM
kayaker's Avatar
kayaker is offline
Member
 
Join Date: Jul 2009
Location: Western Pennsylvania
Posts: 32,325
Quote:
Originally Posted by Frylock View Post
and got this reply:
I prefer the frank, honest, "fuck off" reply.
  #19  
Old 08-03-2014, 03:45 PM
wolfpup's Avatar
wolfpup is online now
Guest
 
Join Date: Jan 2014
Posts: 10,840
So let's summarize:
  • Richard Carrier has written a book that contradicts virtually all existing scholarship on the subject
  • He claims it was peer reviewed by two reviewers that he selected, but he refuses to say who they are
  • The publisher assures everyone that the book was peer reviewed, but refuses to describe the process or provide any further information
Does that about cover it?
  #20  
Old 08-03-2014, 03:52 PM
Frylock is offline
Guest
 
Join Date: Jun 2001
Posts: 19,731
Yes. But Carrier makes good arguments though.
  #21  
Old 08-03-2014, 04:16 PM
Ruken is offline
Charter Member
 
Join Date: May 2003
Location: DC
Posts: 7,462
Quote:
Originally Posted by wolfpup View Post
I think what the OP is really asking is, and certainly the question that would be foremost in my mind, is how do we know a book like this is credible. And the answer, in brief, is that we don't.
If the OP is asking that, then the nature of the peer review of this book isn't really going to answer that question. Peer review has many purposes, but it doesn't make a book credible or not.

I write this as someone who has published some incredible work that passed peer review.
  #22  
Old 08-03-2014, 04:36 PM
wolfpup's Avatar
wolfpup is online now
Guest
 
Join Date: Jan 2014
Posts: 10,840
Basically I agree, but I'd prefer to say that peer review wouldn't establish whether the book's conclusions were correct or not. Credibility is a subjective concept. It was really my delicate way of saying that the real question is whether or not the book was an utter pile of crap. Peer review basically validates some minimal threshold of methodological competence.

Last edited by wolfpup; 08-03-2014 at 04:41 PM.
  #23  
Old 08-03-2014, 04:58 PM
Exapno Mapcase is offline
Charter Member
 
Join Date: Mar 2002
Location: NY but not NYC
Posts: 31,454
Quote:
Originally Posted by wolfpup View Post
I think it's fair to say that something as outlandishly contentious as the claims that Carrier is making, if submitted as a journal paper, would receive far more scrutiny than a couple of reviewers of the author's own choosing.
I don't understand why this is so outlandish. The contention that Jesus either did not exist, or was a personage used as a name to give life to beliefs, or was a composite character, has a long history. It is not prima facie foolish and in fact would be the default assumption were it not for Christianity's place in western society. The consensus that Jesus was real is based on nothing very concrete and mostly relies on supposition and assertion. I haven't taken part in the many, many discussions about it because I have no expertise but, to be frank, I can think of no other subject here that would be accepted on the flimsy arguments made for the affirmative. It's probably true that there is not sufficient evidence of any kind to prove the negative either, but that doesn't make the attempt outlandish.
  #24  
Old 08-03-2014, 05:19 PM
wolfpup's Avatar
wolfpup is online now
Guest
 
Join Date: Jan 2014
Posts: 10,840
I'm certainly no expert, and no doubt the real story, if we could somehow ever know it, would be full of amazing surprises, and that's why the whole subject of something so fundamental to modern religion is so fascinating. And it's by no means certain that Aslan got it right, either, even in the most rudimentary outlines. But to me his main argument -- that Jesus was a political rebel in a time that we know was politically turbulent and had many such movements -- is a persuasive one, and that he was subsequently deified by a combination of historical distortion and historical revisionism, notably by the Gospel writers in the process of institutionalizing Christianity, is typical of how this sort of mythologizing works. But to claim that Jesus never existed at all just runs contrary to most secular Biblical scholarship. The claims of "peer review" and the secretiveness surrounding it is suggestive of shady and shoddy work.
  #25  
Old 08-03-2014, 05:35 PM
Emiliana's Avatar
Emiliana is offline
Guest
 
Join Date: Feb 2011
Posts: 1,620
Authors typically submit suggestions for reviewers, but it's the publisher's job to select them, as well as to orchestrate the communication between author and reviewers which is theoretically anonymous.

One can't send a book to a couple of buddies, hash out a few comments and suggestions, and submit it to a credible publisher as, voila!, peer reviewed.

In my case (history), my book was reviewed by two people, as were most articles. One was reviewed by four, but that was an unusual practice (and it took fricking forever).

Practices vary a bit by discipline, but I imagine every discipline has its less than credible publishers. I don't know anything about the case at hand though.
  #26  
Old 08-03-2014, 05:42 PM
Frylock is offline
Guest
 
Join Date: Jun 2001
Posts: 19,731
Quote:
Originally Posted by Emiliana View Post
Authors typically submit suggestions for reviewers, but it's the publisher's job to select them, as well as to orchestrate the communication between author and reviewers which is theoretically anonymous.

One can't send a book to a couple of buddies, hash out a few comments and suggestions, and submit it to a credible publisher as, voila!, peer reviewed.

In my case (history), my book was reviewed by two people, as were most articles. One was reviewed by four, but that was an unusual practice (and it took fricking forever).

Practices vary a bit by discipline, but I imagine every discipline has its less than credible publishers. I don't know anything about the case at hand though.
It looks like in the present case the reviewers weren't anonymous to the author. I don't know that that's fatal to the publisher's peer review credibility, but it looks like it's a little less credible than the standard methodology.

To me though, it seems like it's saying quite a bit that there are two (albeit unnamed) "major NT scholars" who'd say "yes, with revision, the field needs a book like this." As to doubts about how major the NT scholars in question are, I guess we can't know for sure but the publishers themselves seem to stand behind the claim and I'd imagine they feel _some_ pressure to be credible here...
  #27  
Old 08-03-2014, 05:53 PM
Emiliana's Avatar
Emiliana is offline
Guest
 
Join Date: Feb 2011
Posts: 1,620
Quote:
Originally Posted by Frylock View Post
It looks like in the present case the reviewers weren't anonymous to the author. I don't know that that's fatal to the publisher's peer review credibility, but it looks like it's a little less credible than the standard methodology.

To me though, it seems like it's saying quite a bit that there are two (albeit unnamed) "major NT scholars" who'd say "yes, with revision, the field needs a book like this." As to doubts about how major the NT scholars in question are, I guess we can't know for sure but the publishers themselves seem to stand behind the claim and I'd imagine they feel _some_ pressure to be credible here...
It can be difficult to maintain anonymity. In small fields, people tend to all know each other's work. It's standard practice not to name the peer reviewers publicly though. Reviewers can, but aren't expected, to reveal their identities to either the author or the public (via a book blurb or the like). One would not recommend against publishing crappy work by a powerful leading scholar who decided to start phoning it in (it happens) if the review wasn't anonymous.

A lot rests on the prestige and credibility of the publishing house. Again, field dependent. But a publisher who published crap (as opposed to controversial but scholarly) work would not attract good historians. I'm speaking of the academic, not the popular, market here. I should have thought that a reputable scholarly publisher would have been willing to explain the per review process (although not reveal the names of the reviewer).
  #28  
Old 08-03-2014, 06:58 PM
CalMeacham's Avatar
CalMeacham is offline
Member
 
Join Date: May 2000
Location: Massachusetts
Posts: 44,400
Before Oxford agreed to publish my book, they ran it by two academic reviewers. Only after they got their approval did they agree to publish. I'm not sure if it constitutes "peer review", though. I think they just wanted to see if I tried to publish some howlers.*

Oddly, when I submitted the Optics book, they didn't do anything of the sort. I guess they figured I knew enough about optics.












*Besides, I don't have any peers. That's not a brag -- it's just that there aren't a lot of physicists writing about mythology. The follow-up to the book that came out a couple of months ago, however, did get "peer reviewed" by classicists, something which made me realize the difference between physicists and classical scholars (one thing, for your edification -- classical scholars live and die by the Word. Make absolutely certain your etymologies and translations are absolutely perfect)
  #29  
Old 08-03-2014, 10:31 PM
kayaker's Avatar
kayaker is offline
Member
 
Join Date: Jul 2009
Location: Western Pennsylvania
Posts: 32,325
Quote:
Originally Posted by Emiliana View Post
A lot rests on the prestige and credibility of the publishing house.
Ahh, a publishing house that replied:

Quote:
We can assure anyone who asks that all our books are peer reviewed before being accepted. But we cannot undertake to describe the process just to any person who asks us to do soólife is too short.
Rather than just sending their editorial policies which, if they existed and sounded ok, would have been simpler than typing the quip.
  #30  
Old 08-03-2014, 11:33 PM
njtt is offline
Guest
 
Join Date: Jul 2004
Location: U.K.
Posts: 12,068
Quote:
Originally Posted by kayaker View Post
Ahh, a publishing house that replied:

Rather than just sending their editorial policies which, if they existed and sounded ok, would have been simpler than typing the quip.
This is not a big corporation we are talking about here. Although it is associated with a decent university, and needs to avoid publishing anything that might damage the university's reputation, the publishing "house" is probably a very small organization (and I think probably a fairly new one), with only a handful of staff. It very likely does not have a pre-written policy document about reviewing, and dealing with this enquiry may well have been handed off to an unpaid and inexperienced intern.

FWIW, from the outline linked in post #2, the book looks pretty legit to me (which is not to say it is right, of course), but it is not my area.

Last edited by njtt; 08-03-2014 at 11:37 PM.
  #31  
Old 08-04-2014, 06:45 AM
APB is offline
Member
 
Join Date: Apr 2000
Posts: 2,117
The section for potential authors on Sheffield Phoenix Press's website says, 'Manuscripts offered by the author will always be sent for evaluation to a series editor or a reader for the Press.'

http://www.sheffieldphoenix.com/authors.asp

That's absolutely standard for history books published by UK academic presses, although frequently it's two readers, often one of series editors and an outside reader.

Series editors will be major experts in the field covered by the series. That they are the editors of a particular series is not secret. Typically the one who will read it will be whichever of them is most qualified. If a second reader is used, this will usually be an outside expert. The series editors then take a collective decision on whether to accept the book. Obviously the views of the one who has read the book tends to carry the most weight. Given that one of the readers' reports will probably be by a series editor, it may not be too difficult for the author to work out who wrote that particular report. But the author will know that it's the series editors who have the final say anyway. The author may well also find out who the other reader was, as this is where the publishers get their blurbs from. Or the reader may just tell the author.

The process for collections of essays in history is usually much the same. Most academic presses will send them out to one outside reader. That's because the editor(s) of the collection will usually have invited the contributors to contribute and so cannot be considered independent.

Academic publishers will sometimes ask authors for recommendations for possible readers. But of course they do so in the full knowledge that authors will recommend names they think will be sympathetic. This can be used as a way of working out who not to send the book to.

Trade publishers are rather different. Some do retain prominent academics to read submissions. But this may involve them reading submissions on subjects on which they're not really an expert. Indeed, what the publisher may want from them is not a thorough check so much as a general sense of its quality. Others trade publishers do not use academic readers at all.

In any case, saying that a history book has been peer-reviewed does not mean that what the book says is correct. Criticisms that a reader might make are not necessarily taken on board. An author is entitled to try to convince the editors that those criticisms are wrong, irrelevant or simply a matter of opinion. Yet those criticisms could still prove to be well-founded. It is also not unknown for readers to recommend publication while saying that they personally disagree with the author's conclusions. Nor is there any requirement for a particular series or a publisher's wider list to be consistent. I'm pretty sure that many of the other authors published by Sheffield Phoenix Press will strongly disagree with Carrier's views. The only person involved in the whole publication process who must agree with Carrier's views is Carrier himself.

Finally, the real process of peer-review of history books happens after publication, when the book gets reviewed in academic journals. The glowingness of the readers' reports counts for little if the journal reviews are damning.
  #32  
Old 08-04-2014, 07:21 AM
njtt is offline
Guest
 
Join Date: Jul 2004
Location: U.K.
Posts: 12,068
Quote:
Originally Posted by APB View Post
. . . It is also not unknown for readers to recommend publication while saying that they personally disagree with the author's conclusions. . . .
I should hope not! As someone who has served both as a publisher's reader and a journal referee, I would consider myself to be acting seriously unethically if I recommended against a work solely because I disagreed with the author's conclusions. (I have, however, recommended rejection of an article because I regarded the conclusions to be so obviously true, and widely recognized as such, that there was no justification for arguing them at such length.)
  #33  
Old 08-04-2014, 07:22 AM
Exapno Mapcase is offline
Charter Member
 
Join Date: Mar 2002
Location: NY but not NYC
Posts: 31,454
Quote:
Originally Posted by APB View Post
That's absolutely standard for history books published by UK academic presses, although frequently it's two readers, often one of series editors and an outside reader.
A series editor review is somewhat different from peer review. The series editor determines whether the book meets the standards of the press; the peer review determines whether the book meets the standards of the profession. You see the same distinction in academic journals where the journal editor accepts the paper and then sends it out for peer review. A bit oversimplified, of course.

Quote:
Trade publishers are rather different. Some do retain prominent academics to read submissions. But this may involve them reading submissions on subjects on which they're not really an expert. Indeed, what the publisher may want from them is not a thorough check so much as a general sense of its quality. Others trade publishers do not use academic readers at all.
Trade publishers operate strictly on a CYA basis. I learned that with my first book. Since I didn't have a doctorate they sent the book out for an expert review, a nightmare line-by-line critique. After that I recruited doctors to sign their names to a foreward I would write. Presto, no review.
  #34  
Old 08-04-2014, 02:50 PM
Voyager's Avatar
Voyager is offline
Charter Member
 
Join Date: Aug 2002
Location: Deep Space
Posts: 46,169
Quote:
Originally Posted by Exapno Mapcase View Post
Trade publishers operate strictly on a CYA basis. I learned that with my first book. Since I didn't have a doctorate they sent the book out for an expert review, a nightmare line-by-line critique. After that I recruited doctors to sign their names to a foreward I would write. Presto, no review.
Not always. I've reviewed a book proposal from someone who not only had a PhD but who was very well known in the field. I think publishers often want second opinions.

However these kind of reviews, though paid, are usually not done anywhere near as completely as paper reviews (though it sounds like you drew an eager beaver.) If for no other reason papers go in when they are ready, but books have publication schedules.
  #35  
Old 08-04-2014, 08:20 PM
Exapno Mapcase is offline
Charter Member
 
Join Date: Mar 2002
Location: NY but not NYC
Posts: 31,454
YMMV. No generalization about publishing goes very far. It's a profession in which idiosyncrasies are pushed to dottiness.
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Forum Jump


All times are GMT -5. The time now is 12:01 AM.

Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2019, vBulletin Solutions, Inc.

Send questions for Cecil Adams to: cecil@straightdope.com

Send comments about this website to: webmaster@straightdope.com

Terms of Use / Privacy Policy

Advertise on the Straight Dope!
(Your direct line to thousands of the smartest, hippest people on the planet, plus a few total dipsticks.)

Copyright © 2018 STM Reader, LLC.

 
Copyright © 2017