So an acquaintence and I have been discussing the Documentary Hypothesis. He was raving about Radday’s book on Genesis and I quoted the arguments against it from Friedman’s “Some Recent Non-Arguments Against the Documentary Hypothesis” (the whole essay’s available via Google Books; it’s in a festrichft to Menahem Haran ). My acquaintence has recently made the following statistical argument. Being that I know nothing about statistics, I was wondering if somebody could explain to me if what he’s saying is true or false and why? I have slightly changed his words in order to protect his identity.
“…Friedman…asserts that Radday brings bad proofs for J, E, and P for his model…even if Radday was off in many places that doesn’t necessarily mean anything. I have a lot of experience with Statistics…and there is a type of study known as meta-analysis. In effect it is a method of aggregating data from many studies to attain general results. A lot of the aspects of this statistical method are much more recent (past 20 years or so). There is a confound with this measure, studies which haven’t been conducted yet that might debunk your theory. So statisticians created models that can test how many studies would be necessary to debunk your theory to prove that your meta-analysis is worthless. In general when you have enough data (in the thousands or so) it takes hundreds of studies to debunk the findings of a meta-analysis. When you have tens of thousands and a strong percentage (you never get 80% in meta-analysis studies), it would take a lot of data to undue your results. Radday’s study is very similar to a meta-analysis in that he is aggregating data from many different sources. Just as a rough estimate, based on his effect size (which is incredibly high) and data it would probably require well over a thousand mistakes on his part (I doubt anyone would claim he was so way off with his computer model) to debunk his results in the slightest. It would take much more than that to completely disprove him. These are some more recent statistical models that Friedman probably didn’t have as much access to (or I doubt even knows about)…
Friedman is not an authority to argue this point. He is unaware of effect sizes and how sometimes it can take a preponderance of data to disprove something…On the topic of argument of methodologies…Chockalingam Viswesvaran…[who] has created statistical methods argues against the use of Factor Analysis. Many people argue him on this, and he’s refused to accept articles when this type of data is employed but others do, but all of these professors are well respected in their fields…[Radday’s] a professor at the Technion, he must have some idea of the use of statistics.”
I haven’t studied the topic, but your friend’s argument is crap.
First and foremost, a meta-analysis has to be properly designed and conducted before a judgement can be made on its validity. You can’t just say “I think if a meta-analysis were conducted it would support my theory.” What data did Radday collect, what statistical analysis was done on that data, and what were the results? He says Radday has a high effect size, what does that even mean? What effect was being measured? Saying Radday did “something similar to a meta-analysis” doesn’t make it one. Look at some of the weaknesses of meta-analysis, particularily agenda-driven bias.
The latter part of his argument is simply an ad hominem attack on Friedman and an appeal to authority for Radday. What evidence does your friend about either’s statistical knowledge? He claims Friedman is unaware, how does he know? He claims Radday must know because he’s a professor - is he a professor of statistics?
I should clarify: When my acquaintence writes, “…Friedman…asserts that Radday brings bad proofs for J, E, and P for his model,” he means that Friedman is asserting that Radday has placed a large number of verses with the wrong sources. Friedman wrote, “Radday and his colleagues had given whole chapters of J to E, and E to J, and then came up with an 82-percent coalescence of J’s and E’s ‘language behavior’! What else could we expect? If you give whole chapters of…Twain to…Melville and [vice-versa], and then do a study of their ‘language behavior,’ …Twain and Melville are going to start looking strangely alike.” (for more, see p. 95 of the book)
I am not a statistician and I have never studied meta-analysis methodology. But it seems to me that if some sort of publication bias existed, then it wouldn’t take much data to undo your results provided you demonstrated that your statistical exercise was sufficiently different from those conducted in the past.
Meta-analysis is being used by analogy: I wouldn’t know if the analogy is proper.
3. Please define “Documentary Hypothesis”.
We’re speaking in terms of multiple authorship of the Bible, particularly 4 authors (J,E, P, & D) with a Redactor [R].
I believe that the hypothesis specifically deals with authorship of the first five books of the Hebrew bible, attributed by tradition to Moses. The letters stand for Yawhist (J), Elohist (E), Dueteronomist (D) and Priest (P). Yawhist and Elohist refer to the names for God used by those writers. The Redactor(R) is someone who is believed to have put disparate sources together and done some other editing. See Wikipedia entry.
I am a statistician, and for the most part your friend’s argument looks like bunk, but since I don’t fully understand Friedman’s criticism and how it relates to the documentary hypothesis and meta analysis, I really can’t comment too much. Can the OP lay out more clearly the arguments that are being made for those of us who are new to the controversy?
Buck:
Radday says that his statistical study shows that it is most likely that the Bible is singularly authored. He took lines which are attributed by Bible scholars to different authors (J, E, P, and D), fed them into a computer as separate works, and came out with a high percentage chance that some of the works were all written by the same author (those he doesn’t come out with such a percentage for, he has a separate argument, which Friedman addresses elsewhere. But that’s not relevant to my query here).
Friedman points to a number of places where Radday’s study takes lines commonly attributed to author “J” and attributes them to author “E,” and similar mistakes. Friedman writes, “Radday and his colleagues had given whole chapters of J to E, and E to J, and then came up with an 82-percent coalescence of J’s and E’s ‘language behavior’! What else could we expect? If you give whole chapters of…Twain to…Melville and [vice-versa], and then do a study of their ‘language behavior,’ …Twain and Melville are going to start looking strangely alike.” My friend is arguing that even if it’s true that Radday misatrributed the lines, the “meta-analysis” argument acts as insurance.
Thanks Bpelta, short answer: I was right your friends argument ** is** bunk. Meta-analysis involves the combining of several studies taken from independent sources and carefully combining them to create a combined study which has greater power to detect subtle differences than any of the studies individually. I assume the multiple studies that your friend is mentioning would be multiple analyses from different scholars of the bible. These are clearly not independent, since they use the same base data set (the bible) and since they also probably read each other’s analyses and incorporated them into their own. So I don’t think meta-analysis is appropriate here. Even if it was I still don’t understand the rest of your friends argument, he seems to be talking about some form of sample size/power calculation but its not at all clear.
Based on what I know about textual factor analysis it would seem to me that miss-attribution of passages of text would fatally flaw the analysis.