# ETAONI SHRDLU **URL:** \ **Category:** Cecil's Columns/Staff Reports\ **Created:** [March 23, 2001, 8:39pm UTC](https://boards.straightdope.com/t/etaoni-shrdlu/59208 "2001-03-23T20:39:59Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![BVCC](https://avatars.discourse-cdn.com/v4/letter/b/839c29/32.png) [@BVCC](https://boards.straightdope.com/u/BVCC)\ **Post date:** [March 23, 2001, 8:39pm UTC](https://boards.straightdope.com/t/etaoni-shrdlu/59208/1 "2001-03-23T20:39:59Z")
Dear Cecil, In reporting the result of his letter-counting project, your correspondent Skip Newhall say, “You will notice that the order of n and i is reversed from that in etaoin, though the counts differ by less than 0.2 percent, a statistically insignificant difference.” As you no doubt noticed, he’s wrong. With a sample this large, virtually any difference is bound to be statistically significant. In fact, the P-value is something less than 0.0000001. That is (assuming his sampling design is sound) we can be reasonably certain that the observed difference in the two letters represents an actual difference in the English language. If n and i in fact occur at the same rate in English, we would find n ahead of i by as much as it was in Skip’s sample less than 1 time in 10 million. If i is in fact more common than n, Skip’s sample is correspondingly less likely. BVCC ---
**Author:** ![yabob](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/yabob/32/2821_2.png) [@yabob](https://boards.straightdope.com/u/yabob)\ **Post date:** [March 23, 2001, 9:50pm UTC](https://boards.straightdope.com/t/etaoni-shrdlu/59208/2 "2001-03-23T21:50:58Z")
In fact, assuming the sampling design is sound is a BIG question mark, IMO. I’ll bet that letter distribution would vary drastically with source - comparing classic literature versus popular magazine copy loaded with modern day trade names, for instance (lots more x’s). Or current computer trade rags - lot’s of j’s from java and acronyms derived from java-this and java-that. How you are going to weight all those sources to arrive at “average” English usage is questionable. In fact the frequencies from the “Brown Corpus”, often used as a sample for things like this comes out as: etaoin srhldcu according to this source: [http://lists.village.virginia.edu/lists\_archive/Humanist/v05/0181.html](http://lists.village.virginia.edu/lists_archive/Humanist/v05/0181.html) ---
**Author:** ![Arnold\_Winkelried](https://avatars.discourse-cdn.com/v4/letter/a/3d9bf3/32.png) [@Arnold\_Winkelried](https://boards.straightdope.com/u/Arnold_Winkelried)\ **Post date:** [March 23, 2001, 9:56pm UTC](https://boards.straightdope.com/t/etaoni-shrdlu/59208/3 "2001-03-23T21:56:24Z")
Welcome to the SDMB, and thank you for posting your comment. Please include a link to Cecil’s column if it’s on the straight dope web site. To include a link, it can be as simple as including the web page location in your post (make sure there is a space **before** and **after** the text of the URL). Cecil’s original column can be found on-line at this link: [What’s the origin of the mysterious phrase “etaoin shrdlu”?](http://www.straightdope.com/classics/a2_262a.html) The follow-up can be found here: [Another note regarding ETAOIN SHRDLU](http://www.straightdope.com/columns/010323.html) (near the bottom of the page) * * * moderator, «[**Comments on Cecil’s Columns**](http://boards.straightdope.com/sdmb/forumdisplay.php?forumid=1)» \*\* ---
**Author:** ![tracer](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/tracer/32/20578_2.png) [@tracer](https://boards.straightdope.com/u/tracer)\ **Post date:** [March 24, 2001, 8:51pm UTC](https://boards.straightdope.com/t/etaoni-shrdlu/59208/4 "2001-03-24T20:51:47Z")
Arnold Winkelried wrote: > [@](#): > > The follow-up can be found here: [Another note regarding ETAOIN SHRDLU](http://www.straightdope.com/columns/010323.html) (near the bottom of the page) I’m surprised Cecil didn’t notice something important about the, ahem, “English-language” works selected for survey by Skip Newhall. They included:[ul][li]_Crime and Punishment_ (Fyodor Dostoevsky)[/li][li]_The Iliad, The Odyssey_ (Homer)[/li][li]_Peer Gynt_ (Henrik Ibsen)[/li][li]_Faust_ (Johann Wolfgang von Goethe)[/li][li]_History of the Peloponnesian War_ (Thucydides)[/li][li]_The Forged Coupon_ (Leo Tolstoy)[/li][li]Several writings of Karl Marx[/li][li]_Don Quixote_ (Miguel de Cervantes)[/li][li]Works by Plato and Virgil[/ul][/li] **… none of which were originally written in English!** ---
**Author:** ![Arnold\_Winkelried](https://avatars.discourse-cdn.com/v4/letter/a/3d9bf3/32.png) [@Arnold\_Winkelried](https://boards.straightdope.com/u/Arnold_Winkelried)\ **Post date:** [March 25, 2001, 7:10pm UTC](https://boards.straightdope.com/t/etaoni-shrdlu/59208/5 "2001-03-25T19:10:20Z")
I noticed that also, tracer, but one could presumably assume that the translaters had a sufficient command of the english language that the translation can be considered to be “normal” english writing. Of course, there would be some foreign patronyms in the books that could somewhat skew the results, but in a long book such as “Crime and Punishment” the ratio of foreign patronyms to english words would probably be tiny. The main problem I saw with the sampling is that, from the sources mentioned, it seems pretty clear that this person wrote a computer program for the letter count, and used sources available on the internet: classic novels which can be found for example at the [Gutenberg project](http://www.gutenberg.net), and then newspaper and magazine articles. But I would guess that modern novels are under-represented in the sample since those could not be found on the internet. ---
**Author:** ![tracer](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/tracer/32/20578_2.png) [@tracer](https://boards.straightdope.com/u/tracer)\ **Post date:** [March 26, 2001, 11:20pm UTC](https://boards.straightdope.com/t/etaoni-shrdlu/59208/6 "2001-03-26T23:20:45Z")
I’ve written a few novel-length stories and posted them on my website, if Skip Newhall wants to use them as representative literary samples. 😉 ---
**Author:** ![raoulortega](https://avatars.discourse-cdn.com/v4/letter/r/278dde/32.png) [@raoulortega](https://boards.straightdope.com/u/raoulortega)\ **Post date:** [March 27, 2001, 7:29pm UTC](https://boards.straightdope.com/t/etaoni-shrdlu/59208/7 "2001-03-27T19:29:29Z")
Not only can you gather statistics on pairs, but you can, through tree structures, gather stats on much longer character strings. Twenty-five years ago I wrote a program in Fortran on a PDP-10 that went to 7 or 8 character strings. (It ran out of memory, which if I remember correctly was 512K, with 10Meg hard drives). Then you write a second program which takes these probability tables as input, and generates random text. Unlike the “million monkeys at a million typewriters”, this program can actually generate snippets of recognizable text. You have to include space as a valid character, by the way, in order to get words to begin and terminate properly. You can also include punctuation, too. ---
**Author:** ![ndorward](https://avatars.discourse-cdn.com/v4/letter/n/c2a13f/32.png) [@ndorward](https://boards.straightdope.com/u/ndorward)\ **Post date:** [March 29, 2001, 5:48am UTC](https://boards.straightdope.com/t/etaoni-shrdlu/59208/8 "2001-03-29T05:48:41Z")
I take it that **raoulortega** was attempting a version of Rubenking’s BREKDOWN algorithm for the creation of new texts out of a preexistent corpus. The curious may find an article by John Tranter at: [http://www.austlit.com/johntranter/prose/nonfiction/brekdown.html](http://www.austlit.com/johntranter/prose/nonfiction/brekdown.html) which discusses the algorithm & provides two rather entertaining samples of its work–new poems created in the style of Matthew Arnold & John Ashbery. --N ---
**Author:** ![GalaciDaliciducleic\_Acid](https://avatars.discourse-cdn.com/v4/letter/g/f6c823/32.png) [@GalaciDaliciducleic\_Acid](https://boards.straightdope.com/u/GalaciDaliciducleic_Acid)\ **Post date:** [April 9, 2001, 2:40am UTC](https://boards.straightdope.com/t/etaoni-shrdlu/59208/9 "2001-04-09T02:40:30Z")
In one of those interesting little coincidences, I’m just reading an early short story by the irrepressible Thomas Pynchon, who is known for giving his characters fabulous names like Tyrone Slothrop, Oedipa Maas, Clayton “Bloody” Chiclitz, and Blodgett Waxwing. This one features a character named Etienne Cherdlu, who is, moreover, given to signing his name “80N”. Just thought it was cute.