# Why can't I use Google to search the Straight Dope archives? **URL:** \ **Category:** About This Message Board\ **Created:** [January 7, 2005, 6:01pm UTC](https://boards.straightdope.com/t/why-cant-i-use-google-to-search-the-straight-dope-archives/283608 "2005-01-07T18:01:10Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Roland\_Deschain](https://avatars.discourse-cdn.com/v4/letter/r/73ab20/32.png) [@Roland\_Deschain](https://boards.straightdope.com/u/Roland_Deschain)\ **Post date:** [January 7, 2005, 6:01pm UTC](https://boards.straightdope.com/t/why-cant-i-use-google-to-search-the-straight-dope-archives/283608/1 "2005-01-07T18:01:10Z")
I often use the advanced search at Google to search within domains (including my University) because I find it works better than the in house engines. For example I had about an hour to kill today and I thought I would read some of the sex questions that had been asked at this board over the years. I did a search for “sex” from the “in house” engine and got bact an error message that my search term was too short (or long or just plain wrong). I figured I would overcome this by using Google and restricting the domani to [www.straightdope.com/sdmb](http://www.straightdope.com/sdmb) , but this didn’t work either. Is this an example of the so called “deep web” which typical search engines cannot reach? ---
**Author:** ![ultrafilter](https://avatars.discourse-cdn.com/v4/letter/u/3d9bf3/32.png) [@ultrafilter](https://boards.straightdope.com/u/ultrafilter)\ **Post date:** [January 7, 2005, 6:30pm UTC](https://boards.straightdope.com/t/why-cant-i-use-google-to-search-the-straight-dope-archives/283608/2 "2005-01-07T18:30:51Z")
That’s not a valid url. Try either [www.straightdope.com](http://www.straightdope.com) or [boards.straightdope.com](http://boards.straightdope.com). ---
**Author:** ![elmwood](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/elmwood/32/3060_2.png) [@elmwood](https://boards.straightdope.com/u/elmwood)\ **Post date:** [January 7, 2005, 6:41pm UTC](https://boards.straightdope.com/t/why-cant-i-use-google-to-search-the-straight-dope-archives/283608/3 "2005-01-07T18:41:41Z")
Google _is_ banned from the SDMB. The robots.txt file at [http://boards.straightdope.com/](http://boards.straightdope.com/) reads: ```auto User-agent: * Disallow: / ``` This blocks any bot – at least those that respect robiots.txt, which includes Google, MSN, Yahoo and others – from crawling the site. Why, though? Bandwidth? Database queries resulting from bot visits? A possible flood of traffic from outside the SDMB as they find relevant search results pointing here? ---
**Author:** ![TubaDiva](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/tubadiva/32/507_2.png) [@TubaDiva](https://boards.straightdope.com/u/TubaDiva)\ **Post date:** [January 7, 2005, 7:13pm UTC](https://boards.straightdope.com/t/why-cant-i-use-google-to-search-the-straight-dope-archives/283608/4 "2005-01-07T19:13:10Z")
Letting a search engine such as Google spider a site reduces capacity available for members and tends to slow or block access through system timeouts from overdemand. Our resources have nearly always been compromised as is and we wished to reserve as much access for the community as possible. Chicago Reader management also felt they should have more control over the content and wished to archive for themselves. They backup everything, so there’s no good reason to have it out there somewhere else, somewhere the Reader does not control. I should point out all this is copyrighted material and the Reader spends a great deal of time as is persuading cyberthieves from lifting material wholesale. You will also notice that none of the other search engines or [archive.org](http://archive.org) is allowed to spider here either. Our search engine is far from perfect, but it’s the best we have under the circumstances that prevail. If you are disaccommodated, we apologize. your humble TubaDiva Administrator ---
**Author:** ![Sean\_Factotum](https://avatars.discourse-cdn.com/v4/letter/s/50afbb/32.png) [@Sean\_Factotum](https://boards.straightdope.com/u/Sean_Factotum)\ **Post date:** [January 7, 2005, 9:49pm UTC](https://boards.straightdope.com/t/why-cant-i-use-google-to-search-the-straight-dope-archives/283608/5 "2005-01-07T21:49:21Z")
**Roland** , I have seen posters say several times that three-letter search values are too small for the search engine. That could be why “sex” gave you the answer it did. You may want to make it a larger word. ---
**Author:** ![Roland\_Deschain](https://avatars.discourse-cdn.com/v4/letter/r/73ab20/32.png) [@Roland\_Deschain](https://boards.straightdope.com/u/Roland_Deschain)\ **Post date:** [January 7, 2005, 11:38pm UTC](https://boards.straightdope.com/t/why-cant-i-use-google-to-search-the-straight-dope-archives/283608/6 "2005-01-07T23:38:02Z")
Where does that put my (and I think Google’s) dream of having every word ever published in every language archived at one easy to search location? Perhaps the as yet unwritten Patriot Act III will provide the legal foundation for such a plan to be implemented! In the interim I will have to try and think of words longer than three letters (there goes my Boggle game scores). ---
**Author:** ![alterego](https://avatars.discourse-cdn.com/v4/letter/a/6bbea6/32.png) [@alterego](https://boards.straightdope.com/u/alterego)\ **Post date:** [January 8, 2005, 6:29am UTC](https://boards.straightdope.com/t/why-cant-i-use-google-to-search-the-straight-dope-archives/283608/7 "2005-01-08T06:29:29Z")
Google search would be nice. In the meantime, appease yourself with \*sex. It works. ---
**Author:** ![dogbutler](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dogbutler/32/428_2.png) [@dogbutler](https://boards.straightdope.com/u/dogbutler)\ **Post date:** [January 8, 2005, 2:38pm UTC](https://boards.straightdope.com/t/why-cant-i-use-google-to-search-the-straight-dope-archives/283608/8 "2005-01-08T14:38:34Z")
> [@alterego](#): > > In the meantime, appease yourself with \*sex. It works. That’s your answer to everything, isn’t it? 😃 ---
**Author:** ![don\_t\_ask](https://avatars.discourse-cdn.com/v4/letter/d/e68b1a/32.png) [@don\_t\_ask](https://boards.straightdope.com/u/don_t_ask)\ **Post date:** [January 8, 2005, 2:56pm UTC](https://boards.straightdope.com/t/why-cant-i-use-google-to-search-the-straight-dope-archives/283608/9 "2005-01-08T14:56:14Z")
I’m a convert to \*sexual appeasement.