# Let's all watch Google crawl the Dope **URL:** \ **Category:** Miscellaneous and Personal Stuff I Must Share\ **Created:** [March 15, 2009, 11:39pm UTC](https://boards.straightdope.com/t/lets-all-watch-google-crawl-the-dope/489566 "2009-03-15T23:39:01Z")\ **Posts on this page:** 13\ **Page:** 2
**Author:** ![GuanoLad](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/guanolad/32/42_2.png) [@GuanoLad](https://boards.straightdope.com/u/GuanoLad)\ **Post date:** [March 16, 2009, 8:31am UTC](https://boards.straightdope.com/t/lets-all-watch-google-crawl-the-dope/489566/21 "2009-03-16T08:31:06Z")
Is this a precursor to shifting all searching via Google? Because if that happens, I fear we will no longer have as much control over what kinds of results we get. If I vanity search myself, I can restrict it to time periods, and forums, and then the results appear in chronological order. But Google does not have that kind of set up, so I’ll get results from 5 years ago, 9 years ago, 3 weeks ago, and 25 minutes ago, in a random order. This is not what I want at all. ---
**Author:** ![Mighty\_Girl](https://avatars.discourse-cdn.com/v4/letter/m/7bcc69/32.png) [@Mighty\_Girl](https://boards.straightdope.com/u/Mighty_Girl)\ **Post date:** [March 16, 2009, 12:58pm UTC](https://boards.straightdope.com/t/lets-all-watch-google-crawl-the-dope/489566/22 "2009-03-16T12:58:43Z")
> [@samclem](#): > > “Bring it on” to quote someone. 🙂 How did that work out for you? 🙂 > [@GuanoLad](#): > > Is this a precursor to shifting all searching via Google? > > Because if that happens, I fear we will no longer have as much control over what kinds of results we get. If I vanity search myself, I can restrict it to time periods, and forums, and then the results appear in chronological order. > > But Google does not have that kind of set up, so I’ll get results from 5 years ago, 9 years ago, 3 weeks ago, and 25 minutes ago, in a random order. > > This is not what I want at all. I searched for my username and the posts appear to be in date order, from newer to older. ---
**Author:** ![Revtim](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/revtim/32/1042_2.png) [@Revtim](https://boards.straightdope.com/u/Revtim)\ **Post date:** [March 16, 2009, 6:48pm UTC](https://boards.straightdope.com/t/lets-all-watch-google-crawl-the-dope/489566/23 "2009-03-16T18:48:20Z")
> [@GuanoLad](#): > > Is this a precursor to shifting all searching via Google? > > Because if that happens, I fear we will no longer have as much control over what kinds of results we get. If I vanity search myself, I can restrict it to time periods, and forums, and then the results appear in chronological order. > > But Google does not have that kind of set up, so I’ll get results from 5 years ago, 9 years ago, 3 weeks ago, and 25 minutes ago, in a random order. > > This is not what I want at all. I’ll just be happy to get results, and not an error message. And if I want to alter my search, not wait 5 frickin minutes to try again. ---
**Author:** ![Munch](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/munch/32/5281_2.png) [@Munch](https://boards.straightdope.com/u/Munch)\ **Post date:** [March 16, 2009, 7:07pm UTC](https://boards.straightdope.com/t/lets-all-watch-google-crawl-the-dope/489566/24 "2009-03-16T19:07:51Z")
> [@GuanoLad](#): > > If I vanity search myself, I can restrict it to time periods, and forums, and then the results appear in chronological order. > > But Google does not have that kind of set up, so I’ll get results from 5 years ago, 9 years ago, 3 weeks ago, and 25 minutes ago, in a random order. False. If you want to do a Cafe Society search, then you include “cafe society” in your search criteria. If you want to limit it to the last month, then you go into advanced search and limit it to the last month. ---
**Author:** ![alterego](https://avatars.discourse-cdn.com/v4/letter/a/6bbea6/32.png) [@alterego](https://boards.straightdope.com/u/alterego)\ **Post date:** [March 19, 2009, 2:27am UTC](https://boards.straightdope.com/t/lets-all-watch-google-crawl-the-dope/489566/25 "2009-03-19T02:27:57Z")
Mar 13 - 3,500 Mar 14 - 4,700 Mar 15 - 8,100 Mar 16 - 10,000 Mar 17 - 12,100 Mar 18 - 15,000 At this rate it will take Google half a year to crawl all 500,000 threads. I suspect we will see some major spikes in Google’s crawling rate once it “realizes” how much original content is here. ---
**Author:** ![GuanoLad](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/guanolad/32/42_2.png) [@GuanoLad](https://boards.straightdope.com/u/GuanoLad)\ **Post date:** [March 19, 2009, 2:43pm UTC](https://boards.straightdope.com/t/lets-all-watch-google-crawl-the-dope/489566/26 "2009-03-19T14:43:24Z")
Wow, I got a little bit of a pile-on there. I use the Google Toolbar exclusively, so had no idea there was an Advanced Search for Google. ---
**Author:** ![Zeldar](https://avatars.discourse-cdn.com/v4/letter/z/b77776/32.png) [@Zeldar](https://boards.straightdope.com/u/Zeldar)\ **Post date:** [March 21, 2009, 7:31pm UTC](https://boards.straightdope.com/t/lets-all-watch-google-crawl-the-dope/489566/27 "2009-03-21T19:31:33Z")
Just a post to keep track of this thread. ---
**Author:** ![alterego](https://avatars.discourse-cdn.com/v4/letter/a/6bbea6/32.png) [@alterego](https://boards.straightdope.com/u/alterego)\ **Post date:** [March 21, 2009, 8:24pm UTC](https://boards.straightdope.com/t/lets-all-watch-google-crawl-the-dope/489566/28 "2009-03-21T20:24:10Z")
Mar 20 - 18,000 Mar 21 - 20,800 ---
**Author:** ![astro](https://avatars.discourse-cdn.com/v4/letter/a/9dc877/32.png) [@astro](https://boards.straightdope.com/u/astro)\ **Post date:** [March 22, 2009, 1:42am UTC](https://boards.straightdope.com/t/lets-all-watch-google-crawl-the-dope/489566/29 "2009-03-22T01:42:30Z")
Ummm… let me understand this. The SDMB administration has been trying for years to find a viable way of making this a self supporting enterprise either through subscriptions or delivering eyeballs to advertisers, and we’ve been deliberately preventing access to the SDMB content until approx 10 days ago by the largest, most powerful search engine in the world. Is that understanding correct? ---
**Author:** ![alterego](https://avatars.discourse-cdn.com/v4/letter/a/6bbea6/32.png) [@alterego](https://boards.straightdope.com/u/alterego)\ **Post date:** [March 22, 2009, 1:44am UTC](https://boards.straightdope.com/t/lets-all-watch-google-crawl-the-dope/489566/30 "2009-03-22T01:44:44Z")
> [@astro](#): > > Ummm… let me understand this. The SDMB administration has been trying for years to find a viable way of making this a self supporting enterprise either through subscriptions or delivering eyeballs to advertisers, and we’ve been deliberately preventing access to the SDMB content until approx 10 days ago by the largest, most powerful search engine in the world. > > Is that understanding correct? One of the problems with fighting ignorance for 36 years is becoming old and not realizing that new technologies can solve all your problems. ---
**Author:** ![samclem](https://avatars.discourse-cdn.com/v4/letter/s/a9a28c/32.png) [@samclem](https://boards.straightdope.com/u/samclem)\ **Post date:** [March 22, 2009, 1:56pm UTC](https://boards.straightdope.com/t/lets-all-watch-google-crawl-the-dope/489566/31 "2009-03-22T13:56:19Z")
> [@astro](#): > > Ummm… let me understand this. The SDMB administration has been trying for years to find a viable way of making this a self supporting enterprise either through subscriptions or delivering eyeballs to advertisers, and we’ve been deliberately preventing access to the SDMB content until approx 10 days ago by the largest, most powerful search engine in the world. > > Is that understanding correct? I’m not in the know, but I _believe_ that the rational for not allowing Google’s content bot to search until now was the fact that the servers we had prior to the last upgrade were so limited that the Board performance would suffer terribly. Just what I remember. ---
**Author:** ![Half\_Man\_Half\_Wit](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/half_man_half_wit/32/21766_2.png) [@Half\_Man\_Half\_Wit](https://boards.straightdope.com/u/Half_Man_Half_Wit)\ **Post date:** [March 22, 2009, 2:10pm UTC](https://boards.straightdope.com/t/lets-all-watch-google-crawl-the-dope/489566/32 "2009-03-22T14:10:56Z")
> [@alterego](#): > > I just Googled for my nickname and found this thread. Google even displays that there were 10 posts and 6 authors when it came. It looks like it found it almost as soon as I posted it. > > I just created a Google Custom Search Engine for the Dope. It only returns results from the boards. [Here it is](http://www.google.com/coop/cse?cx=001223136417691641822:x0mhldiy7ko). Thanks a lot – using that, I just figured out something completely new to me, a way to search threads in such a way as to reveal who posted how often to them: enter the url [noparse][http://boards.straightdope.com/sdmb/misc.php?do=whoposted&t=[/noparse]](http://boards.straightdope.com/sdmb/misc.php?do=whoposted&t=%5B/noparse%5D) into your address bar, and put a thread number behind the ‘t=’. Doing so for this thread brings you [here](http://boards.straightdope.com/sdmb/misc.php?do=whoposted&t=510436). Clicking again on the little number behind the username lists only the posts by that poster within that thread. I’ll probably never use this again, and likely everybody knew this except for me, but I think it’s pretty nifty. Are there any other tricks like this that have so far eluded me? ---
**Author:** ![Chronos](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chronos/32/134_2.png) [@Chronos](https://boards.straightdope.com/u/Chronos)\ **Post date:** [March 22, 2009, 2:33pm UTC](https://boards.straightdope.com/t/lets-all-watch-google-crawl-the-dope/489566/33 "2009-03-22T14:33:06Z")
> [@](#): > > Let’s not pretend Gooooooogle has not been aware of us up til now. I googed myself a year ago, and the first two hits were from here. Google will not crawl through the links on a page where it’s not welcome, but it will still be aware of the existence of things linked from pages that do welcome it. So the classic mega-threads like [The Horror of Blimps](http://boards.straightdope.com/sdmb/showthread.php?threadid=160851) and [If LotR Had Been Written By Someone Else!?](http://boards.straightdope.com/sdmb/showthread.php?threadid=138905&perpage=50&pagenumber=1), which got linked in many places outside the Dope, were already in Google’s index, though it didn’t know anything about them other than what it got from the linking pages. > [@](#): > > My very first, and only, question to Cecil (with his answer) is now number THREE on a Google search with the same topic! Amazing! Especially considering the question was asked in 1997!!! Google has always indexed the Straight Dope columns (as opposed to the board), and due to the esoteric nature of the questions Cecil covers, they usually end up near the top of searches. You probably would have gotten the same result if you had searched years ago. Which question was it, incidentally? [Previous page](https://boards.straightdope.com/t/lets-all-watch-google-crawl-the-dope/489566.md?page=1)