Let's all watch Google crawl the Dope

Did you hear that Jerry is letting Google crawl the boards? It’s true, have a look at robots.txt.

I’ve been watching and Google is catching on to this. The query is:

http://www.google.com/search?q=site:boards.straightdope.com

[ul]
[li] Mar 13 - 3,500 pages[/li][li] Mar 14 - 4,700 pages[/li][li] Mar 15 - 8,100 pages[/li][/ul]

Does anyone else think that this place is about to start getting a lot more traffic? A lot more memberships, a lot more features, and a lot more ignorant? Or is our backbone of ignorance fighting skill enough to challenge the unwashed masses of the world, which we have never before confronted? Or will Google’s access be shut off when the servers buckle under the load?

I really think we’re up to it.
“Bring it on” to quote someone. :slight_smile:

So we weren’t scanned by Google before? And now we’re getting more traffic because, as Google “assimilates” us (sorry) we turn up on more search results so more folks find us??

How do you interpet this ? I got :

Do those “disallowed” mean it’s been shut off ? Or is it the blankness after ‘Disallow’ that means Google can scan the Dope ? I’m clueless about this . . .

Holy shit! Scary crap.

The invasion begins…

It’s saying that unless you’re Google to go away. If you are Google, then you aren’t disallowed from anywhere.

Thanks.

That’s the idea.

Well fight my ignorance!

The best part is that we can use Google ourselves instead of the god-forsaken search board engine, once it’s all indexed.

Are you sure about that? I just checked, and the fourth result is “the horror of blimps”.

Let’s not pretend Gooooooogle has not been aware of us up til now. I googed myself a year ago, and the first two hits were from here.

Jeez, I used to be concerned about letting people here know my real name.

Now, I need to worry about the people I know in the RW knowing my username here.

If they Google “Boyo Jim”, not only can they find my batshit crazy posts, but even pictures of me. :eek:

Ah, WTF… :wink:

Google doesn’t index all the sites it’s aware of. It does obey robots.txt

Isn’t there going to be an Invasion of Zombie Threads? People will google something and land up in some old thread about pan frying gophers on a conveyor belt in the Marianas Trench, sign up and start posting to the old thread. I think a mass lock-off is in order

You left out…“while traveling by blimp”…

My comment generally:

[“Newman” voice (from “Seinfeld”)]
JERRY!
[/"Newman v.]

Just googled for my recent MPSIMS OP on the “One word–Plastics” dialog from The Graduate, and there it is. I don’t know whether to e happy or sorry.

I just Googled for my nickname and found this thread. Google even displays that there were 10 posts and 6 authors when it came. It looks like it found it almost as soon as I posted it.

I just created a Google Custom Search Engine for the Dope. It only returns results from the boards. Here it is.

My very first, and only, question to Cecil (with his answer) is now number THREE on a Google search with the same topic! Amazing! Especially considering the question was asked in 1997!!!
Plus, I am kind of amazed that DMark (the German currency from after WWII until the conversion to Euro) turns up my name on Google on page 5!

I’m tellin’ ya’ - there’s gonna’ be a flood.