Why can't I Google this guy's penis?

Or more generally, how do pay sites (adult or otherwise) keep their content from coming up on Google and/or Google Image searches? Safe Seach is off so that’s not it. Google’s the main search engine I use but if you feel like answering for other engines, feel free.

Robots file. Google talks about it on their How can I remove content from Google’s index? page.

The same way they keep their content from anyone else who’s not a paying customer. I.e. various security schemes.

Google doesn’t, or can’t, search behind secure parts of sites, and all pay sites member’s areas are secure.

For a given value of “secure” anyway.

That’s key since, for some, their idea of “secure” is “hope nobody figures out what directory everything’s in”, or they could be setup so that the member’s username and password is passed in the URL itself. In either case, if a user were to ever link to something, suddenly the whole website would be exposed.

“Robots” alone is pretty good at keeping search engines out, so long as the search engines follow the largely voluntary guidelines.

Another way is to simply block the known IP addresses of search engines.

You can also ask the HTTP requester what kind of program it is, and refuse to serve pages if it identifies as a bot. Obviously, you could just write a bot to lie and say it’s, for example, IE, but the major search engines aren’t going to do that.

A classic thread title!

If there’s an honor roll of Most Memorable SDMB Titles this deserves to be on it.

Furthermore, how flexible would the guy have to be to Google his own penis?

Slight hijack, but can the various government survelliance programs check out “members only” sections of sites without the government buying a membership to goatsecxgirlsgonewild.com (for example)?

Maybe he’s saving himself for marriage? :slight_smile:

I don’t know what kind of clever trickery governments may have, but if a site is very carefully kept secure, it should keep every unauthorised person out, including hackers of immense skill.

Not that guessing a login isn’t that difficult sometimes (stupid users and their simple login details…).

Are you kidding? The government is a Charter Member.

If I could Google my own penis I’d never leave the house.

I wonder if telling the really hot girl in accounts that I’d love to Google her would be out of line. Yeah, probably. Damn Google is screwing everything up. Can’t find penises, can’t subtly harrass people. Dogs and cats living together! What is the world coming to? :mad:

Anyway, I’ll side with the mention of security being the best filter to a search for content that is limited to members. Though, the government has always had the upper hand. If the CIA can crack the security of servers being run by foreign governments, I doubt a porn site would offer much of a challenge.

Otto, are you looking for a specific person? Or just wondering why search results are somewhat limited? I know of a few search engines that you may be thinking of, but will have to mention them later if I can remember them. But no promises.

I have no idea, but dude, that’s an AWESOME site.