I have long thought that the SDMB is a great repository of knowledge, and it seems as though it should be one of the places that Google can index to find answers.
Unfortunately, it seems you can’t find SDMB threads on google. Is this intentional? If so, is this just for bandwidth/performance reasons, or is there a philosophical opposition to the SDMB threads being available for search? It seems that now that there is advertising, driving more traffic to the site via google is a revenue generator for the site.
To test this, go to Google and search for pages on boards.straightdope.com, you’ll see nothing:
I’ll take a stab at this until an official person comes around:
It is intentional, for several reasons.
Just after Ed Zotti implemented the google ads, SDMB threads started showing in the google search results. It came to light that Ed had allowed both google bots to crawl the boards (one for ad content, one for search results). The board performance suffered (and has since been relieved as the search bot no longer has access). Additionally, one benefit paying members receive is the ability to search. Offering this ability to anyone lessens the value of a subscription, some said.