# With search broken, why not let search engines in temporarily?

**URL:** <https://boards.straightdope.com/t/with-search-broken-why-not-let-search-engines-in-temporarily/442243>\
**Category:** About This Message Board\
**Created:** [March 21, 2008, 6:28pm UTC](https://boards.straightdope.com/t/with-search-broken-why-not-let-search-engines-in-temporarily/442243 "2008-03-21T18:28:05Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![elmwood](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/elmwood/32/3060_2.png) [@elmwood](https://boards.straightdope.com/u/elmwood)\
**Post date:** [March 21, 2008, 6:28pm UTC](https://boards.straightdope.com/t/with-search-broken-why-not-let-search-engines-in-temporarily/442243/1 "2008-03-21T18:28:05Z")

</div>

From the robots.txt file of the SDMB:

```auto

User-agent: Mediapartners-Google*
Disallow: 

User-agent: *
Disallow: /

```

Search engines are currently blocked from the SDMB.

Since search is broken for what seems like an indefinite period of time, why not let search engines crawl the SDMB? Google can temporarily be the SDMB search engine; for example [search term] site:boards.straightdope.com. Once search is repaired, robots.txt can be restored, and the search engine results from the SDMB will eventually disappear.

---

<div class="post-metadata">

**Author:** ![Joey\_P](https://avatars.discourse-cdn.com/v4/letter/j/919ad9/32.png) [@Joey\_P](https://boards.straightdope.com/u/Joey_P)\
**Post date:** [March 21, 2008, 8:20pm UTC](https://boards.straightdope.com/t/with-search-broken-why-not-let-search-engines-in-temporarily/442243/2 "2008-03-21T20:20:29Z")

</div>

Won’t they always be cached though? (I really don’t know).

Also, once you allow the spiders (that’s what they’re called, right) on the board, how long before they actually show up?

---

<div class="post-metadata">

**Author:** ![jackelope](https://avatars.discourse-cdn.com/v4/letter/j/a6a055/32.png) [@jackelope](https://boards.straightdope.com/u/jackelope)\
**Post date:** [March 21, 2008, 8:35pm UTC](https://boards.straightdope.com/t/with-search-broken-why-not-let-search-engines-in-temporarily/442243/3 "2008-03-21T20:35:01Z")

</div>

I think it would take maybe a day or two and then Google would have the site cached. Not sure about that though; my blog used to be Googlable, then I hid it from Google, and there’s no cache of it on there now.