# Is there a way to search a site that has not been spidered by a search engine? **URL:** \ **Category:** Factual Questions\ **Created:** [September 21, 2005, 6:07pm UTC](https://boards.straightdope.com/t/is-there-a-way-to-search-a-site-that-has-not-been-spidered-by-a-search-engine/322748 "2005-09-21T18:07:13Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Cagey\_Drifter](https://avatars.discourse-cdn.com/v4/letter/c/b2d939/32.png) [@Cagey\_Drifter](https://boards.straightdope.com/u/Cagey_Drifter)\ **Post date:** [September 21, 2005, 6:07pm UTC](https://boards.straightdope.com/t/is-there-a-way-to-search-a-site-that-has-not-been-spidered-by-a-search-engine/322748/1 "2005-09-21T18:07:13Z")
Is there a way to perform a search for keywords in a site that has not already been searched and cataloged by a search engine? ---
**Author:** ![Erinaceus\_europaeus](https://avatars.discourse-cdn.com/v4/letter/e/e47774/32.png) [@Erinaceus\_europaeus](https://boards.straightdope.com/u/Erinaceus_europaeus)\ **Post date:** [September 21, 2005, 6:25pm UTC](https://boards.straightdope.com/t/is-there-a-way-to-search-a-site-that-has-not-been-spidered-by-a-search-engine/322748/2 "2005-09-21T18:25:24Z")
> [@Cagey Drifter](#): > > Is there a way to perform a search for keywords in a site that has not already been searched and cataloged by a search engine? Unless the site provides its own searching mechanism, you can’t. Search sites like google download and index a site before you can search on them mainly for performance and bandwith reasons - you really wouldn’t want to wait for google to download and search all the available pages on the internet before giving you a result. And even if you’re “just” searching one site, it can still take a lot of time and bandwith to download and search all the available pages. On a related note, if a page isn’t reachable (linked to) from some other page in the list of avaiblable pages in the index, it won’t be found at all. If you have that problem, registering your page with the search engine will help. ---
**Author:** ![Revtim](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/revtim/32/1042_2.png) [@Revtim](https://boards.straightdope.com/u/Revtim)\ **Post date:** [September 21, 2005, 6:36pm UTC](https://boards.straightdope.com/t/is-there-a-way-to-search-a-site-that-has-not-been-spidered-by-a-search-engine/322748/3 "2005-09-21T18:36:35Z")
Perhaps you can run a spider yourself and sic it on the site. ---
**Author:** ![MikeS](https://avatars.discourse-cdn.com/v4/letter/m/919ad9/32.png) [@MikeS](https://boards.straightdope.com/u/MikeS)\ **Post date:** [September 21, 2005, 7:46pm UTC](https://boards.straightdope.com/t/is-there-a-way-to-search-a-site-that-has-not-been-spidered-by-a-search-engine/322748/4 "2005-09-21T19:46:32Z")
If it’s a single page, something you can load into your browser all at once, then you can use the “Find in Page” feature that most web browsers have. It’s Ctrl-F or Cmd-F on every browser I’ve ever used. But I suspect this isn’t what you’re asking. ---
**Author:** ![iamthewalrus\_3](https://avatars.discourse-cdn.com/v4/letter/i/258eb7/32.png) [@iamthewalrus\_3](https://boards.straightdope.com/u/iamthewalrus_3)\ **Post date:** [September 21, 2005, 8:35pm UTC](https://boards.straightdope.com/t/is-there-a-way-to-search-a-site-that-has-not-been-spidered-by-a-search-engine/322748/5 "2005-09-21T20:35:13Z")
If it’s more than one page, you could use [wget](http://www.gnu.org/software/wget/wget.html) on the site recursively to download linked pages and then do a text search on the downloaded pages. It’d take a while, though.