As you know, there are lots of search engines on the web (Excite, Yahoo, Alta Vista, Hotbot, Lycos, Northern Lights etc.). I hate all of them.
When I have a topic in mind, I have a list of words that I know will appear in the article I want. So, I don’t want a heirarchical engine (like Yahoo or Mining Company). I also don’t want overly clever engines like Excite.
Let me take that last one as an example of my problem. It has an input field labelled “Must contain [the phrase]”. Well, when I input the phrase, I get lots of hits from pages that most certainly do not contain that text. (Yes, I realize that web sites change their content, but this problem occurs so often that the reason must be a bit more complicated than that.)
As an experiment, last night I tried searching the web for my name (“Timothy Campbell”). So far, all the search engines I’ve tried have utterly disappointed me. I know my name is listed on many web sites, exactly as shown, but the engines would try to “guess” what I “meant”. So I’d get hits on “Timothy X. Campbell”, “Fred Campbell/Timothy Smith”, along with some other brilliant “hits” that I simply can’t fathom.
Taking what I’ve said into account, can I hear some opinions on which search engines produce the best results from specific words and phrases?
I believe you can do this with most search engines, but you may need to use the advanced search modes. By default, if you just type in Timothy Campbell, the search engine may just and those two together (or maybe even or them) giving you a bunch of matches you don’t want.
Several things to note: Often (depending on the search engine) you can quote a phrase, like this:
"Timothy Campbell"
Which will then match those words in that order, instead of the two words appearing anywhere on the page.
Also, you can (also depending on the search engine) often use your own boolean logic, with parens if you want, to specify things as accurately as you want. For instance,
“Timothy Campbell” & (“web pages” | ardvaarks)
The exact semantics are different for different search engines. But most of the major ones have a pretty decent boolean language. Often there are limitations for searching on very short or common words that almost every web page has (like “the”).
Try going to www.altavista.com, clicking over to advanced search, and entering your name surrounded by doublequotes. That should give you closer to what you want. Also note that just because a web page doesn’t display the string you searched for does not mean it doesn’t contain it. The HTML may contain that phrase somewhere even if it’s not visible to you.
I use google a lot, which does NOT have a boolean search language, but often returns good results anyway. You can still quote phrases in google, even if it lacks a boolean “or”.
Personally I feel that as long as you limit yourself to the major players, which search engine you choose makes less difference than whether you use that search engine optimally or not.
Pick one you can stand and then practice with it until you get a feeling for how it works. I’ve tried others, but when the crunch is on I still go back to AltaVista because I know how to use its advanced search features and I can keep refining my question until I get just what I’m looking for.
The only specific advice I’d have regarding which site to use is to pick one with a porno filtering option (or which filters those out ab initio). I find that when searching for terms that I would expect to occur only rarely together (“Virgin Mary” and “Star Trek”, for instance, if I happened to be interested in Picard’s views on Catholicism) I get dozens if not hundreds of porno sites. Something like AltaVista’s family filter cuts those down to a manageable number.
Thanks for the comments, folks. Yes, I know about “Advanced Search Options”. They still don’t behave the way I want.
I must say, though, that http://www.alltheweb.com appears to behave precisely the way I want a search engine to behave. Thanks, aaronp, for suggesting it!
I like Infoseek for the fact that you can do a Boolean search (Timothy+Campbell) and then refine. Unfortunately, the results aren’t as reliable as Google or Altavista.
http://www.alcanseek.com http://www.alaskasearch.com
My favorite is Alta Vista. My favorite mass earch engine is Metacrawler. My suggestion is stick to one and learn its idiosyncracies. There are newsgroups and forums just on how a webmaster can get their website onto a search engine and towards the top of the list. Search engines don’t always search the entire web but only webpages registered to them. Sometimes you’ll luck out and find a webpage that matches your criteria and is/isn’t registered to them.
This page supports the best Boolean searches. If you want the words to a speech by an orator named James Jones in which he spoke about the hidden meanings of the phrase in a song that went something like “I’ve been riding across the desert on a horse with no name” but you aren’t sure of the exact words, and you know he was addressing the Rotary Club of Portland Maine, you can set it up without ripping your hair out and without getting a hit for every damn web page that has any mention of either Portland or horse or desert or Rotary Club somewhere on it:
*“James Jones” and ((“been through the desert” or “been across the desert” or “gone across the desert”) and “horse with no name”) and “Rotary Club” and “Portland” and not “Oregon” *
E1Skeptic: Thanks for recommending www.metacrawler.com – it also does what I want, the way I want it.
To everybody else whom I haven’t thanked: thanks for the many suggestions, even if I can’t apply them all to my case (and personal preferences).
I pointedly reject any search engine that requires a URL to be manually registered. I run three web sites and I can’t tell you how many hours (nay, days) I’ve spent registering my URL’s with these guys – and you have to go back and check them, because it doesn’t always “take” the first time.
An engine like that doesn’t elicit in me any confidence. I always wonder how many zillion web pages it doesn’t know about that might have precisely what I’m looking for.
All I can say is that I’ve done several searches today using www.alltheweb.com, and it has been bang on every time. I’ve never before been happy with a search engine. Now I am. Whee!
