Search tools

1. Google & Yahoo
2. Other search engines
3. Metacrawlers
4. About keywords
5. 2006 issues with Yahoo
6. Sites about search engines

Search engines used to be specialized by function.

Keyword search, also called "crawler-based," is a brute-force approach based on an automatically generated database. Fast Internet servers progressively scan and index the entire Web, maintaining a huge index database, which you can then run keyword searches against. Use keyword search when you know a particular word, words, or phrase that will tag the kind of content you're looking for, or when there don't seem to be any categories relevant to your subject matter in catalog subject trees.

A subject catalog is a human-maintained collection of categories which contain links to sites selected as being high-quality, with the categories arranged in a hierarchical subject-based tree structure. Subject catalogs are useful when you don't have good keywords, when you're looking for general information on a particular subject, and when you're looking for particularly credible or complete resources on that subject.

Top search engines now typically offer both kinds of functions. In 2nd quarter 2005 it was reported Google and Yahoo! together accounted for two-thirds of all Web searches; as of 3rd quarter 2007 reportedly Google alone handled 60% of searches.

For some more-specialized search resources, see my Tools for writers & researchers page.

Google & Yahoo

First year of service
in parentheses

Google (1998) http://www.google.com/

The big dog of search: fast and accurate. Hits on Google are ranked higher if more sites link to them, especially if the other sites are high-ranking themselves. Image search, other specialized search tools, Google Maps, Froogle shopping, experimental new tools at Google Labs, and other services including Gmail. Click more on the main page for even more Google stuff.

Google Directory http://www.google.com/dirhp

Google's subject-catalog content

Google Groups (2001) http://groups.google.com/

Google took over the Usenet newsgroups search site that used to be known as DejaNews

Yahoo! (1994) http://www.yahoo.com/

One of the early Web search tools, still very popular. Originally primarily a subject catalog, now mainly crawler-based. Yahoo! Mail Web-mail, image searches, yellow pages, shopping, many other services.

Yahoo! fast-loading version http://search.yahoo.com/

A minimalist version of the Yahoo! search interface; great for dialup. (It's amusing how much it looks like Google.)

Yahoo! Directory http://dir.yahoo.com/

You can still search just the Yahoo! subject catalog by starting here; this is how Yahoo! used to work.

top

Other search engines

top

Metacrawlers

Metacrawlers can be very useful; they send your search text to multiple search engines, and then combine the results in some innovative, useful, or interesting manner.

top

About keywords

A keyword is just a particular word that you suspect will appear on pages that meet your needs, and fail to appear on pages that don't. Canny and subtle keyword selection is very important in getting results out of this type of search engine; you'll get better at it with experience.

Actually, a keyword search string can be a single word, a list of words, an exact phrase, or more complicated strings. Each system is a little different, but usually you can do things like specifying words that must not appear on selected pages, specifying that two words must appear "near" each other, exact capitalization match or not, and pulling a list of Web pages that link to a particular URL (yours, for instance).

There are some tricks that seem to work the same on many search pages:

You can do much more complex and advanced searches. For each search system, there will be help pages; often there will be one called something like "advanced search help" that will guide you in the use of the site's most sophisticated features. Learning some of those, at least for the keyword search sites you use most often, should be worth your while.

top

2006 issues with Yahoo

Sometime in second quarter 2006, I think, Yahoo completely redesigned www.yahoo.com, their default Web search and wannabe portal page. Historically Yahoo had been known for decent speed optimization. Unfortunately their new design is not as fast, and is also more likely to force horizontal scrolling if your browser isn't maximized. Those are both bad things. There was a link at the top that set a cookie and switched you back to the old design, but that option seems to be gone now. Of course, if all else fails, you can always switch to using the very Google-ish looking and fast search.yahoo.com page.

In early May 2006 there was an egregiously bad Yahoo DSL ad banner that was all over Yahoo Mail. It featured a guy wearing nose glasses, was done mostly in shades of blue, and was supposed to be marketing Yahoo's DSL service to people still on dialup. Unfortunately if you were actually on a dialup/modem connection, this ad would tend to tie up your browser until it sloooowly finished its animation, thereby annoying the heck out of the very people at whom it was supposedly aimed. Turning off image loading in the browser was no help. I did have some luck interrupting the animation with a second click on a hyperlink. I complained to support online and got absolutely no response. By the last week of May this ad seemed to be pretty much gone.

top

Sites about search engines

Search Engine Watch http://searchenginewatch.com/

An excellent guide to dozens of general-purpose and specialized Web search tools

Traffick http://www.traffick.com/

"Search Engine Enlightenment." It says so, right on the home page.

Search Engine Showdown http://searchengineshowdown.com/

URouLette http://www.uroulette.com/

A random page with each "spin"

Autopilot http://www.mit.edu/people/mkgray/autopilot.html

A new random page every twelve seconds. This might be a fun thing to run on your PC when you throw a party, if you have "big pipes" such as DSL, cable modem, satellite, or faster; may be broken.

Dogpile SearchSpy http://www.dogpile.com/info.dogpl/searchspy/
Excite SearchSpy http://www.infospace.com/info.xcite/searchspy

Watch other people's search strings, scrolling display (both)

MetaCrawler MetaSpy http://www.metaspy.com/info.metac.spy/metaspy/

Another view of other people's search strings; refreshes every 15 seconds

top