Klixxx Home
> Webmaster Article Archives Home > Traffic
Who's Who in Search Engines: Behind the Scenes Maneuvering
by Dyanna S. Culp, Temos
Most Web surfers know the search engines Yahoo!, AOL, maybe even Looksmart, or my personal favorite Google. But, what about Excite- with four times the traffic of Looksmart, and almost half that of Yahoo!? Or Dmoz and All the Web, behind the scenes suppliers of site information, to heavyweights such as AOL and Lycos? And, no serious Marketing guru ignores Overture these days. SEO (Search engine Optimization) with perfect meta-tags and killer content is no longer enough.
Who's who in the search engines, and how does a Webmaster go about the torturous process of using these engines/directories? Keep in mind that the environment is quite fluid, with collaborations between major players changing frequently. Almost all engines now require payment just to be considered for inclusion in their database. There is no guarantee of acceptance, or ranking, for the required annual fees.
PPC (Pay Per Click) positioning is becoming the dominant ranking method for all of the big engines, and even many of the smaller ones. Sponsored PPC listings appear before all non-paid search results. Remember, we must now romance not only the big boys, but also the lesser known "behind the scenes" information suppliers. Incestuous relationships run rampant among the directories/search engines.
According to a Media Metrix 2001 study, the top search engines had over 345,000,000 unique visitors each month. Do you see Dmoz, Inktomi, All the Web, or Overture, included in this elite group?
Behind the Scenes Players
Ah Ha (www.ah-ha.com) - is a Family engine with no pornography allowed.
All the Web/alias Fast (www.alltheweb.com) - acts as the main source for Lycos Web page results. Determining their name is rather confusing; they seem to go back and forth between Fast and All the Web, and are currently using both.
Dmoz, also known as Open Directory Project (ODP) (www.dmoz.org) - feeds Yahoo!, AOL, Netscape Search, Alta Vista, Lycos, Hotbot, and a multitude of smaller sites. ODP states that over 211 sites use their data. They strive to be comprehensive, unbiased, and do not accept any form of paid submissions. A real human being (a volunteer editor exists for each category) reviews all submitted sites, so select your submission category and wording carefully. Dmoz frowns heavily on the submission of multiple Web page URLs unless the individual page's content is truly unique.
Epilot www.epilot.com) - feeds many of the smaller engines, including Dogpile.
Find What (www.findwhat.com) - feeds many of the smaller engines, including Excite, Metacrawler, and Dogpile.
PPC payment required at http://findwhat.com/static/ab_promote.html
Google (www.google.com) - is both a serious player, and more recently a behind the scenes data supplier. Google recently switched its paid ad word boxes to a broader pay-per-click program similar to the Overture program. Overture is suing Google. On May 1st Google announced its contract as provider of sponsored ads at AOL and AOL properties, including AOL Search, CompuServe, and Netscape.
Set up a pay-per-click account, but be prepared for possible frustrations, as Google branches out into this new venue. Don't try to use you're American Express. The billing information page "says" they take American Express, but after trying three different cards I still received card declined messages. Overture, AOL, and Yahoo! all have Marketing phone numbers if you run into problems- Google does not. For four days I received e-mail responses that asked me moronic questions, such as: Did I put my expiration date in correctly? Eventually someone at Google woke up and informed me that American Express only worked on their form if you called it in, and they didn't take calls. So why American Express is listed as an option is the million-dollar-question. Your American Express card will work just fine on any of the other pay-per-click engines.
Inktomi (www.inktomi.com) - supplies results to such major portals as MSN, Overture, Iwon, Hotbot, Looksmart, and dozens of customers in Europe and the Far East.
Inktomi submit is a reasonable $39 annual review fee, but there are no inclusion guarantees.
Kanoodle (www.kanoodle.com) - feeds Cnet, Dogpile, and they "claim" 9,900 other search-enabled sites.
Overture (www.overture.com) - formerly known as Go To, is the big shark in the dark murky Internet waters. They provide those top sponsored listings on MSN, Yahoo!, Ask Jeeves, Hotbot, Lycos, Excite, and others.
In late April they picked up an unprecedented three-year agreement with Yahoo!, but lost AOL to Google on May 1st. This prompted lawsuits filed against Google for alleged patent infringement on their pay-per-click territory.
PPC payment is required for submissions, and they are the most expensive pay-per-click program online.
Sprinks (sprinks.about.com) - feeds About.com, Ask.com/Ask Jeeves.
Sprinks is involved in major upgrade changes, which as of May were still causing serious problems for many Sprinks users. Bid maintenance and customer support issues are still troublesome according to online webmaster chat rooms.
PPC payment is required for submissions.
The Lightweights with Loyal Users
All the Sites www.allthesites.com - another tentacle of Fast or All the Web.
Claymont www.claymont.com - Claymont is partnered with a terrific service "Internet Seer" which monitors your Web site 24 hours a day, and notifies you by e-mail of any server downtime.
Dogpile www.dogpile.com provides grouped results from roughly 40 different engines - You can edit your listing at http://dpcatalog.dogpile.com/texis/websearch/modify.htm, but to be in the database at all, you must submit to the engines it utilizes, such as Ask Jeeves and Overture.
Exact Seek www.exactseek.com - Exact Seek will not add a site to its database if the site does not contain Title and Meta Description tags. Free submit at http://www.exactseek.com/add.html
Iwon www.iwon.com Fed by Inktomi and Dmoz - Growing fast due to extensive marketing campaigns and cash contests for users.
Metacrawler www.metacrawler.com a multi-index search engine that appears to be dropping in popularity.
Webcrawler www.webcrawler.com owned by Excite.
Excite technical support quote: "The WebCrawler index is much smaller than the Excite index and cannot index as many sites. I have been informed that the Webcrawler index is at capacity, and we are not taking any more site submissions."
The Medium Weight Players
About.com (www.about.com) - Stay away from their Bid program "Sprinks" until reviews improve.
Alta Vista (www.altavista.com) - feeds Looksmart; it uses Looksmart's directory and Dmoz.
Slow free submit or faster paid option at www.altavista.com/addurl.
Ask.com better known as Ask Jeeves (www.ask.com) -
In the process of building their own index at Teoma. Alta Vista, Excite, WebCrawler, and Overture influence their results.
Paid premier listings information at static.wc.ask.com/docs/advertise/premier.html.
Excite (www.excite.com) - Purchased by Info Space after a bankruptcy, Excite survives by searching the Overture database for its results. Powers Webcrawler.
Google (www.Google.com) - influenced heavily by Dmoz. Feeds search data to AOL and Netscape.
Google is well known among serious surfers for highly relevant search results, and is always my first stop in a Web search. Google claims to have the largest database on the Internet with over 1 billon-indexed URLs. They crawl deeper than most engines into a submitted site, and therefore index more of the Web site's pages. Google places a heavy emphasis on what is termed, "Page Rank," which means the number, and type, of sites that link to yours. They now completely ignore Meta tag descriptions and keywords. Don't bother submitting until you have at least two similar sites (that already exist in their database) linking to you, and your Web site text is full of keywords. Digging for the spot to submit a site? It's at www.google.com/addurl.html.
Looksmart (www.looksmart.com) - powered by Inktomi, and feeds AltaVista, Excite, Juno, Iwon, MSN and CNN.com.
At Looksmart, a human being reviews your site and determines if it is worthy of inclusion. Free submissions are limited to five URLs, after decrypting their submission puzzle page. The engine is falling behind, becoming out of date, and has many users and site owners very irritated. Looksmart has been known to take three months or longer to index a no-charge submitted site.
The faster route is paid submissions. Looksmart recently started charging $299 per year for inclusion. Then they decided to anger a great many people by changing to a recurring fee structure, based on the number of clicks a site receives, rather than the flat yearly fee. To keep the exact same listing (bought at $299), some people are being forced to pay anywhere from $150 per month, to as much as several thousand dollars per month. Rumors are that Looksmart is refusing refunds to the people who had signed up for the $299 a year plan. The only real reason to worry about Looksmart is their partnership with MSN, but Overture covers you there, in sponsored listings. If you still want Looksmart, submit at listings.looksmart.com (PPC payment required).
Lycos/ Hotbot www.lycos.com and www.hotbot.lycos.com are powered by Inktomi, Dmoz, and All the Web.
www.lycos.it (Italy) www.lycos.co.uk (UK)
Netscape Search (search.netscape.com) powered by Dmoz, About.com, and most recently by Google.
The Heavy Weights
AOL, MSN, and Yahoo! garner a huge share of the search traffic, because of their captive ISP audiences. The top ranker varies from month to month, as they dance about the 67 million mark. But as surfers become more savvy, we are seeing Lycos, Excite, Google, and others taking a bite out of the big boys' traffic numbers.
AOL (www.aol.com) - Their major data suppliers have been Dmoz and Inktomi for many years. On May 1st Google replaced Inktomi and Overture's sponsored PPC listings. If you want in AOL, you have to get in Dmoz (but its free!), then take a Valium and deal with the new Google Ad Select Program.
MSN (www.msn.com) - powered by Looksmart and Inktomi. Sponsored PPC listings from Overture.
MSN has no free submissions. Paid inclusion is accomplished through their partners, Looksmart or Microsoft Small Business Solutions at www.bcentral.com/products/ or
big money MSN advertising ($10,000 and up every three months), as featured sites that appear above Overture results. Very few have this kind of money. Use Overture for top exposure, and forget the headache of Looksmart.
Yahoo! (www.yahoo.com) - is fed data by Dmoz and Google, but the sponsored PPC listings come from Overture, who just signed a three-year contract with Yahoo!
Human beings will review your submitted site. Submission directions exist at docs.Yahoo.com/info/suggest/ where you have a free and a paid option. Commercial sites must use the paid submission. Not too many years ago, everyone could submit to Yahoo! for free, but commercial sites now pay $299 per year for inclusion. If you were lucky enough to get in before the payment requirements, then you're "grand-fathered" in, for free, for eternity.
Having trouble-getting Yahoo! to accept your submissions? They require all commercial Web sites to provide a business address somewhere on the site itself. It doesn't matter if it's on the Home page or somewhere else, but you've got to place a street address or PO Box, or city and state, somewhere that a search spider can reach. Pick the category that includes the best keywords in its name to match up with your site name/description.
The title of your Web site on Yahoo! is usually your company name. You'll see it as the hyperlink portion of your Yahoo! listing. When you submit to Yahoo!, your company name needs to match the company name used when you registered the domain. Include the company name in the submitted Web site title, and on your contact page. If you're inconsistent, Yahoo! rejects you or changes your listing title to whatever they decide better represents your company name. Yahoo! also rejects submissions of URLs that exceed 54 characters.
Most of the search engines now require payment for inclusion, and pay-per-click programs have the top spots on every engine of any importance. All of this is rather depressing to the poor webmaster trying to get some exposure for a hot new Web site. BUT Web users are not idiots (at least most of them), and most recognize that sponsored listings are advertisements. And, how often do you find the search result you want in the top ten listings? It rarely, if ever happens. So, optimizing your site for search engine success still matters, just don't get your heart set on the top spots unless you have some very deep pockets.
|
 Printer
Friendly Version
 Submit
Questions or Comments to Klixxx
Need More Information?
Search the Webmaster Articles at Klixxx
Submit
an Article | Link to Klixxx
| Earn With Klixxx Cash
Become
a Sponsor | Advertising Information
Klixxx Home
> Webmaster Article Archives Home >
|