Statistics for Traffic Referred by Search Engines and Navigation Directories to Useit

by Jakob Nielsen on April 11, 2004

by Jakob Nielsen

The following table shows the number of visits that have been recorded in the Useit server logs as coming from search engines and directory services (so-called "portals") in a one-month period (March) in each of the years from 1998 through 2004. The true number of visits is slightly larger since some browsers don't report accurate referrer URLs. Also note that the number of users is likely to be smaller than the number of visits since people sometimes use multiple search engines or repeat a search.

Portals and search engines accounted for about half of the "unique visitors" to Useit. In comparison, the HypertextNow site gets 30-40% of its visitors from portals and search engines.

Overall search-engine-driven traffic increased by 9% during the last year. This was much slower growth than the 95% recorded from 2001 to 2002. The extreme growth in 2001 was likely due to the emergence of Google as the dominant search engine, since Google has a tendency to drive traffic to high-quality websites.

The slower growth since 2002 is realistic for the more stable Web we are seeing now that Internet use is close to saturation in English-speaking countries. Even though Useit has an international outlook, it is written in English and thus attracts the majority of its traffic from the U.S., Canada, the U.K., and Australia. Other countries still have rapid growth in Internet use, but people are unlikely to find an English-language site through search engines if they enter query terms in other languages.

Traffic to a site from a search engine depends on three factors:

  • the engine's raw traffic (so almost everybody gets a lot of visitors from Google and Yahoo)
  • the interests of people using a certain kind of engine (so Useit got uncommonly many visitors from Google in its early years since early adopters tend to be the kind of advanced users who care about my topic)
  • whether the engine emphasizes quality or quantity (so, since Useit aims at high quality content, it will get more visitors from Google, Ask Jeeves, and Mining Co. because of their emphasis on guiding users to the best sites - in contrast Useit gets relatively little traffic from weblog-dominated search engines like DayPop because Useit only posts a new article every two weeks).

The most interesting observations from the table are:

  • Google continues to dominate Internet search . It accounted for 76% of the referrals in 2004 and 72% in 2003. In a study of how journalists use the Web in early 2003, we found that half of the journalists turned to Google as their first action when asked to perform research for a story assignment. Still, Google is not the only game in town. Yahoo and MSN are important as well.
  • Looking back, Google went from number six in 1999 to a clear winner in 2001 and a virtual "takes-all" position in 2002, showing that the Web is nowhere close to being locked down : there is still plenty of opportunity for new sites . Google didn't even exist the first year I collected these statistics.
  • Still, things seem to have quieted down somewhat lately: the top-10 list was the same in 2004 as in 2003, and the only change in the rank ordering of the top-10 search engines was that Netscape and About swapped places (#8 and #9). A few new search engines appeared and bear watching even though they haven't yet cracked the top-10: myway.com, MyWebSearch, and WebSearch.com.
  • Microsoft Network is getting to be big as a source of visitors. This is probably partly due to an increased customer base, but it must also indicate that MSN is better than AOL at letting users out on the open Internet and helping them find higher-quality content than simply the service's own walled garden. AOL is much bigger than MSN and yet lets a much smaller part of its users find Useit.
  • There are only about 7 search engines worth paying attention to : Google, Yahoo, MSN, Ask Jeeves, Lycos, AltaVista, and AOL. Together they account for 97% of traffic (same as in 2003).
  • Services can stagnate as was the case for AltaVista , Lycos , and About (originally known as The Mining Company).
  • Excite virtually died in late 2001. In fact, since Excite@Home closed shortly after suffering this big drop in traffic, it's remarkable that there are any visitors from Excite.
  • Disney's Go (formerly Infoseek) went out of business in 2001; in fact, it is surprising to find any referrals from a dead search engine. More interesting, I predicted Go's troubles based on my analysis of the search referrals in March 2000.

The type of statistics shown in this table may be seen as a leading indicator for the future of portal companies : their ability to guide users to high-quality content is likely to influence their future use since people will tend to stay with services that give them good results and will tend to recommend the good services to their friends. Services that do not succeed in guiding users to good pages may suffer a drop in usage over the long term since people will change service when they discover a better one.

 

Number of visits referred by navigation services during one-month periods.

 
  March
1998
March
1999
March
2000
March
2001
March
2002
March
2003
March
2004
Growth
2003
-2004
 
Google   455 4,582 23,778 67,784 85,065 98,604 16%
Yahoo 2,482 3,698 5,402 10,895 15,676 15,718 14,735 -6%
Search.MSN   68 1,636 3,534 5,445 7,240 7,512 4%
Ask Jeeves 12 85 1,127 1,422 1,633 3,062 1,800 -41%
Lycos 211 163 2,586 2,266 3,670 1,980 1,396 -30%
 
AltaVista 771 3,033 5,015 3,188 3,157 1,310 1,210 -8%
AOLsearch 51 72 514 220 916 1,096 901 -18%
Netscape Directory   28 541 525 219 586 633 8%
About/Mining Co. 73 312 1,184 1,062 777 636 578 -9%
WebFetch/Dogpile     485 423 358 437 435 0%
 
Search.myway.com           39 533 657%
MyWebSearch             532 -
WebSearch             383 -
AllTheWeb     191 381 253 250 169 -32%
Mamma 4 32 90 97 95 88 167 90%
 
Go2Net 159 387 675 350 100 176 151 -10%
Earthlink       10 247 138 144 5%
Cometsystems         152 135 143 6%
Excite 275 736 300 1,673 329 172 140 -19%
MySearch           154 119 -25%
 
HotBot 149 118       209 119 -43%
3721.com             117 -
Search.com/Snap 12 653 536 212 256 33 116 252%
Overture/GoTo 25 85 136 634 361 211 106 -50%
iWon     57 123 86 122 90 -26
 
Teoma         16 80 59 -27%
Naver         9 143 59 -27%
LookSmart 194 388 393 236 81 18 35 94%
BBC search         22 56 30 -47%
Kvasir   18 7 11 26 12 18 52%
 
Sympatico.ca       28 132 93 16 -83%
SirSearch             16 -
DirectHit     140 238 92 230 8 -96%
DayPop         270 77    
Evreka   65 27 65 57 29    
 
Findexe       275 23 24    
WebCrawler 285 81 356 43 85 6    
Go/Infoseek 1,304 2,266 1,769 253 5 5    
EuroSeek 61 24 8 6   2    
RealNames   19 25 127 4      
Searchopolis       296        
Newssearch.Userland   9 24 5        
Infind     75 4        
Savvysearch   7 45          
Underhound   93            
Metafind 3 12            
DevSearch 144              
McKinley 12              
 
Total 6,227 12,926 27,678 52,380 102,336 119,628 130,410 9%

The next table compares the referral statistics to Useit in 2004 with general referral statistics for a large sample of the Web at StatMarket during the same period.

 

Proportion of referrals from various navigation services to Useit and on the Web in general in early 2004.

 
  to
Useit
across
the Web
 
Google 75.6% 40.9%
Yahoo 11.3% 27.4%
MSN 5.8% 19.6%
Others 7.3% 21.1%

The last table compares the referral statistics to Useit in 1999 with general referral statistics for a large sample of the Web at StatMarket during the same period.

 

Proportion of referrals from various navigation services to Useit and on the Web in general in early 1999. A * indicates missing data.

 
  to
Useit
across
the Web
 
Yahoo 28.7% 43.4%
AltaVista 23.5% 10.5%
Go/Infoseek 17.6% 4.5%
Excite 5.7% 22.3%
Snap 5.1% 2.7%
 
Google 3.5% *
LookSmart 3.0% 0.7%
Go2Net 3.0% *
Mining Co. 2.4% *
Lycos 1.3% 1.8%
HotBot 0.9% 3.0%
Underhound 0.7% *
GoTo 0.7% 2.0%
AskJeeves 0.7% *
WebCrawler 0.6% 3.6%
 
Netfind.AOL 0.6% 0.2%
Search.MSN 0.5% 3.0%
Evreka 0.5% *
Mamma 0.3% 0.5%
Netscape Directory 0.2% *
 
EuroSeek 0.2% 0.1%
RealNames 0.2% *
Kvasir 0.1% *
Metafind 0.1% *
Newssearch.UserLand 0.1% *
 
Savvysearch 0.1% *

 


Share this article: Twitter | LinkedIn | Google+ | Email