Improving Usability Guideline Compliance

by Jakob Nielsen on June 24, 2002

Summary: Over the last 1.5 years, the average compliance with established usability guidelines increased by 4%. If we can sustain this level of improvement, we'll reach the ideal of 90% guideline compliance in 2017.


Web usability is finally improving. Slowly, yes. But surely.

The data behind this statement? In late 2000, before we published our 2001 guidelines for e-commerce usability, we analyzed twenty major e-commerce sites. On average, the sites followed 45% of the e-commerce usability guidelines.

In mid-2002, we repeated the study with 15 different e-commerce sites (we did not study the same sites as in 2000 , because the field has changed a lot, to put it mildly). Now, the average e-commerce site complies with 49% of established usability guidelines.

Thus, over 1.5 years, usability guideline compliance has increased by four percentage points.

Although certainly not improvement at the rate of Moore's Law, if this usability growth trend continues for another fifteen years , we'll reach full usability guideline compliance.

Truth be told, fixing the Internet in 15 years is as much as we can hope for. Yes, I'd like more rapid improvement. It's annoying to spend so much time documenting 207 principles that are known to make online shopping easier, only to discover several months later that most of these best practices are still not in place on most sites. Still, let's be realistic: New technologies take time to mature, and the fact that sites do implement ever more of the recommendations is cause for celebration.

Now for the fine print: Assuming linear growth, adding 4% every 1.5 years means that we'll add 40% over 15 years. Thus, by 2017, websites will follow 90% of the usability guidelines. A 90% compliance rate is the closest we can get to perfection, since there will always be special circumstances that make it acceptable for any individual site to deviate from a few of the established usability guidelines.

In our 2002 survey, the highest scoring e-commerce site was L.L. Bean, which followed 66% of the guidelines. Thus, even among the best sites, there is room for improvement. The fact that a traditional mail order company scored best in the new survey confirms data from an earlier study that showed that the online operations of catalog companies are the most profitable e-commerce sites.

Key Improvements: Search and Beyond

I am particularly pleased to see great improvements in e-commerce sites’ search usability, which increased from 35% in late 2000 to 48% now. Search is a key element of Web usability, and yet many a website's search function is still so bad that users are misled more often than they are helped.

Based on our 2002 study results, the three key areas of usability improvement since 2000 are:

  • Presenting product options in product pages. More sites are now putting all options on one page, using plain color names, showing an image for each color, making customers select options before adding to cart, showing chosen options in cart, and so on. Guideline compliance in this area increased from 40% to 63% .
  • Simple search. More sites are now offering one input box with one button, linking to advanced search instead of featuring it, explaining the search scope, defaulting the scope to “All,” and so on. In this area, guideline compliance increased from 30% to 53% .
  • Checkout forms and registration. More sites are now asking only for required data and explaining its uses, using clear labels and reasonable sizes for input fields, differentiating shipping and billing addresses, saving and auto filling previously supplied info, activating the enter and tab keys, providing clear explanation after errors, and so on. Guideline compliance in this area increased from 49% to 66% .

In less happy news, U.S. sites continue to treat foreign customers poorly : The score on guidelines for serving international users was only 28% (up a mere 6% since 2000).

Also, while websites are doing better at presenting simple search features (53% compliance), other search usability areas are still lacking . This explains the somewhat lower overall score (48%) for the full set of search usability guidelines. For example, for the search engine itself , tolerance of variable user input is still only 24% . But, even in this backwater of poor usability, relative improvements have been great: Search engine software scored only 8% in 2000.

International Websites Lag Behind

In addition to the American sites, we analyzed a smaller sampling of six overseas e-commerce sites. Their score was substantially lower; they followed only 40% of the guidelines (compared to a 49% score for the U.S. sites).

Given our estimated rate of usability score improvement, overseas sites are three years behind U.S. sites on usability. Obviously, this is only true on average: The U.S. has many atrocious sites, while other countries have a few great sites that equal the best the U.S. has to offer.

Why do overseas sites have poorer usability than U.S. sites? One explanation could be greater U.S. maturity in terms of executive management's emphasis on usability. Not only does the U.S. have a stronger and longer usability tradition, but most usability guidelines are published in English.

That said, I've presented many seminars and lectures in Europe, Asia, and Australia over the last three years, and have witnessed a hugely growing appreciation of usability in these regions. For example, our conference in Sydney last week had a 40% bigger audience than we had there last year. Thus, any U.S. head start in terms of talented usability professionals and executive buy-in will soon vanish.

A second explanation for higher U.S. usability scores could be the relative lack of resources for overseas sites. Certain usability areas, such as good search, require significant investments that might be easier to come by in bigger countries, where websites have a larger customer base. On the other hand, most aspects of usability are cheap to implement: They mainly require that websites simply avoid wasting money on fancy design, and focus instead on simplicity and user needs.


Share this article: Twitter | LinkedIn | Google+ | Email