QA & UX

by Jakob Nielsen on February 17, 2013

Summary: Quality assurance impacts the user experience: when things don’t work, users question their understanding and develop superstitions and inefficient workarounds.


Quality assurance (QA) and user experience (UX) have a two-way relationship:

  • Most obvious, usability is a quality measure for design. To ensure usability, a good UX thus requires QA thinking.
  • Beyond the user interface itself, many other quality issues also impact the total UX.

Usability as QA

By definition, usability involves measurable quality attributes such as ease of learning, efficiency of use, and user satisfaction.

My 30 years’ experience in usability has repeatedly shown me the same lesson: QA beats QC (quality control) many times over as the approach to design quality. It’s much better to bake-in usability from the very start—before you’ve even begun to design anything—than it is to wait until there’s a near-final design and then subject it to “validation” in user testing.

Early focus on usability also vastly boosts ROI; it’s 100 times cheaper to fix a design flaw on the drawing board than after product launch.

(Remember the First Law of Usability: Your design will be tested by users — your only choice is whether to run the test yourself before launch so that you can fix the inevitable problems while it’s cheap instead of playing expensive catch-up later.)

This is exactly the same conclusion found in other quality disciplines : whether for manufacturing or airline service or hospital cleanliness, the best results are achieved by proactive QA that considers quality as part of the basic process. After-the-fact QC measurements are surely better than nothing, because they’ll concentrate the mind for next time. But they won’t produce the best quality.

One more way in which UX QA resembles other quality disciplines? Quality design doesn’t come from wishing for it. It comes from scheduling quality activities in the project plan.

How QA Impacts UX

Okay, we want quality design. To get there, we have to address the many quality issues beyond the user interface that can undermine the total UX.

I’ve discussed response times many times already. It’s pretty obvious that snappy code isn’t just a matter of programmer pride. System speed impacts all usability areas:

  • For sure, a sluggish system is annoying and immediately cuts into satisfaction. In every user test I’ve run since 1994, users have complained about slow websites and praised fast ones.
  • Efficiency of use is also clearly impacted by system speed: if each action is slower, then task performance is slower. However, the total impact is often much worse than the sum of the system-caused delays: users will hesitate to perform beneficial actions if they dread the resulting wait.
  • That slow code hurts learnability might be less obvious, but it’s true nonetheless. Human short-term memory decays rapidly, and thus users are more likely to be confused if they have to wait. Also, users become more reluctant to explore (and thus learn) new options when slowness increases the interaction cost.

Other quality problems impact the UX as well. Bugs and crashes infuriate users and impede their task completion — sometimes to the extent of demanding time-consuming workarounds. But these implementation errors also reduce learnability, because users often develop elaborate superstitions around the problem. (“If I want to do X, I can’t do Y, because that causes a crash.”) When users’ mental models go off track, their ability to use the system’s bug-free parts is correspondingly impaired.

Also, when things go wrong, users don’t necessarily know it’s the computer’s fault. In fact, it’s common for users to blame themselves.

In early January, for example, the huge brokerage firm that holds my retirement account sent me a mailer advising that the beginning of a new tax year is a good time to add money to my account. Good enough. I went to the site and attempted to transfer the appropriate amount from my bank account to my retirement account. Result? An error message saying that I was investing more money than allowed.

There are many reasons this might happen. The first two that came to mind were:

  • I entered the number wrong — say, $65,000 instead of $6,500. So, I tried again, being very careful to type in the correct number. Same error message.
  • I misunderstood the changes in the tax laws that supposedly allow people my age to save $6,500 per year as of 2013. After all, these rules are overly complex and change every year. So, I reread the lengthy information.

In fact, all that time was wasted. I had typed the number right, and I had the correct amount allowed. The system was wrong and erroneously applied the 2012 tax rules to retirement investments made in 2013. Of course, it didn’t say so. Instead, it left the poor user to blame himself and try again and again, only to get slapped down every time. A week later, I tried again, and it worked. (In this case, I kept the account, but if this had been a repeat problem — or if I had been a new customer — my money would have gone to a competitor.)

Obviously, a computer system should be updated to the new year’s rules on January 1, not January 10. It’s equally obvious that bugs are bad and that response times should be fast.

So, if it’s obvious, then why do problems run rampant? Because companies don’t take quality seriously. They’d rather invest in new features than in making old features robust. But UX would be much improved if we all acknowledged that QA is the foundation for user confidence and customer satisfaction. Once everything works, then think about new features.

I said 10 years ago that it was time to make technology work. Although we probably suffer slightly fewer bugs today than a decade ago, poor quality is still far too prevalent. So, I’ll repeat myself: now it’s really time to make tech work.


Share this article: Twitter | LinkedIn | Google+ | Email