Full day training course offered at Usability Week San Francisco

Measuring User Experience

UX metrics and quantitative usability studies to measure the effectiveness and business value of your design

User-experience metrics give you objective, persuasive data on which to base your design recommendations.  They can help you answer questions such as:

  • How usable is your site?
  • Have you met your benchmarks? 
  • How do two designs compare? How does your site compare with a competitor’s?
  • What’s the impact of usability on return on investment (ROI)

In this course, you will get a set of tools and methods for planning, running, and analyzing your own quantitative user studies. 

Topics Covered

  • Qualitative vs. quantitative studies
  • Formative vs. summative studies
  • Why you may need metrics 
  • Usability and ROI 
    • Calculating ROI from usability-related changes
  • Planning a quantitative study
    • Independent and dependent variables
    • Evaluator's effect
    • Measurement errors
    • Performance metrics in quantitative studies
      • Success (error) rates
      • Time on task
    • Satisfaction scores
      • Post-task scores
      • Post-test scores
    • Usability questionnaires
      • System Usability Scale (SUS)
      • Net-Promoter Score (NPS)
    • Relation between subjective satisfaction and performance metrics
    • Avoiding bias and writing good tasks
    • Recording the dependent variables accurately
    • Finding top tasks
      • Quality function deployment (QFD) method 
    • Remote versus lab testing
  • Analyzing data from a quantitative study
    • Means, medians, geometric means: when to use each
    • Continuous vs. binary measures
    • Measurement errors
    • Confidence intervals
    • Statistical differences between designs
      • T-tests
      • Within-subjects and between-subjects comparison
    • Comparing with benchmarks
  • Determining the number of participants: which decisions can you make based on small samples, and which require a big N?
    • Number of participants for formative studies
    • Number of participants for summative studies


The basis of the course is a lecture format with some group exercises to reinforce the learned principles and guidelines.
The course also includes:

  • Findings from our own studies
  • Screenshots and case studies
  • Excel and text spreadsheets for all examples used in class
  • Opportunities to ask questions and get answers

Bring Your Laptop

These tools will benefit you during class, but are not mandatory:

  • a laptop
  • Microsoft Excel so that you can access the online tools and spreadsheets that we'll be using in class


Raluca Budiu

Raluca Budiu is a Senior Researcher with Nielsen Norman Group. At NN/g she consults for clients from a variety of industries and presents tutorials on mobile and tablet usability, cognitive psychology for designers, and principles of human computer interaction. She coauthored the NN/g reports on tablet usability, mobile usability, iPad usability, and the usability of children's websites, and the book "Mobile Usability". Raluca also served as a judge for GSMA Global Mobile Awards 2012-2014 and Tabby Awards 2013. She holds a Ph.D. from Carnegie Mellon University. Read more about Raluca