Full day training course

Measuring User Experience

UX metrics and quantitative usability studies to measure the effectiveness and business value of your design

User-experience metrics give you objective, persuasive data on which to base your design recommendations.  They can help you answer questions such as:

  • How usable is your site?
  • Have you met your benchmarks? 
  • How do two designs compare? How does your site compare with a competitor’s?
  • What’s the impact of usability on return on investment (ROI)

In this course, you will get a set of tools and methods for planning, running, and analyzing your own quantitative user studies. 

"Very good complement to qualitative theories, practical, accessible (within reason) and extremely relevant for UX practitioner's challenges."

Guilhem Ganitou
Poland

Topics Covered

Topics Covered

  • Qualitative vs. quantitative studies
  • Formative vs. summative studies
  • Why you may need metrics 
  • Usability and ROI 
    • Calculating ROI from usability-related changes
  • Planning a quantitative study
    • Independent and dependent variables
    • Evaluator's effect
    • Measurement errors
    • Performance metrics in quantitative studies
      • Success rates
      • Time on task
    • Satisfaction scores
      • Post-task scores
      • Post-test scores
    • User experience questionnaires
      • System Usability Scale (SUS)
      • Net-Promoter Score (NPS)
    • Relation between subjective satisfaction and performance metrics
    • Avoiding bias and writing good tasks
    • Recording the dependent variables accurately
    • Finding top tasks for testing
      • Quality function deployment (QFD) method 
    • Remote versus in-person usability testing
  • Analyzing data from a quantitative study
    • Means, medians, geometric means: when to use each
    • Continuous vs. binary measures
    • Confidence intervals
    • Statistical differences between designs
      • T-tests
      • Within-subjects and between-subjects comparison
    • Comparing with benchmarks
  • Determining the number of participants: which decisions can you make based on small samples, and which require a large sample?
    • Number of participants needed for formative studies
    • Number of participants needed for summative studies

Format

Format

The basis of the course is a lecture format with some group exercises to reinforce the learned principles and guidelines.
The course also includes:

  • Findings from our own studies
  • Excel and text spreadsheets for all examples used in class
  • Opportunities to ask questions and get answers

Bring Your Laptop

Bring Your Laptop

These tools will benefit you during class, but are not mandatory:

  • a laptop
  • Microsoft Excel so that you can access the online tools and spreadsheets that we'll be using in class

Related Course

Related Course

NN/g offers two courses that focus on quantitative UX methods, but these courses address two different types of data collection:

  • Measuring UX, the course described on this page, focuses on designing and analyzing quantitative usability studies, where a sample of representative users perform assigned tasks. The course provides you with tools to understand how much you can trust results obtained from any given quantitative study. While some of those tools can be applied to analyzing analytics data, Measuring UX presents them in the context of controlled usability studies.
  • Analytics and User Experience is a different course which covers the use of data that is passively collected by recording all user actions on a live website or application.  Analyzing trends and outliers in this large body of data can identify UX opportunities and evaluate the success of UX designs.

Both of these methods can be applied to a variety of different websites and applications. Analytics data comes from a larger group of users in uncontrolled, but real-life situations. Quantitative usability studies allow you to put a number on your site’s usability and gather more in-depth and specific information about which tasks are easy or difficult to perform on your site.

Participant Comments

Participant Comments

"Very nice course. A lot of info to help you start measuring yourself or understanding incoming data. It has a bit of maths, but nothing too intimidating."

Marina Yaricheva, Clarity International

"Finally, ROI of UX explained. Congrats!"

Andrea Cauti

"Her experience and knowledge are very evident. I feel as though I have been gaining experience from the masters of the field."

Lejjy Lefour, University of Alberta

"The exercises in this class were fantastic. Learned new skills and am walking away with an action plan. What made these exercises great was the length of them and amount of detail provided."

Kortnie, UX Reseacher, CA

  • More Participant Comments
    "Really loved this session. It helped me move from a STRONG qualitative preference to a qualitative PLUS quantitative preference. Basically, I just didn't know how to do quantitative."

    Reed Jones, SubHub

    "Very practical. Really liked the aspect of drilling down to specific methods and equations."

    Carly Chiao, Auto Gravity

    "This is a really thorough introduction to conducting accurate, meaningful user experience studies that yield solid, significant empirical results — I'm delighted by the rigor of the instruction and I'm pleased the curriculum encompasses basic statistical concepts in a very approachable way, that will significantly enhance the credibility of results."

    Adam Turnbull, Rail Europe

    "Great class! Very organized with specific tools to measure user experience."

    Michaela Mora, Relevant Insights, LLC

    "Anyone that may want to or is looking into redesigning a site/system/tool, and wants to learn how to measure the success of your efforts — come to this course! Raluca is amazing."

    Lyndsey Lindsay, PLTW

    "Very, very relevant and practical info. Super well delivered by instructor."

    Ritika Mathur, Fiserv

    "This class was excellent. Review of confidence intervals brought true understanding to what this "street-trained" researcher found herself using all too often."

    Bev Inzarry, Velvet Hammer

    "I thought that some of the math and calculators would lose me, but it all tied in and became clear! Very useful information."

     

    "I have a lot of requests to take back to our market research team. I'm excited to get data from them to help with priorities of new features. The QFD matrix will be applicable in many different ways."

    Brandie Moore, American Fidelity

    "You will definitely win an argument if you make a point based on correctly collected data and insightful analysis. This course will tell you how."

    Marco Yue Yin, GuruJam, Beijing

    "Really fantastic speaker. Only complaint was a little data-heavy, but overall amazing."

    Robert Larson, StreetEasy, New York

    "Love it! Including the excel formulas and calculator links let me focus on the methodology and meaning. Immensely helpful."

    Jennifer Church, Pitney Bowes Inc.

    "I wish my college statistics professor would have explained some of these theories this clearly."

     

    "Kate is wicked smart. Some of the formulas were difficult to grasp but she was patient and explained well."

     

    "It's a hard topic, lots of math and stats, but Kate did an amazing job explaining the complicated issues. Really knowledgeable of the materials."

     

    "I had to keep paying attention to keep up with Kate. Luckily she is an easy speakers, and the way she responds to questions is nice (and thorough). Course topic isn't really vivid, but delivered well."

    Tom, Ordina, Netherlands

Instructors

Instructors

Kate Meyer

Kate Meyer is a User Experience Specialist with Nielsen Norman Group. She plans and executes user research to help guide strategy and implementation for websites and applications. She also leads UX training courses and conducts independent research for NN/g. Her research findings and recommendations are informed by her background in information theory and design, as well as her development experience. Read more about Kate.

Raluca Budiu

Raluca Budiu is a Director of Research with Nielsen Norman Group. At NN/g she consults for clients from a variety of industries and presents tutorials on mobile and tablet usability, cognitive psychology for designers, and principles of human computer interaction. She coauthored the NN/g reports on tablet usability, mobile usability, iPad usability, and the usability of children's websites, and the book "Mobile Usability". Raluca also served as a judge for GSMA Global Mobile Awards 2012-2016 and Tabby Awards 2013. She holds a Ph.D. from Carnegie Mellon University. Read more about Raluca.

Therese Fessenden

Therese Fessenden is a User Experience Specialist with Nielsen Norman Group. Prior to joining NN/g, Therese was a user experience consultant at Microsoft, designing and auditing the user experience of the intranets, Windows applications, and SharePoint sites for Fortune 500 corporations. While pursuing her graduate degree, she worked as a freelance web designer, focusing on responsive web design for small- to medium-sized businesses. Her experience with varying industries and types of clients lends to a unique perspective of how user experience research can better inform design decisions. Read more about Therese.