a/b testing Articles & Videos

  • Validate Your Visual Design: 6 Methods

    Test your site’s visual design using these 6 methods: 5-second test, first-click test, preference testing, visual design questions after usability testing, eyetracking, and A/B testing.

  • A/B Testing 101

    A/B tests incrementally improve the user experience of a product while effectively reaching business goals.

  • A/B Testing Roadmap

    6 steps to consider when planning a systematic set of A/B tests to optimize a design for the long haul.

  • Handling Insignificance in UX Data

    After collecting KPI numbers for two versions of a design, the difference between the two metrics is not statistically significant. Now which version should you launch?

  • Is A/B Testing Faster than Usability Testing at Getting Results?

    If A/B testing can quickly show which design is best, why should a UX team bother doing usability studies and other user research?

  • Don't A/B Test Yourself Off a Cliff

    A/B testing often focuses on incremental improvements to isolated parts of the user experience, leading to the risk of cumulatively poor experience that's worse than the sum of its parts.

  • A/B Testing 101

    What is A/B testing, and why should you consider this method for measuring the business value of design changes?

  • A/B Testing vs. Multivariate Testing for Design Optimization

    Like A/B testing, multivariate testing is a design optimization method that involves experimenting with live traffic to find the best impact on conversions.

  • How to Test Visual Design

    Visual design details like fonts and colors can have subtle but important effects on the overall user experience. Use research methods that are sensitive to these effects to test your visual design.

  • When to Use Which UX Research Method

    Don’t just do usability tests. Do what you need. Consider five criteria to help you determine the most effective UX research method to use for your situation, to meet research goals, and achieve desired outcomes.

  • Pitfalls of Conversion-Rate-Only Concern

    Numbers don't paint the full UX picture, so in the quest for conversion rate optimization, don’t lose sight of the fact that we’re designing for humans.

  • Multivariate vs. A/B Testing: Incremental vs. Radical Changes

    Radical redesigns are best tested using an A/B experiment, while multivariate tests indicate how various UI elements interact with each other and support incremental improvements to a design.

  • Usability Testing of Icons

    To ensure that people understand the meaning and purpose of icons, conduct multiple types of tests at various stages of the product-development cycle.

  • Define Stronger A/B Test Variations Through UX Research

    Complement A/B split tests with user research to identify true causes and develop well informed design variations.

  • A/B Testing, Usability Engineering, Radical Innovation: What Pays Best?

    3 approaches to better design: each has its uses, but the costs, benefits, and risks differ dramatically.

  • Quantitative Studies: How Many Users to Test?

    When collecting usability metrics, testing with 20 users typically offers a reasonably tight confidence interval.

  • Putting A/B Testing in Its Place

    Measuring the live impact of design changes on key business metrics is valuable, but often creates a focus on short-term improvements. This near-term view neglects bigger issues that only qualitative studies can find.

  • First Rule of Usability? Don't Listen to Users

    For good UX, pay attention to what users do, not what they say. Self-reported claims are unreliable, as are user speculations about future behavior.