Ecommerce User Experience Vol. 13: Ecommerce User Research Methodology

5th Edition

Part of Series: Ecommerce User Experience

The recommendations based on any user research study are only as good as the data and information collected. The quality of this information relies on the methods used to collect the data.

This 138-page report describes the methods and specific details of our ecommerce user research:

• Diary-based longitudinal studies
• Usability testing studies
• Eyetracking research
• Expert reviews

This report explains the reasons for the methodologies selected, the tasks and instructions used for each part of the study, and the lessons learned from using these methods.

 

Topics

• Goals of each round of research and reasons for selecting the methods chosen
  • Think-aloud usability studies
  • Eyetracking
  • Diary studies
  • Expert reviews
• Recruiting users: Requirements for testers and participant demographics
• Selecting testing locations
  • Conference rooms
  • Rented research facilities
  • Remote studies
• Sites and apps tested
  • Selecting representative websites and apps to test: Considerations in selecting diverse retailers for research 
  • List of designs evaluated: Comprehensive lists of websites and apps involved in our research
• Test logistics
  • Tasks and scenarios: Full list of tasks used across all rounds of research, provided as a reference for understanding our results as well as for creating your own tasks
  • Equipment setup: How we set up each type of study
  • Conducting the studies: Details of facilitating the user sessions
  • Questionnaires and instructions: Pre- and post-task questionnaires and diary study instructions
  • Eyetracking studies: Advice and lessons for getting the best results
• Diary studies for longitudinal data
  • Recruiting users
  • Explaining study instructions
  • Keeping participants motivated in a longitudinal study
• Lessons learned and tips for running your own study

Research Method

The information in this report is based on five separate rounds of ecommerce studies for this and the previous four editions of this report. These studies included a mix of the following methodologies.

• One-on-one usability testing
• Diary-based longitudinal studies
• Eyetracking
• Expert review

Representative users tested 350+ ecommerce websites. The studies took place in the United States, the United Kingdom, Denmark, India, and China