Apr 7-12

May 19-24

Jun 16-21

Jul 21-26

Going Beyond What Users Are Doing to Why

Interview with Colyn Brown

E-commerce provides reams of data on what the customers are doing. But why are they doing it? Colyn Brown, senior user interface engineer at MotorSport Inc., in Portland, Oregon, is hoping to learn the answer as it relates to his firm’s e-commerce and catalog sales by attending the Nielsen Norman Group Usability Week conference in San Francisco.

“I’m here trying to understand the difference between qualitative and quantitative testing,” Brown explained. “We’ve been using the quantitative approach, since with e-commerce, you have a lot of numbers that you can crunch. We can change the design of a page and watch the sales go down—and then you know it didn’t work. Or you can do A-B testing, where you test two or more versions on random people and see how their reactions differ. But even with A-B testing, if the sales do go up or down you still can’t figure out why.

“With qualitative testing you can figure out why the users do what they do—if you set the test up correctly, and that is what I am learning to do here,” Brown explained.

Successful testing can be as much art as science, and Brown said he’s learned that the little things are important. For example, he’s learned that his own body language and demeanor can impact a test’s outcome since a test facilitator’s behavior colors the reactions of the subjects.

“If the test subjects are floundering, your instinct is to help them, but you can’t do that,” he noted. Meanwhile, he said, he’s learned that he must find ways to remove distractions and noise to minimize the number of test variables. “You want to know that there are no other factors involved. It’s hard to simulate real users,” he noted.

However it’s carried out, Brown said his company is likely to do more testing in the future. “We are growing very fast, and a new CIO came in with experience in e-commerce best practices, and has started moving in a new direction that lets us grow and expand smoothly.”

Overall, “if there are complaints about our Web site, we respond to them right away—that’s easy. But we have to stay ahead, not only of the competition, but of the customers’ expectations” which he said constantly transform.

“At this point I am still digesting the practical implications of what I have learned,” he observed. “Being here gives me an increased level of confidence in my work. I have a better sense of what best practices are, even if everything does not immediately relate directly to my job. Of course, I am familiar with Jakob Nielsen as a guru of usability. He has a solid, reasonable approach that’s hard to argue with—no fluff.”