|
|
The most significant finding from our usability research on email newsletters is that users have emotional reactions to them. This is in strong contrast to research on website usability, where users are usually much more oriented toward functionality. Even a website that users visit daily seems to feel like a tool: users want to get in and get out as quickly as possible rather than "connect" with the site.
Users tend to glance at websites when they need to accomplish something or to find the answer to a specific question. In contrast, newsletters feel personal because they arrive in users' inboxes, and users have an ongoing relationship with them. Newsletters also have a social aspect, as users often forward them to colleagues and friends. The positive aspect of this emotional relationship is that newsletters can create much more of a bond between users and a company than a website can. The negative aspect is that newsletter usability problems have a much stronger impact on the customer relationship than website usability problems. For example, in one of our studies, a user received an error message that read "Email address is not valid." This would be a poorly worded error message in any user interface, but the emotional aspect to newsletters increased the user's anger: "Mine's as valid as the next person's! ... It's questioning my validity as an entity in cyberspace." Sixty-nine percent of users said that they look forward to receiving at least one newsletter, and most users said a newsletter had become part of their routine. Very few other promotional efforts can claim this degree of customer buy-in. | ||||
User Research |
To assess how people use email newsletters, we conducted four rounds of user studies, as well as pilot studies to refine the test methodology. In total, 109 users participated in our testing. Most participants were in the United States, but we also studied users in Australia, Hong Kong, Japan, Sweden, and the United Kingdom.
The first study focused on testing newsletter usability in terms of subscribing, unsubscribing, and maintaining the user's account. For this study, we systematically tested ten different newsletters, which we assigned to users to ensure that they hadn't previously used a newsletter's subscription interface. Most of the study was conducted as a traditional laboratory test: we observed users individually as they read newsletters and attempted to subscribe and unsubscribe. We conducted additional parts of the study remotely, through telephone calls. We conducted our second study remotely, using a diary methodology that allowed us a much wider geographical distribution of participants. The second study included all of our international participants and all of our U.S. users who were not on the East Coast. The first study had the benefit of systematically testing a set of design variations, with multiple users for each design. However, it also had the distinct disadvantage of people getting newsletters that they had not selected themselves. For the second study, we looked in detail at users' experience receiving and reading newsletters that they'd already subscribed to on their own initiative. In total, the participants subscribed to 345 different newsletters, and we studied 101 of those. We studied users' newsletter experience over a four-week period for most participants, and over two weeks in a few cases. This longitudinal approach allowed more emphasis on how people deal with incoming newsletters during their workday. We were also able to test many more B2B and intranet newsletters than we could cover in the first study, which mainly tested B2C newsletters. Of the newsletters received by the users in our second study, 65% were for personal purposes and 40% were for business purposes (users viewed 5% of newsletters as both personal and business, so we counted those twice). We conducted the third study using an eyetracker. Eyetracking let us record where users were looking on websites as they subscribed or unsubscribed. We also recorded how users looked at their inboxes and how they read individual newsletters. During the third study, we systematically tested twelve newsletters (using a controlled methodology to ensure that all newsletters were used evenly) and tested forty newsletters in a less controlled manner (users were free to pick newsletters of interest from an inbox, so some newsletters were read much more than others). Finally, we tracked users' eye movements as they read a total of sixty-five newsletters from their personal inboxes. By definition, each of those newsletters was read by only a single user. In addition to studying newsletters, the third study included a component in which people used a variety of RSS (Real Simple Syndication) readers to read news feeds. This let us compare the newer medium of feeds with newsletters, which are now an established media form. Our third round of research furthermore included a field-study component in which we observed users in their offices during a normal workday. This ethnographic approach let us learn about the use of newsletters and news feeds in an environment with many competing information sources and demands on users' time. The fourth study was conducted as lab-based usability tests in the U.S. and the U.K. In this round of research we returned to testing the subscribe functionality of websites that publish newsletters. We also tested users accessing newsletters on their mobile devices. Finally, a major focus of this newest research was to access the impact of tone of voice in newsletter content. | ||||
High Nominal Usability |
Our test users experienced unprecedentedly high levels of task completion in their attempts to subscribe and unsubscribe to the newsletters in the study: 85% for subscribing and 91% for unsubscribing in our most recent studies.
Although high, these rates could still be improved. If, for example, a newsletter with 50,000 subscribers ensured that everyone could correctly operate its subscription interface, it could add an estimated 8,800 subscribers on average. Still, most usability studies find success rates of around 78% for other Web design areas. Clearly, relative to this, the newsletter usability success rates are incredibly high—even though they're still lower than anything we would deem a truly great user experience. There are probably two reasons for the high success rates here. First, the tested functionality is very simple: Get on or off a mailing list. In fact, the main failures came on websites that complicated this functionality, such as by combining newsletter subscriptions with site registration. In general, it's easier to design a simple user interface when the underlying functionality is simple. The second reason that the subscription process had much better usability than other Web designs is that newsletter designs are highly accountable. In many other Web design areas, project managers can delude themselves and their bosses that user-hostile designs, such as splash pages, offer some benefits. Create a design where people can't find what they want and page views might even go up as users wander aimlessly before they leave (and give up doing business with the company). With a newsletter subscription design, users either subscribe or they don't. In the latter case, websites will eventually tone down their design excesses and focus on simplicity, and subscriptions will increase accordingly. If a site were to replace a simple design with a complex one, it would soon notice a decline in new subscriptions and revert to the previous design, writing off the bad design as an expensive usability lesson. | ||||
Increasing Newsletter Usability |
We have tested subscription user interfaces in three rounds, and the measured usability has improved each time:
The faster task times are particularly impressive, with an improvement of 43% over an eight-year period. When user experience metrics improve over time, there are two possible explanations: higher-usability design or higher-skilled users. In the case of email newsletters, there's probably a bit of both. But, overall, users don't seem to be much savvier now regarding newsletter subscriptions than they were eight years ago. So, the main explanation for the improved performance is probably that websites are indeed getting better at designing to attract newsletter subscribers. One data point to support this assessment comes from our analysis of political campaign newsletters from national elections in 2004 and 2010. Averaged across the parties, the newsletters' compliance with our usability guidelines was:
| ||||
Low Perceived Usability |
Even though users successfully unsubscribed 91% of the time during the test sessions, they often refrained from even trying to get off mailing lists that they no longer wanted to receive.
The four main reasons people didn't attempt to unsubscribe were:
Some newsletters deliberately make it difficult to unsubscribe by hiding the instructions or making them overly complex. The motive is probably to retain as many subscribers as possible to maximize the reach of permission marketing programs. In reality, however, you don't have users' "permission" once they stop wanting the newsletter, regardless of whether they jump through the hoops required to get off the list. If users keep getting unwanted newsletters, the messages will start to backfire and become regular reminders that they're annoyed with your company. Better to let them go. | ||||
Speed Matters |
In our latest studies, the subscribe process took three and a half minutes, and the unsubscribe process took one and a half minutes. Even though these task times are not prohibitive, they're much too long for the simple functionality involved.
We recommend setting a usability goal of allowing an existing user to unsubscribe in less than a minute, assuming that the user has a recent copy of the newsletter at hand. New subscriptions should also take less than a minute when subscription requires only the user's email address. Even if additional information is required, users should be able to subscribe to free newsletters in less than two minutes. Only newsletters that involve a subscription fee should be allowed so many steps that the average user can't subscribe in two minutes. Users are very demanding with respect to the efficiency of operations like subscribing or unsubscribing. For both tasks, we found extremely strong correlations between the task time and the users' subjective satisfaction: r = -.63 and -.95, respectively. These correlations basically say that the slower the subscribe or unsubscribe process, the less people will like the site. For each additional minute it takes to subscribe, you will lose 0.3 satisfaction points on a 1 to 7 scale, and for each additional minute it takes to unsubscribe, you will lose 0.6 satisfaction points. As indicated by the numbers, users are substantially more critical of a slow unsubscribe process. Once they want out, they want out quickly. A perfect satisfaction rating of 7 would require instantaneous task performance according to the regression estimates. It seems impossible to create a design that allows users to subscribe and unsubscribe in 0 seconds, but that's ultimately what users want. It's nobody's goal in life to "manage subscriptions," so any overhead becomes an annoyance. Extreme simplicity and ease of use are necessary to make a positive impact on customers. | ||||
Mobile Use: Quick Reads, Slow Reads |
Since our first research with mobile devices in 2000, we've found that killing time is a killer app for mobile use. When people are out and about, they often find themselves in situations with a few minutes to kill, and mobile content can fill that need.
The new study replicated this old finding. Many users read newsletters on their mobile devices when they had time to spare. In these circumstances, some users said that they were more willing to look at longer content than they'd normally read while processing email on their desktop computers. On the other hand, much mobile use is characterized by even more time pressure than desktop use. People often check email on mobile devices during quick breaks when they want to allocate time only to high-priority messages. Thus, some newsletters should be even more quick and to-the-point for mobile use, while others can afford to present more leisurely content. It might be better to do one or the other rather than aim for a middle-of-the-road approach that will satisfy neither usage scenario. One big problem in the latest study was that many newsletters were poorly formatted for reading on small mobile screens. Indeed, people rated the ease of reading newsletters on their mobile devices a miserable 3.3 on a 1–7 scale. | ||||
Significant Platform Diversity |
The Web is a fairly uniform environment. Most users have either Internet Explorer or Firefox, and almost all run these browsers on Windows. Yes, some people use Macs and browsers like Safari or Opera, but these browsing environments offer pretty much the same features as the main PC browsers. The differences between Web browsing platforms are like the differences between Indian and African elephants, not like those between crabs and eagles.
In contrast, email newsletters must contend with platform diversity that is much more like the biodiversity of the Cretaceous Period (before the comet hit). Although Yahoo! was the most commonly used email reader in our recent study, it accounted for only 31% of users. Eight additional platforms were represented, but people also commonly use others, including Eudora, Lotus Notes, and a variety of mainframe systems and Unix mail variants. Each email platform has a different way of displaying the From line, the Subject line, and the newsletter content. They also have different approaches to spam filtering and other things that influence the subscriber's user experience. This diversity makes it crucial that newsletter designers test their subscribe and unsubscribe processes—as well as the actual newsletter delivery and display—on all major email platforms. | ||||
Spam Is a Fact of Life |
There's a little good news (but mostly bad news) about the impact of spam on email newsletters. The good news is that users in our recent studies were better able to differentiate legitimate opt-in newsletters from unsolicited messages than they could in the past. In our earlier newsletter usability studies, users sometimes confused the two. Now, spam has a very prominent profile in terms of popular awareness, press coverage, and the sheer amount of it hitting inboxes. Users have thus developed a reasonable understanding of the spam phenomenon as opposed to simply being baffled about unexpected messages.
The bad news is that the increased burden on email users has caused people to become even more stressed and impatient when processing their inbox. Users have less tolerance for newsletters that waste their time. We have also found that people often use their spam filters as a shortcut to eliminating newsletters they no longer want. Instead of unsubscribing, which users often view as too cumbersome, they simply tell their spam-blocker that the newsletter is spam. Voila, that newsletter no longer shows up in the inbox. The fact that many users will declare a newsletter to be spam when they tire of it has terrifying implications: legitimate newsletters might get blacklisted and thus be undeliverable to other subscribers who still welcome new issues. This is a compelling reason to increase the usability of the unsubscribe process: better to lose a subscriber than to be listed as spam. | ||||
The Battle for the Inbox |
Users are getting pickier and pickier about which newsletters they'll read. Some are purposefully cutting back on the number of newsletters they receive. These users view newsletters as being in direct competition with each other for a limited number of slots in the inbox. Users will unsubscribe from a newsletter or stop reading it—even if it's good—if they come across a different one on the same topic that better serves their needs.
People get a lot of email. They don't have time to read a lot of text. In our research, users spent an average of 51 seconds on each of the newsletters they read from their own inbox. Users spent an additional 33 seconds on information found by pursuing newsletter links to websites. One clear trend since our last study is the ever-increasing amount of mail in people's inboxes. In both the third and fourth studies, we asked users to log in to their own email accounts, and the number of new or unread messages is now 300% higher than it was just four years ago. Not only do you have to compete with other newsletters to get people to subscribe, you must compete with all other email and get users to open your messages, pay attention to your content, and click through to your site. This has always been true, but the competition is becoming more intense as users are getting more swamped by online information sources than they were in the past. Email services are offering increasing amounts of storage space, often in the gigabyte range. This allows users to save more old newsletters than was possible in the past. On the one hand, having users archive your newsletter means that it becomes a form of permanent outreach and will show up when they search their personal information space. On the other hand, many newsletters might be saved and never read. It's worth it to use informative and enticing subject lines that encourage users to read a newsletter while it's fresh. | ||||
Scannability and Immediate Utility |
The most frequent complaint in our study was about newsletters that arrived too often. And, when we let them vent, the most frequent advice our study participants had for newsletter creators was to "keep it brief."
Newsletters must be designed to facilitate scanning. In our first study, 23% of the newsletters were read thoroughly. In our third study, four years later, only 19% of the newsletters were read thoroughly. The drop in percentage of thoroughly read newsletters is a good indication of the increased volume of email that users have to process. The dominant mode of dealing with email newsletters is to skim them: that's what happened to 69% of the newsletters in our most recent study. Of the remaining newsletters, users only glanced at them or at most read a few items. Sometimes users will simply skim the headlines to get an update or overview of what's going on in the field covered by the newsletter. As one user said, "I like to keep up-to-date in the industry, but rarely delve deeper than the cover page." Other times, users deliberately pick out those few elements that are most important to them and ignore the rest. As another user said, "I review the contents by company and only read the companies of interest to me." Designing for users who scan rather than read is essential for a newsletter's survival. Scannability is important for websites as well, but it's about 50% more important for newsletters. This implies the need for layouts that let users quickly grasp each issue's content and zero in on specifics. Content and writing styles must support users who read only part of the material. In addition to subject lines, users now pay more attention to message previews. This change is partly driven by the increasing email volume (users can decide whether to dispose of or keep messages without opening them all) and partly driven by more mobile access (users can't see much on a small screen). It's always been a guideline to start a newsletter with the most important stuff, but the increased use of previews makes it even more important to focus on high-value content at the start of a message, since users are less likely now to look beyond it. (It almost goes without saying that "high-value" is judged based on what's valuable to the recipients — not on what you feel like promoting today.) Newsletters must be current and timely, as indicated by three of the four main reasons that users listed for why a certain newsletter was the most valuable they received. All of the following four reasons were given by more than 40% of users:
Because newsletters build relationships with readers and because it's so easy to ignore individual issues, newsletters do get some leeway if they are predictably relevant at certain times. During those periods when a newsletter isn't relevant to the user's immediate needs, the user might simply ignore it rather than unsubscribe. For example, a speech pathologist at an elementary school said that she could only purchase new products at the end of the school year, and so ignored product-related newsletters most of the year. Still, she didn't unsubscribe, and simply receiving the sales newsletters reminded her of the brand when she received her budget. Users will often avoid signing up for newsletters because they feel crushed by information overload. It is the job of the newsletter publisher to convince users that the newsletter will be simple, useful, and easy to deal with. A predictable publication frequency that is not too aggressive is usually best, except for newsletters that report breaking news. A regular publication schedule lets users know when to look for the newsletter and reduces the probability that they'll confuse it with spam and delete it. Also, writing good subject lines is crucial, both in encouraging users to open the newsletter and helping them distinguish the newsletter from spam. We recommend including content from the issue in each subject line, even though it's a difficult job to write good microcontent within the fifty- to sixty-character limit that many email services impose. | ||||
Competing With Social Networks |
Email newsletters are a better way to stay in touch with customers than updates posted on social networks like Facebook or Twitter:
Although users weren't really interested in receiving company updates through social networks, they are information sources that compete for people's attention. Some users reported hearing about breaking news through Facebook before they received a news alert via email. Your newsletter subscribers are usually your most loyal customers and fans, so it's important to treat them better than the more fickle audience on social networks. Obviously, having enhanced content in the newsletter is one way of doing so. But you should also make sure to send out the newsletter announcing, say, sales or new products before tweeting such news. On the positive side, your newsletter can confer social advantages to subscribers: you can feed them tidbits that they can post to their own contacts, making them feel more informed. This rewards your most loyal followers, while still spreading your message on social services. | ||||
RSS/News Feeds |
The third edition of this report included 16 guidelines about RSS news feeds. The chapter about RSS has been removed from this edition, because our earlier RSS findings have been superseded by a new report on RSS (and about social media), titled Streams, Walls, and Feeds. This other report contains 24 guidelines for RSS feeds (and 85 guidelines for social media, such as Facebook and Twitter). The social media/RSS report is available from http://www.nngroup.com/reports/streams . | ||||
Video in Email |
Video has a role in newsletters, but it's a small one for most topics. Users are often rushed when processing email, and watching videos takes time. As one study participant said, "I probably wouldn't watch this. It's a video, not text. I expected an article, not a video. With video, you have to watch the whole thing. Even if it's just a minute, I'm not into watching video. And if I was on the phone, I couldn't watch it. I wouldn't want to watch it on my phone, anyway."
In most cases, newsletter videos should be secondary and supplementary to text and images that users can scan directly in the newsletter. To set proper user expectations, the design must make it absolutely clear whether something links to a video or to an article. Users were hesitant to click on videos within newsletters if they weren't sure what they would get. It's important to clearly describe the video in words. Also, carefully pick a preview image that communicates the video's nature instead of simply showing the first frame. Finally, state the video's duration. | ||||
Long Live the Internet (and Your Newsletter) |
The Web, email, and newsletters are not exactly new phenomena anymore. As a result, we're seeing an intriguing longevity effect. For example, one study participant decided to subscribe to a newsletter after reading about it in a (dead-trees) book. Another signed up for a newsletter while attending a conference ("honestly, to get a free tote bag") and proceeded to read that newsletter after the event.
50% of our users said that email newsletters influenced their B2B purchases, but the influence was only occasional, when the timing happened to be right. Often, the newsletter served to grow or retain a vendor's reputation or to maintain a relationship during dry spells when users lacked the budgets needed to actively conduct business. When it comes to customer relationships, newsletters must be seen as a long-term investment: they work their magic over time. On the strategic level, this is why you should emphasize value-added publishing instead of simply spamming too-frequent newsletters to any email address you can lay your hands on. On a more tactical level, it's why you should follow old guidelines like keeping the same URL year after year, instead of building (and abandoning) new microsites every season. | ||||
Future of Email Newsletters |
Eight years ago, in our first report about newsletter usability, we said about the future of email newsletters: "There may be none. Legitimate use of email is at war with spam, and spam may be winning."
Although eight years is a short period in which to assess big trends, we now believe that this assessment was too negative. Email newsletters are so powerful that the best of them do have a future, despite ever-more adverse conditions. Ever-increasing information overload is definitely making users reluctant to sign up for more email. And once newsletters arrive in the user's inbox, they might simply be deleted as part of the ruthless mass deletion procedure aimed at the morning's spam. Finally, as discussed above, fear of spam and other email abuse is keeping users from dealing rationally with newsletter subscriptions. When we asked users why they liked email newsletters, more than one-third highlighted the following three benefits:
Comparing email newsletters with other media, one user said: "Bottom line, I'd rather have it in an email newsletter than in the regular mail. I can click Delete if I don't want it; I don't have to throw anything away; and it is usually easier to unsubscribe if you don't want to get anymore." Convenience rules. This is one of the few times we have found that the virtual world was better and more convenient than the physical world. Usually, websites have such poor usability that they compare very unfavorably with real-world stores or in-person services and communities. In contrast, email newsletters have a very strong position. |
||||
|
Return to Email Newsletter Usability, report description
|
|||||
