Card Sorting: Uncover Users' Mental Models for Better Information Architecture

A card-sorting study is a specialty UX research method used to uncover users’ mental models of the information architecture (IA) of your digital product. In other words, card sorting can help your users find the information on your site more easily.

Definition of Card Sorting

Card sorting is a research method in which study participants place individually labeled cards into groups according to criteria that make the most sense to them.

The set of cards can include pages, links, images, or descriptions of pages that users would navigate to on your website.

To run a card-sorting session, present each participant with the unsorted set of cards. Then ask the participant to sort the items, information, or concepts into groupings.

For example, an athletic-clothing ecommerce site might give a participant a set of cards labeled:

• Sweatshirts
• Gloves
• Hats
• Tank tops

A participant might put Sweatshirts and Tank tops into a category labeled Tops, and Gloves and Hats into a category labeled Accessories.

In-Person vs. Remote Card Sorting

Card sorting can be conducted in person or remotely. In-person participants use physical index cards, while remote participants use digital cards via web-based tools like Optimal Sort.

Remote card-sorting studies tend to be more common because the supporting software can analyze and reveal common groupings, category names, and item pairings, reducing manual work for researchers. 

When to Conduct a Card-Sorting Study

Card sorting provides insight into how users naturally categorize information on your site and helps teams develop an information architecture that best matches users' mental models, ultimately improving the findability and discoverability of your content.

Let's imagine that you're redesigning the car-rental site Paylesscar.com. The company offers 26 vehicle models that customers can choose from. How would you organize those vehicles into categories so people find their preferred rental quickly? Technical car classifications like economy, intermediate, and full-size are unclear for many users. Instead of using these categories, you could run a card sorting study where users organize vehicles into groups that make sense to them and then see what patterns emerge.

Use card sorting when you need help organizing content, services, or product offerings in an easy and intuitive for your users. (A related method, tree testing, is best used for evaluating existing information architectures.)

Card-Sorting Process

Conducting a card-sorting study requires four steps:

1. Create the cards.
2. Ask participants to organize the cards into groups.
3. Have participants label each group.
4. Analyze the data you’ve collected.

Prepare Materials for a Card Sort

Decide the Number of Cards

The number of cards depends on the complexity of your content, study goals, and user familiarity with your site. We recommend 30­–50 cards to prevent user fatigue and encourage thoughtful groupings from participants. This range may seem high, but once users conceptualize a few groupings, the rest become easier to place.

In a recent client project where we conducted a card sort with around 50 cards, participants took time establishing key groupings initially, but once they did, the rest of the cards became much easier and quicker to sort.

Label the Cards

Create each card label within a tool of your choice. OptimalSort is a popular option for many UX researchers.

The cards can include pages, links, images, or descriptions of pages that users would navigate to on your site or application.

Avoid using identical words in labels for different cards because participants will automatically group those instead of thinking about deeper relationships between items. For example, using cards like Toyota Camry, Toyota Matrix, and Toyota Land Cruiser may cause users to group all Toyota cars, overlooking vital details like the size of the car and seat capacity.

Conduct a pilot test with a smaller group of participants to assess the appropriate number of cards and whether the card labels are easy to understand.

Recruit Participants

The number of participants you need to recruit varies on whether you run a quantitative or qualitative card-sorting study.

• Qualitative card sorts help researchers understand why users place cards into specific groupings. Researchers often ask users to think out loud to understand their mental models and the reasons behind their groupings. Researchers need to recruit at least 15 participants for this approach.
• Quantitative card sorts help researchers understand the statistical validity of common groupings. It focuses on how many times people grouped certain cards together. For quantitative studies, we recommend at least 30–50 participants to ensure that results are generalizable to a broader population. If you still aren’t seeing consistent patterns, recruit more participants for the study. Recruiting participants who represent your target audience or current user base is crucial. Aim for a diverse set of participants to ensure a representative sample.

Increasing the number of participants in both approaches won’t negatively impact data findings, but these recommendations consider limited resources and budgets.

Conduct a Card Sort

Even though it is possible to conduct in-person card sorts (and you may need to do so for certain user populations such as children or people with accessibility challenges), most card-sorting studies today are conducted remotely using a remote video platform.

In a remote moderated study, the researcher is present and asks follow-up questions live. Video-conferencing tools like Zoom and Microsoft Teams will suffice in this scenario.

In a remote unmoderated card-sorting study, a platform administers all instructions and questions without a researcher present. Each participant completes their card-sorting session on their own. Depending on the platform, a recording of the session or only the final groupings may be available at the end of the session.

Instruct Participants

Before the start of the session, inform users that:

• Card-group sizes can vary.
• There is no preset number of groups they need to aim for.
• They can rearrange cards if they change their mind about the placement.
• If they don’t know what a card means, they should leave it off to the side rather than assigning it to a random group.

Here’s a template for card-sorting instructions. Modify it to fit the needs of your study:

Thank you for agreeing to participate in our study. This study should take [insert time] to complete.

In this study you will organize a set of cards into groups based on perceived similarities. We encourage you to think out loud and explain your thought process as you sort cards. Your insights will help us understand how to better structure the content on our [website/application].

Step 1: Scan the set of cards.

Step 2: Organize cards into groups that make sense to you. There are no right or wrong ways of grouping cards. Here are a few considerations to keep in mind as you group cards:

• Card group sizes can vary.
• There is no preset number of groups the needs to aim for.
• You can rearrange cards if you change your mind at any point.
• If you don’t know what a card means, leave it off to the side rather than assign it to a random group.

Step 3: Label each group of cards.

Participants Organize Topics into Groups

Ask participants to scan the set of cards and organize them into groups that make sense to them. Remote tools automatically place ungrouped cards separately from the grouped cards. Encouraging users to think out loud during this process can help you understand why they are making certain placement decisions.

Participants Label Each Group

After each participant organizes the cards into groups, ask them to label each group. It's essential to do this step after (rather than before) they form groups, so participants don't limit themselves to specific categories during the sorting process. Users should be free to rearrange groups at any point.

How your participants label their cards will depend on whether you’re running an open, closed, or hybrid card sort:

• Open card sorting is the most common type of card sort. Users create their own groups with no predefined categories. We recommend this approach because you get a holistic view of users' mental models, and you may discover new ways to categorize certain items.
• Closed card sorting is sometimes used for validation. Users are given predefined categories and asked to organize the individual cards into those categories. Instead of closed card sorting, we typically recommend tree testing (also known as reverse card sorting) as a way to evaluate navigation categories.
• Hybrid card sorting is only used in specific scenarios. Researchers provide users with some predefined categories and the freedom to create their own categories. We generally recommend against this approach because the predefined categories bias users into identifying similarly structured categories.  You can consider running a hybrid card sort if you are unsure about most categories but feel confident in one or two.

Specify the card-sort type in the web-based tool of your choice and input predefined categories accordingly.

In an open card sort, don’t expect participants to create effective labels. Category names they come up with will not necessarily be your final labels. Card sorting doesn't consider other context cues on your website; these cues can influence final label names.

Ask Follow-up Questions

Ask users to explain the rationale behind the groups they created. This step is optional but highly recommended. Follow-up questions in a card sort may include:

• Were any items especially easy or difficult to place?
• Did any items seem to belong in two or more groups?
• What are your thoughts on the items left unsorted (if any)?

In an unmoderated study, where the is no live facilitator, you may be able to include these questions in a post-sorting survey to get additional insights (depending on your tool). You may choose to include rating-scale questions as well; for example:

On a scale of 1 (very easy) to 5 (very difficult), how easy was it for you to group the cards into categories?

In moderated or in-person studies, where a facilitator is guiding the session, tailored questions are possible. For example:       

I noticed you placed card X in group Z.  Can you tell me more about why you feel that card belongs in that group?

Analyze Card-Sorting Data

Compile Data

Compile your data into a format that will be useful for analysis. Card-sorting tools offer advantages over physical index cards because sorting results are compiled automatically and displayed through various data visualizations helpful for analysis. Researchers find web-based tools more efficient than manually inputting data into spreadsheets to track groupings and card placements.

Identify Patterns and Common Groupings

Identifying patterns, common groupings, category names, and themes can be challenging due to the varying placements of cards by participants. Tools like OptimalSort offer diverse visualizations for in-depth analysis of placements.

For example, a similarity matrix visually represents the relationship between two cards. If you were to conduct a card-sorting study without a card-sorting tool, you would have to manually identify frequencies of card groupings.

Combine Patterns Identified with Qualitative Insights

Even if you run a quantitative unmoderated card sort, it’s important to run a few moderated sessions to understand what participants are thinking as they group items.  You’ll gain insight into the rationale behind the groupings and why a particular card was hard to place. For example, was a card hard to place because the card label was unclear or because it seemed unrelated to the rest of the topics?

Conducting an In-Person Card Sort

In-person card-sorting studies are less common because online tools have proven more efficient for analysis. The steps of an in-person card-sorting study are mostly the same as for a remote one, but here are a few key differences:

• Prepare physical cards: Participants group physical index cards rather than digital cards.
• Collect data manually: Researchers must collect and process data manually. Consider using a camera to capture card groupings quickly after each session and use a spreadsheet to track data.
• Designate spaces for grouped and ungrouped cards: Designate spaces next to each other for grouped and ungrouped cards to prevent confusion while users handle multiple cards and groupings. Remote tools automatically place ungrouped cards on the left, with a line separating them from the grouped cards. 

Limitations to Card Sorting

Card sorting is best suited for generating information-architecture structures. It has the following limitations:

• Lacking context: Card sorting focuses on isolated content items and doesn’t take into account the broader context of the site’s images, structure, and links that carry significant information scent for users.
• Limited navigation pathways: Card sorting doesn’t reveal the full paths and hierarchies that users may go through to get the content they need. Users can create only one level of categorization, not subcategories.
• Limited number of cards: There is a limited number of cards that you can fit in a session before users get tired. When you have over 50 cards, participants may create a large Miscellaneous category and stuff cards in it. It's challenging to discern whether they placed cards in this category because they didn’t know where to place it or because they got tired.
• Challenging to interpret results: Interpreting card-sorting results can be challenging because users may have different mental models. The researcher's expertise and the use of other methodologies, along with card sorting, can help with a comprehensive analysis.

Conclusion

Card sorting is a valuable method in information architecture because it gives insight into users' mental models about content groupings on your site. There are a lot of nuances involved in carrying out this research method; therefore, it's crucial to carefully think through your research objectives and the type of card sort you want to conduct. 

References

Tullis, Tom, and Wood, Larry. (2004) How Many Users Are Enough for a Card-Sorting Study?, Usability Professionals Association (UPA) 2004 Conference, Minneapolis, MN, June 7–11, 2004.

Ethan Lantz, Jared W. Keeley, Michael C. Roberts, Maria Elena Medina-Mora, Pratap Sharan, and Geoffrey M. Reed. 2019. Card Sorting Data Collection Methodology: How many participants is most efficient? Journal of Classification 36, 3 (2019), 649–658. DOI:http://dx.doi.org/10.1007/s00357-018-9292-8