Remote Card Sorting– How We Did it for the Redesign of the SAP User Experience Community

Card sorting is a user research method which is typically done manually and with pen and paper. In this article, I will explain how we turned it into a remote method. To be honest, we improvised. We had done the card sorting activity in person already with many customers, but we wanted to get feedback from users further afield. Our approach did not involve any fancy tools; only software common in corporate environments – just PowerPoint and a screen-sharing software (in our case, SAP Connect), but no special remote card sorting tool.

We are not saying this is the best or the only way to do remote card sorting. It is just the way we did it. We liked it. We had the impression our participants liked it as well. At least, they said so ;-). And it worked out very well, which is why we want to share our experience with you.

Project Background

In 2013, we started with our research project that paved the way for the redesign of the website you are browsing at this very moment (www.experience.sap.com). This research project comprised a variety of different research activities, ranging from early user research to usability validation sessions.

We decided to do card sorting because the information architecture is a key aspect of the usability of the Community. As card sorting helps to get insights into the needs and terminology expectations of users, this was our method of choice. This method enables the project team to better understand the user’s mental model in order to create appropriate and meaningful content categories.

Remote_Card_sorting_picture 1

Picture 1: Examples of content categories on virtual “cards” for the new site

Taking the (Remote) Challenge

We had face-to-face card sorting sessions at a mobile usability lab at an SAP user group conference as well as with users locally. Here, paper, pens, and printed sticky notes were all we needed. Oh yeah, and a camera to take pictures afterwards. Yet, we also called for participation globally to get a broader picture of the needs of our future audience and users. Thus, we had to prepare for remote sessions in order to do the card sorting with users worldwide by leveraging SAP’s Customer Engagement Initiative.

The challenge was to prepare a setup that works for non-techies and would not distract the participant from the card content. Looking at it from a results perspective, we are very happy we took the challenge. For one, more users shared their feedback with us than we could have involved in the timeframe of our project if we were to travel to the users in person. Second, we collected very diverse feedback from different cultural backgrounds. Third, we are happy we didn’t miss out on this exciting experience to expand our method portfolio.

“The mechanics” – Overcoming Technical Hurdles, Gaining Valuable Global User Insights

Preparation was key! In PowerPoint, we prepared a mock-up screen of the landing page of the website, the SAP User Experience Community. As you can see, it was very basic, more or less a blank page with a header and a footer that should convey the notion of a website. On the left, we placed “cards” (green) for potential content categories that would later serve as navigation labels. (In PowerPoint, we used the text box drawing tool, colored it green, and then right-clicked on the box and chose “Save as picture…”, saved as a png, deleted the text box, and then reinserted the images into the PowerPoint document. This was an important step so that users could easily move the boxes around without the boxes going into edit mode.) The content category proposals were derived from the results of the stakeholder interviews that we had conducted before running the card sorting activity with a different sample of users and stakeholders. (Note: The small numbers in the bottom right corner on these notes helped us to quickly find the detailed descriptions of the categories which we had written down and numbered in a separate document. When a user asked for more information about a category, we could quickly find the more detailed info because they were numbered.)

remote_card_sorting_picture_2

Picture 2: Screenshot of the PowerPoint slide as it appeared when users first saw it

On the right-hand side of the slide, we arranged blue and yellow text boxes (not images, but editable boxes) for the creation of headers (blue) and new content categories (yellow). On the very right edge of the screen, we placed frames (dark blue) to allow users to group several content categories. Finally, we placed stars on the very right-hand side for the final prioritization activity.

Running the Sessions

The first step in interacting with our participant on the other end of the telephone line was to open the virtual meeting room and to start the conference. SAP Connect is our company tool for virtual online meetings – but any other screen-sharing tool would work. Once the participant was connected with us, and the introduction of the session had been completed, we shared our PowerPoint slide and gave the participant control of the screen (see picture 2). We then explained the purpose of the various boxes and shapes on the slide we had prepared and then asked the participant to take a look at the different cards, to read through them and to ask us if a card was not clear. We also encouraged them add whatever they liked to the screen during the session by using the yellow cards. In a next step, it was the task of the participant to “grab” the cards and place them on the white space in the way that was most logical and meaningful to them. By dragging and dropping the cards with the mouse from their end, they were able to move the cards around and could place them on the screen as they saw fit. The notetaker on our end wrote down the comments of the user as he or she “thought” aloud. SAP Connect allows the host to give full control to the participant, but the host can take back control if necessary. This allows the user to interact freely with the content, and there is no need to send the file to the participant ahead of time.

If the labels on the cards did not fit to what the participant had in mind, he or she created their own cards by writing on the blank text boxes on the right-hand side of the screen. If they found that some cards should be grouped together to form a larger category, they could pick one of the dark blue frames on the right-hand side for grouping. At the end of the session – after the user had arranged and grouped and labeled their cards – we had them use the stars from the right-hand area of the PowerPoint slide to prioritize the content. Below you find an example of one of our outcomes of the remote card sorting activities.

remote_card_sorting_picture3

Picture 3: Example of a final outcome of one of the remote card sorting activities

That way, we gained very, very valuable insights into the future structure and the content areas our users wanted most. Very important: Don’t forget to press the save button!

In general, what we discovered was that the terminology that we originally had on the cards, did not match very well at all the terminology that our users would have used. The card sorting activities helped us tremendously to completely rethink our information architecture and find a structure and terminology that much better fit our users’ expectations. The findings from our (onsite and remote) card sorting activities nurtured the creation and development of our wireframes and prototypes with which we ran usability tests (again, both onsite and remotely) in the next research phase. As the results of our research activities are reflected in the newly relaunched SAP User Experience Community, we truly hope they stand up to your user test!

Either way, please share your thoughts with us. We are keen on hearing about your experiences with this Community as well as with running remote card sorting activities. A BIG THANK YOU again to all participants of our research project who supported the redesign of the site. We hope you found the sessions as interesting as we did!

In case you are interested in participating in an SAP user research activity, please check out the “Participate in Improving Products with SAP” area of this site.

Not logged in