With an integrated prototyping and user feedback collection tool like Build, design processes can change dramatically and turn into a truly collaborative experience. It’s never been that easy to collect feedback on a UI while still drafting designs. But is “feedback” enough? There are two pathways to effectively inform design – rigorous testing with end users, and rigorous, professional UX methodology. The two are not mutually exclusive, but instead complement each other. Both are needed for designing truly great user experiences. Build is a prototyping tool that also supports you in collecting end user feedback, but anyone can give feedback, right? Feedback doesn’t have to ONLY come from end users, professional reviewers can also give feedback.

Cognitive Walkthrough in a Nutshell

Cognitive walkthrough (CW) is a user experience inspection technique specifically developed for discovering learnability issues in user interfaces. Developed in 1990 by Cathleen Wharton, John Rieman, Clayton Lewis, and Peter Polson, it became famous in the “discount usablity” movement in the early 2000s, when faster and cheaper methods of usability evaluation were being searched out. Quickly after the first publication, more and more streamlined versions were developed. In a nutshell, a cognitive walkthrough consists of challenging a UI design – in fact, any design – with four key questions to be asked at each interaction step:

  1. Will the user try to achieve the right effect?
  2. Will they notice the right action is available?
  3. Will the user associate this action with what they are trying to achieve?
  4. Once the action is performed, how will the user know they made progress?

Whenever one or more of these questions can’t be answered satisfactorily, there will be a usability issue. Apart from digging up those issues, dealing with the questions will quite likely point you directly to design solutions that can answer them.

Let’s look first at an everyday example. In public restrooms, for hygienic reasons, often the faucet for washing your hands is equipped with a light sensor, so you don’t have to touch anything before having washed your hands. Will users try using it? Question 1 already points at a design issue – it becomes immediately evident that making the washbasin appealing to use is way more than an aesthetic requirement. An ugly or filthy sink may stop the process right at step one. Question 2 is not trivial either – users expecting traditional faucet valves will need to find out that something is different here, and what exactly this something is. Often, the light sensor is in plain view right at the faucet, so this part should be easy – if, and only if, the user recognizes the sensor for what it is (not a camera? decoration? a logo?) and has an idea what it does (question 3). Suppose all this worked out all right, and the user held their hands in front of the sensor, and the sensor recognized the approaching hands, the faucet will turn on. If water comes out, this answers question 4 – if this takes a few seconds of the user waving their hands in front of the faucet, they will probably walk away.

Missing out on any one of the four challenge questions will break the process, and users will leave the restroom with unwashed hands! Now think of all the labels, signs, and stickers you have surely seen in public restrooms: they all have their roots in designers not having properly considered the four cognitive walkthrough questions. Eventually, someone had to answer them, so they ended up adding signage.

Working as a designer, I always found the four CW questions incredibly useful in any stage of the design process. Even when analyzing usability test results, those questions help tremendously to understand usability problems, and to find solutions. Given a clear understanding of what users want and know how to do, you can literally debug a UI design.

Well, all this still is a solitary perspective. What Build adds on top of that is the collaborative part. Research on UI inspection methods has shown over and over again that they are most effective when a number of inspectors work first in solitude, then compare notes, discuss, and prioritize findings. Build supports all of these steps elegantly. Here’s how.

CW in Build Step by Step

  1. Set up what to inspect.
    You can build a prototype right in Build, or import a series of screenshots. You can also inspect a live application; then a series of screenshots would be needed to record inspection results in Build.
  2. Identify who the users will be.
    Inspectors will need to assess the UI based on the perspective of an end user, so they need to know that perspective in the first place. If you have a persona description or a POV from a Design Thinking project, that would be perfect. Even better, work with end users who have been briefed to use the method (see 5).
  3. Define the tasks for which you want to inspect the UI.
    CW is always task-based. Inspectors will need to know those tasks, so they can challenge the UI for each task step with the 4 CW questions. If you are working with key users or other people who know what the tasks are, a mere list will do. Others will need step-by-step instructions.
  4. Set up a Build feedback study.
    Prototypes built in Build can be imported right away, so can screenshots. Create one question for each task. For prototypes, you’ll have to indicate a start screen. The question text will be a brief task description, together with the four CW questions. A copy-paste template is provided below.
  5. Brief inspectors.
    Cognitive walkthrough works best with 2-5 inspectors. Inspectors do not have to be usability or design professionals. Unlike in usability tests, design or development staff are perfectly OK here, and key users would be ideal. CW is a professional activity that needs a little explanation, so it’s best to meet inspectors in person or in an online meeting. You will have to go over the user description (POV or persona), the tasks, the four CW questions, and at least one example. A brief introduction about how to annotate screens with Build will be beneficial; you can do that along with a brief (!) demo of the UI to be inspected.
    Novices in CW will struggle with understanding what a task step is, i.e., what exactly to apply the questions to. A task (e.g., filing a leave request) has subtasks (e.g., defining a time range) that consist of basic operations (e.g., selecting a start date in a date picker control). The beauty of CW is that it works on every level, so it’s no showstopper if you miss out on one. Encourage inspectors to switch levels every once in a while, and dig deeper in any area that feels suspicious to them. Often inspectors spot a problem outside of the procedure; the CW questions will help them understand it.
  6. Perform inspection.
    Inspectors receive an email with the link to the Build study. Each inspector goes through the tasks and screens, challenging them each with the 4 CW questions, and records in Build what issues they discovered (lengthy records are not needed – the task and UI context are clear from the Build environment). The study owner, that is you, will be able to monitor progress in the Feedback dashboard of the study.
  7. Consolidate results.
    In a meeting with inspectors (that can be held online, but screen sharing will be necessary), go through the Questions list of the study results. This list shows for each question (i.e., task) and screen the findings of all inspectors. It will be obvious where all agree on an issue. Typically, a lot of issues will be found only by one inspector, but all agree with their actually being an issue. If there is disagreement, guide the discussion towards what further research might clarify whether or not the issue is valid. If the group decides an item should go, inspectors can delete it from their findings. Note that each inspector can only delete their own findings.
    In classic CW sessions, it often is a challenge to keep the team from starting to design and losing themselves in debate. Luckily, the Build feedback study UI does not support this, so you have good reason to take those discussions offline, where they belong.
  8. Communicate results.
    The fastest way to a presentable result is to clean up duplicate or obsolete findings right in the consolidation meeting. Then, you can simply invite the addressees of the results as team members to the Build project, so they have access to the results. However, mind that communicating usability issues is an art form in itself, where social skills will help you more than any tool ever can. Meeting stakeholders in person, in particular when discussiong UI design issues, is invaluable.

Before You Go

CW is an inspection technique, that is, it cannot replace true end user contact. Even end users who have worked with a development team for some time may have developed a vocabulary that ever so slightly drifted away from ordinary users’ language. Research has shown that inspection techiques and usability tests uncover different subsets of usability problems in a UI and are best combined.  So, not testing the UI with end users would be a serious mistake!

 

 

 

Copy/Paste Question Template

BUILD is made for quick feedback rounds with end users, where typically no formatting of questions is required. In CW, we need to separate the task description from the four CW questions. The question template below does exactly that – just copy the text and paste it into the question field.


TASK: <insert task description here, e.g. Take 2 days off starting tomorrow.>

__________________________________

1) Will the user try to achieve the right effect?

__________________________________

2) Will they notice the right action is available?

__________________________________

3) Will the user associate this action with what they try to achieve?

__________________________________

4) Once the action is performed, how will the user know they made progress?


 

Not logged in