A usability test is a method for validating prototypes and designs with actual end-users.
In simple terms, a usability test is a one-on-one structured session between a moderator and a test participant as the participant performs a set of predefined tasks with a prototype or system. It does not involve a group discussion or observations by a group of people. The sessions usually last 60 to 90 minutes; therefore, it is not usually possible to test the entire solution. So the features and functions validated during the sessions need to be chosen wisely.
Usability tests can vary widely in structure, but all tests share five key elements:
- The main objective is to improve the usability of a product or an application.
- The participants are real end-users, who would in real life actually use the product or application.
- The tasks participants should complete are real tasks and are representative of the ones users would perform in their daily work.
- During the usability test, the moderator and/or note taker observe and record what the participant does while performing a task.
- The testing team analyzes the findings of the usability tests after the test. Usability issues are identified and the testing team makes recommendations on how to resolve the issues.
Two kinds of usability test should be distinguished:
Formative usability tests and summative usability tests : “formative” – meaning “forming” design; as opposed to “summative” – “summarizing” data.
The formative usability test is conducted in an early phase of the design process. The focus is on the qualitative feedback for the solution. The predefined tasks are performed on high- or low-fidelity prototypes, with the goal of identifying usability issues and improving the design to support design decisions. Participants are asked to “think aloud” as they perform the tasks, so the moderator and observers will know what is going on in the participant’s mind. The results of these usability tests are recorded in a list of usability issues or problems that are then classified and prioritized. Recommendations for resolving the problems are also made.
In a formative usability test, feedback from 6 to 10 participants is gathered; with this number, about 80% of the usability issues can be discovered.
The summative usability test is conducted after the system is released to the customer. The focus is on collecting quantitative data for the product or application. In this form of testing the predefined tasks are performed with a running system. The goal is to identify a usability KPI to either compare different releases of the same product or compare different products. Measurements like error-rate, time used for task completion, and satisfaction of the participant are taken. This kind of usability testing will not give an insight into what needs to be improved, but instead where the product stands in comparison to other products.
In a summative usability test, you typically gather the feedback from 16 to 20 participants in order to have statistically relevant statements.