Just recently we had a talk with a potential customer about a possible approach to a project involving user research, conceptual and visual design and development. Whereas the steps concerning design and development had been talked over very quickly leaving everybody at the table leaning comfortably in their seats, the first and foremost step, user research, stirred up a conversation about several possible methods. Our customer insisted on conducting a survey aiming at producing a prioritized feature list. This feature list he argued can afterwards be used to make sure that the most important features are represented in the final product. Whereas it is perfectly fine to prioritize features, it was not the time yet for a survey, since the customer had not exceeded the phase of merely gathering feature ideas. These first ideas, however, were not precise enough to be put into a questionnaire leaving plenty of space for interpretation and would ultimately lead to biased and thus invalid data. Furthermore, asked for the target groups and potential end users of the product, the customer only had a vague idea of who could be using the product once it will be finished.
So, there we were with a customer insisting on a method inappropriate for the premature state of the project trying to convince him to go with methods more fit to state of the product. After all, it was all about the difference between qualitative and quantitative methods in the user-centered design of interactive systems, so we explained him the differences of the two methodical directions and the ideal time of their deployment.
Qualitative methods are methods that aim at exploring new things (therefore they are sometimes called “explorative”) and allow deep insights into certain topics. The most popular qualitative methods are the qualitative interview, participative observations and group discussions. All these methods have one major thing in common: although roughly structured by an interview guide, an observation template or a moderator, they leave plenty of room for the subjects to freely articulate their attitudes, their motivations and situational behaviors. Therefore, the application of qualitative methods leads to insights that help researchers and designers understand potential users and they may even derive needs that had been hidden before. According to EN ISO 9241-210 (Human-centered design for interactive systems) one of the first steps is to understand the user and the context of use. Due to the open nature of qualitative methods, it is these methods that are typically applied in this first step.
Quantitative methods are methods that aim at describing actual states with the help of reliable numbers and distributed percentages (therefore they are sometimes called “descriptive”). The most typical quantitative method is a survey that normally encompasses the creation of a questionnaire that a random sample of a certain population fills out. This questionnaire is typically composed of closed questions in order to make sure the comparability of the answers (closed questions are those which can be answered by ticking off a box). However, the prerequisite for the execution of such a survey is the precise knowledge of the topics covered in the questionnaire. This means, the questions should be very precise, leaving no room for subjective interpretation and also all possible answers should be selectable in order to avoid having the subjects answer with “other”. The results of surveys are statistics that help researchers to determine tendencies and attitudes in a population at a given point in time; however, these statistics do not elicit why people have certain attitudes or show certain behaviors.
All in good time – when to use what
Ideally both methodical approaches should be used in combination. Qualitative methods would in such a setting find out which needs people actually have concerning a certain topic or feature; quantitative methods would support and safeguard these findings. In the context of user-centered software design, qualitative methods have proven to be the first approach to be chosen since they not only help understand the user, but also elicit further ideas and features that could make a product the cutting edge. In the IT-industry that quickly incorporates numerous new trends due to rapid technological changes and new technological opportunities, products need to be user-friendly and innovative in order to set themselves apart from competitors and succeed. However, qualitative methods may also be used in later project stages when it comes to the validation of processes, concepts and designs. Quantitative methods also play an important role in user-centered software design. Once several designs have been created, these designs could also be evaluated with the help of a survey bearing results such as “82,3% of the users preferred design number 2 over 1”. Another quantitative method in user-centered software design are A/B-tests that elicit the general handling of different versions of software (mostly websites). On the basis of the findings from such quantitative methods, stakeholders are enabled to make methodically informed decisions about the design of the software. All in all, in most cases it is worthwhile to start off with qualitative and afterwards confirm your findings with quantitative methods.
Wait, what did the customer say?
The challenge many dedicated UX professionals face with regard to the planning of methodological proceedings is that plenty of decision-makers and stakeholders rely on numbers and hard facts (which is absolutely understandable given the far-ranging decisions they have to make). Therefore, they are hardly receptive to methods giving insights into “soft facts” like needs, feelings and behavioral patterns. Although this is slightly changing as more and more gate keepers understand that the first step to innovative products is empathy for the user, it is our task as UX professionals to inform about and explain the difference and relevance of quantitative and qualitative approaches if necessary. Fortunately, in the end, we were able to convince our customer to go with qualitative interviews and execute a survey at a later project stage.