This is the second of two posts. In the first post, I summarized the advantages and disadvantages of remote user research sessions. In this post, I’ll give some tips and tricks about what you have to consider before, during, and after the sessions. This is kind of long, but if you are going to conduct a remote session, you might really want to hunker down with a pot of coffee and read through this or print it out for later reference.
In case you’re already planning your remote sessions and are looking for an overview of the most important tasks, take a look at the post Checklist for Remote Usability Tests.
PREPARING THE REMOTE SESSIONS
When preparing for remote user research activities, make sure to acquire all the knowledge that you need for conducting such activities ahead of time. For example, if you plan to conduct an interview remotely, try to get your hand on all the information you can get about good interviewing techniques. If you plan a remote usability test, find out everything there is to know about conducting usability tests in general. Next, try to have all the material and documents you need for the session ready about 1 week ahead of the session, for example, an interview guide, a task script, a prototype, or a test system with test data.
You will need a quiet environment for conducting the remote session in order to be able to focus on what the participant says. Therefore, an open-plan office might not be the best choice. If you have the chance, conduct the session in a separate meeting room with a phone conference facility. The meeting room should have space for all the colleagues from the project team who want to join you in the meeting room during the session. Don’t forget to book the meeting room not only for the timeframe of the session, but also 30 minutes before the start and 30 minutes after the end of the session. You will need the extra time before the session for setting up your equipment. The extra time after the session is needed, for example, for the following activities:
- In case the session takes longer and the participant has agreed to stay over time.
- If team members have joined you in the room, it’s nice to sit together for a while right after the session and exchange first impressions.
- You might need some time for cleaning up the room.
- Remote connections tend to be more stable if you have access to a reliable internet connection, preferably a LAN connection (no wireless). If you have doubts about the booked meeting room having a functioning LAN cable, take one with you to the session.
- You need access to a hands-free telephone to be able to interact with the mouse while also handling the print-outs of your documents (task script, etc.).
- In case you want to record the session, make sure you take a recording device with you to the session or have a recording software installed on your computer.
- If you plan to use 2 monitors, you can share one with the participant and use the other one for communicating with your team members via, e.g., a messaging tool. This way the team can post questions that everyone except the participant can see and that the moderator can pick up when it suits best during the session.
- For the type of research activity you are going to conduct remotely, you may need other or special equipment, e.g. a monitor with a certain resolution. If this is the case, try to supply this equipment at least 2-3 weeks prior to the start of the remote sessions in order to be able to test all equipment and conduct a dry-run.
For any user research activity it is helpful to conduct a dry run prior to conducting the actual activity in order to test the equipment, the application/prototype, the task scripts and interview guides, and to practice the procedure. This holds true even more so for remote research activities since a dry run gives you the chance to also test the settings of your online conferencing tool and connection, and to find out which part of the session the moderator needs to explain in more detail. If you are going to share documents with the participants, particularly mockups and prototypes, it is crucial that the documents are displayed clearly for the participants and that everything is legible. To this end, it’s advisable to test how the colors and texts of the documents you want to share are displayed in a dry-run with a colleague. I recommend that you conduct a dry-run of the study in its entirety, including all of the required software, about 2 weeks prior to the start of your actual remote sessions. This way you have enough time to adjust settings, the interview guide or anything else that didn’t work as planned during the dry-run.
Communication to Participants
The upfront communication to the participant is crucial, not only for already building trust and rapport, but especially to provide him or her with all the required information. It is recommended to have at least one call in advance with each participant in order to prepare him/her for the test session and allow for questions regarding technical needs and set-up of the whole test. This also gives you the chance to test the language proficiency of the participant (in case the test is not conducted in the native language of the participant) and to make sure that the participant matches the profile.
Communicate to the participant in advance what he or she needs:
- A separate conference room to avoid disturbance during the session
- A LAN line connection (no wireless)
- Ideally, a hands free phone
- A print-out of the task script if you’re going to conduct a usability test based on a task script
Send to the participant the following in advance:
- Task script Each task should be on an individual page, as this allows you to keep control of when the user sees each task.
- Access to your online conferencing tool Send the participant information about how he or she can access your online conferencing tool, e.g., a link with dial-in numbers for their region, and a description of the steps.
- Legal documents If there are any legal documents that need to be signed by the participant, such as a Non-Disclosure Agreement, send these documents to the participant ahead of time and tell him or her by when you need them signed and returned.
- Post-Test Questionnaire If you conduct a formative usability test, you will probably ask the participant at the end of the session to fill out a post-test questionnaire about the application he or she has just tested. You can send this questionnaire to the participant before the session and explain that you will ask him or her at the end of the session to fill it out (that is, the participant does not have to print out the questionnaire). The participant opening the questionnaire on his or her PC ensures that your colleagues who joined the session in your online conferencing tool cannot observe the participant fill out the questionnaire, thus the participant has more privacy. Ask the participant to send you the filled-out questionnaire after the session.
Communication to the Project Team
As some project teams do not have a lot of experience with remote user research sessions, make sure to brief your team well ahead the start of the sessions. Tell the team everything they need to know about the nature of the remote user research sessions, for example, what the procedure is during the session and how the observers and note takers should conduct.
SHORTLY BEFORE THE SESSION
I recommend that the moderator arrives 30 minutes before the start of the remote session at the meeting room. The moderator will need some time to set up the technical devices, to sort through the papers (e.g., interview guides), and to access the online conferencing tool. The observers and note takers can arrive to the meeting room 15 minutes prior to the session start, or can join a couple of minutes before start if they join online. Here are some more recommendations for the moderator:
- If you are using a meeting room, install a sign on the door indicating that a live test session is running.
- Access your online conferencing tool and dial-in about 15 minutes before the session starts. This allows you to reconnect in case you have a bad connection for the first try.
DURING THE SESSION
Once the participant has joined the session, the focus lies on welcoming the participant and making him or her feel at ease. In the following you can find some tips and tricks for moderating and note taking during the session.
- The audio quality is very important, both for your project team and the participant. Ask the participant at the beginning of the session if the audio quality is fine for him or her.
- If you know that connectivity issues with your online conferencing tool can happen (e.g., the participant can get thrown out in the middle of the session), it’s advisable that you mention possible issues at the start of the session and explain how the participant can react in such a situation.
- If there is anything that you want to show to the participant or that the participant has to interact with (e.g., a prototype), share the document in the online conferencing tool, and ask the participant if everything is displayed clearly.
- If you want to get feedback to anything included in a presentation slide deck and you give the participant control over the slides, make the mouse pointer visible after starting the presentation mode so that you see the participant’s mouse movement.
- If the time allows at the end of the session, you can ask your colleagues in the room or on the phone who joined the session if they have any further questions.
- Ideally, the session ends at the scheduled ending time. In case you’re running out of time, ask the participant if he or she has some extra time and if it is ok to prolong the session by 15-30 minutes (try to estimate the extra time you will need to finish the session).
- At the end of the session, wrap up, thank the participant, and explain any further steps or ask for participation in upcoming or follow-up user research activities.
Note Taking Tips
- If the note takers use their laptops for taking notes and are located in the same room as the moderator, I recommend that the note takers do not sit too close to the microphone or soundstation so that the participant does not get distracted by the typing noise.
- If several colleagues of the project team are trained in taking notes, they may want to take turns. This way, the note taking task and especially the time-consuming task of digitalizing the notes after the session is spread across several team members.
AFTER THE SESSION
Here are some tips for the action items after the remote sessions, which can be easily forgotten when the project team gets busy analyzing the data:
- Each participant should get a Thank-you email in a timely manner. If the participant filled out a post-test questionnaire at the end of the session and has not sent you the questionnaire yet, you can ask him or her in the email to do so.
- If you have a gift for the participant, send the gift after the session.
DIFFERENT TYPES OF REMOTE SESSIONS
In this section you will find a couple of different types of remote sessions that I and colleagues of mine have conducted and the experiences we have gained.
We have made good experiences with acquiring knowledge about high-level business processes through remote interviews. Usually, it is a customer (be it an IT specialist or a business process expert, that is a person managing end users) and not an end user who provides this type of information. In such an interview context information from the working place of the participant is of minor importance and the participant often has little system interaction to show you, therefore this interview is well suited for being conducted remotely. If you want to show some slides at the beginning of the interview, e.g., for setting the stage or informing the participant about the background of the project, try to use as few slides as possible. This way you will have more time for the feedback from the participant; also, the participant is not overburdened with information he or she might not need for the interview.
If you are conducting a formative usability test, send the participant the task script in advance so that he or she can print it out (otherwise the participant would have to switch back and forth on the PC between the prototype and the online task script). When you send the task script, tell the participant that he or she should not familiarize with the tasks; it also helps to send the task script only 1-2 hours prior to the session so that the participant has on the one hand enough time to print out the document but on the other hand not too much time to familiarize with it.
You can run remote validations not only on paper-mockups and high-fidelity prototypes, but also on concepts, story boards, user profiles, process maps, mind maps, use cases, and so on. Here are our recommendations for dealing with validations:
- Clarify the purpose of the validation before the session with the participants so that they don’t expect, for example, a formative usability test. This, of course, holds true for both in-person and remote validations.
- As far as story board validations are concerned, it helps to create an overview story as introduction to the design comics as well as to the high-level scenario. This again holds true for both in-person and remote validations.
A recommendation referring only to remote validations is to send the participants the document that you want to validate before the session, so that they can print it out. It is easier for the participants to conduct the validation with a print-out version of the document.
Although this post is kind of long, I’m sure that there are loads of tips and tricks that I haven’t thought of. And there are surely also plenty of other types of user research activities that can be conducted remotely that I haven’t covered here. So, if you have any experience in this domain and some good tips at hand, I’d be more than happy if you shared your knowledge.