How to improve user engagement in interviews

Most of the time, users' participation in our interviews and tests is a passive behavior - the users we invite are not those who find problems in the process of using the product and want to actively give us feedback. When participating passively, users' enthusiasm, initiative and readiness for the interview topic may not be very high. They may come for various reasons such as compensation and curiosity. Therefore, in qualitative research projects, we often encounter a group of users whose interview results are uneven.

If the interview topic is to understand the user's objective behavior, attitudes and opinions, which are strongly related to the specific product, the differences in results between users will be smaller, and they will be less troubled by the problem of low "user quality"; and when the interview questions tend to be sensitive and brain-burning, the differences in the effects of interviews between individual users will be magnified.

I paid attention to this issue because I have recently experienced several projects that required me to "have a good chat about money" with users, such as understanding the income and expenditure of a family (where the money comes from, how much is spent on different places), the business/capital situation of a merchant (incoming and outgoing goods, capital flow, loans, etc. through different channels). In these topics, we can have a happy chat with some users and get the information we want naturally, while some users are very wary and are reluctant to talk more.

Indeed, the more difficult and sensitive the questions are, the more "selective" they will be. Therefore, in order to achieve better interview results, we will initially ask suppliers to help us screen out the types of users who are "good at chatting, have good expressive skills, and are willing to share" when inviting users.

We hoped to find higher-quality users to improve the effectiveness of the interviews, but we soon discovered that this trick was actually useless: first, it is not easy to find people who meet the requirements for some projects, and there is no room for screening; second, users' performance when faced with different chat content is not the same, and willingness to share is a subjective description that is difficult to set standards for - users may behave like they are good at chatting when communicating with suppliers, but will have a certain resistance when talking about sensitive information, and their cooperation will drop a lot - this is human nature.

Therefore, it is far from enough to conduct a survey simply by recruiting users with specific "dominant personalities" during the user invitation stage.

Of course, another way is to improve your interviewing skills, to the point where you can chat with anyone and anything, which can also solve this problem.

However, improving interviewing skills is not something that can be done overnight, and the project will not be suspended for a day. Therefore, what this article wants to share is that in addition to this - not relying on the luck of recruiting "good users" or the strength of skilled techniques - can we use some other means to slightly improve some of the issues that users are reluctant to talk about or say more during interviews, so that users can be less guarded and more honest and trusting when facing our curiosity.

Let’s talk about it in chronological order of our contact with users, and see which areas deserve more attention from “recruitment - pre-interview - interview execution”.

1. Recruitment: Provide interview instructions to eliminate information gaps

The first thing we should believe is that users who are willing to spend more than an hour participating in the interview are prepared and sincere in sharing.

It’s just that some people are happy to share all information, while others are happy to share on specific topics. This is related to the user’s expectations for participating in the interview. However, in the process of recruiting invitations, we did little to control user expectations. Ordinary users will not know in advance what issues will be discussed in the interview (in actual operation, we only write an additional outline of questions for them when interviewing experts). They may usually get a general direction from the supplier, such as talking about live shopping, payment tools, and insurance purchases. Therefore, it is normal for users to resist when they are suddenly asked about family income, financial management amount, etc. during the interview.

From this perspective, we need to prepare an interview description for the interviewed users during the invitation phase, just like we prepare for the experts in the expert interview, so as to briefly explain to the users what the purpose of our interview is, what types of questions will be asked, and how each type of question can help our products or research. This is the first step to contact the users before the formal interview, and this work should not be left to the supplier to complete.

2. Before the interview: seize the opportunity for informal communication; give users the right to refuse

Another communication opportunity before the formal interview is when the user arrives and we go to meet him.

I strongly recommend that the interview host go downstairs to pick up the user instead of sitting in the room and waiting. During the time from meeting to entering the experience room, you can chat with him casually to make the user relax and get to know each other. Because the most important relationship in the experience room is the relationship between the host and the user, you should make good use of this informal occasion. Don't wait until the user has sat down opposite you to "create a relaxed and friendly interview atmosphere", because the most natural opportunity has passed.

Before the interview, we should also formally inform users of their "right to refuse to answer questions" to give them a sense of control. Usually, users in in-depth interviews are in a one-to-many situation, and it is easy for them to feel defensive when facing the host, recorder, and observers. Let users know that they can refuse to answer any question in the interview, rather than just giving them a promise like "Don't worry, I will keep it confidential." The freedom to not speak will make them more relaxed in a one-to-many environment.

We also don’t have to worry about users making up stories to avoid questions they don’t want to answer, which improves the quality of the data. If necessary, such reminders should be given multiple times to avoid getting wrong data and causing misjudgment.

An example I encountered recently is that when conducting a company situation research in a certain industry, we need to ask the respondents about their company's team size, number of projects, operating income, etc. Some respondents will be very nervous, worried that providing such information in a private survey will leak the company's sensitive data. Therefore, we have repeatedly told the respondents that they can refuse if they don’t know or don’t want to answer. In the end, we didn’t receive data for every question, but at least we can ensure the authenticity of the existing data.

3. During the interview: let users understand the significance of the problem; establish user sense of value

It is also very important to let users know why we are asking this question during the interview.

Questions that users answer without hesitation often have the characteristics of being "strongly related to the product", including usage behavior, opinions and attitudes, pain points and needs, etc., because users understand the meaning of these questions and know that their answers will help the business understand the users of the product and improve the product.

However, we are usually concerned about some topics that are "weakly related or even unrelated to the product" during research. For example, when studying how small and micro businesses use WeChat Pay, a large part of our questions are about operations, such as how to purchase goods and how to sell them. When studying how users buy insurance, we will ask about the source of family assets, how to distribute them, and plans for the next three to five years, etc.

These questions are more about helping us understand users as a whole, understanding the environment they are in, the context of their decisions, or finding more possibilities for the business. But for users, the necessity of these questions is not obvious, and the values ​​that are self-evident to us are far from the point of view for users.

Therefore, many times, users resist sensitive questions not because of the sensitive questions themselves, but because they do not know the significance of the questions.

A good example is that in a study of WeChat payment, we wanted to understand the impact of WeChat's annual limit on small merchants. Users would open their WeChat bills and sort out their monthly cash inflows and outflows with us. Once we calculated the accounts, we found that many small merchants would use up their annual limit in the first six months or even the first two months. One important reason why users are willing to share their bill details with us is that they know that this information is directly helpful for product improvement.

In fact, all questions that are strongly related to the product have this advantage: users can understand why we ask and how the information will be used without us saying much. Because the purpose of the information is clear, users' insecurity about sharing private information is reduced, or in other words, users are willing to give up some privacy to help the product improve the experience.

Therefore, when questions that are "weakly related to the product" appear in the interview, we need to deliberately add a step of "letting users understand the meaning of the question". This also falls into the category of giving users positive feedback to encourage them to express themselves. There can be different designs for how to implement this step. Ideally, all privacy issues can be tied to product demands or solutions, and do not ask about privacy without considering the product. If there is really no specific product idea, then tell the user bluntly, "We discuss this issue to find out xxx and see if we can find some new possibilities for the product."

When interview questions have clear meaning, users are very willing to share even if it involves privacy.

We often say that we need to build trust with users during interviews in order to obtain high-quality communication to conduct research. In the past, we talked more about building emotional connections such as respect, empathy, and acceptance to generate trust, but I want to emphasize that we should pay more attention to establishing users' sense of value during interviews.

In short, users should feel that what they say, do, and spend time on are valuable and meaningful. Being driven by a sense of value and being driven by emotions can both make users more willing to express themselves and provide more information, but the difficulty of establishing a user's sense of value and establishing an emotional connection is completely different. Establishing an emotional connection is limited by the topic and duration of the interview. It is difficult to say how deep an emotional connection can be established. For example, in the example of the industry research mentioned above, there is no room for emotion to play, and it is strange to find points of empathy. In comparison, it is much easier to let users establish a sense of value.

Jesse Schell mentioned the concept of essential experience in his book “The Art of Game Design”, which means that we need to grasp “which experiences are essential in the process of experiencing something”. I think for the users participating in the interview, the valuable and meaningful experience is the core experience.

In summary, during interviews we can improve the quality of communication through some routine operations or attention to details. These things have nothing to do with interview skills or the quality of users, and can be tried in research on any topic.

Author: lotusluo

Source: Tencent CDC Experience Design (ID: tx_cdc)