[Special Topic on Overseas User Research] Top Ten Challenges Faced by Google User Researchers

In user research, researchers will also encounter some difficulties. This article summarizes the top ten challenges faced by Google user researchers. Let’s take a look.

Peron's core view:

1. Jess Nichols is a user researcher at Google. The top ten challenges for user researchers she proposed largely represent the difficulties of overseas user research, and these difficulties seem to be the same in China as well.

2. She mentioned research operations (this role is called supervisor or PM in the second party). At present, the domestic user research team of the first party lacks such a role (there may be a role like "team assistant"). The execution of many projects (such as user recruitment, questionnaire push, etc.) requires researchers to implement and manage by themselves. It is strongly recommended that teams with conditions set up a research operation role to provide execution support for researchers, so that user researchers can better focus on research.

Researchers are often very passionate about research work to produce product insights, but we also encounter many challenges when we try to conduct research.

1. Research is slow and expensive

User research as a separate team is relatively new compared to engineering, product, or design, and many stakeholders may not have worked with researchers in the past and therefore do not understand the value that researchers can bring.

Because some research methods take time to execute or require external vendor support to gain the best insights, partners may perceive research as slow or expensive, which can be a hindrance to product development or launch.

These issues can result in user researchers not being included early in product discussions, or not at all, limiting the ability of the analytics to provide strategic or directional support.

Or, it results in research not receiving enough budget to work effectively, forcing researchers to skimp, act rashly, and deprioritize research that could eat up a large portion of the budget.

2. Being in an island: A single-person research team may be isolated from other teams

When research is not valued, organizations won’t invest in a team. Many organizations will have just one researcher across the entire company or across multiple product areas, which forces researchers to prioritize their work and can frustrate other teams if they don’t get research support.

Researchers may also be embedded in different or independent product areas, making it difficult for researchers to collaborate or pair up with other researchers on projects. When researchers come together for reviews or to share feedback and experiences, they need to take the time to present their ongoing work to colleagues (especially in non-consumer-facing experiences) to ensure that colleagues can provide meaningful feedback to support their projects.

3. Research execution and results do not meet expectations

Sometimes, after researchers have spent time and effort creating a detailed research report, the report is not used. The reason for non-use is often due to a mismatch in expectations between the team and the researcher. Therefore, researchers need to ensure that stakeholders are involved in the research design process to ensure that the results achieved have stakeholder buy-in and support.

Sometimes researchers may “do research behind closed doors” – they may not invest the energy to create research outputs that resonate with stakeholders, nor do they take the time to engage in conversations and presentations with stakeholders, creating conversations that help stakeholders understand how the research can be used.

4. Research is not used

In other cases, product priorities may have changed, resulting in research findings not being integrated into the product and design. Research that ends up bearing fruit can make it difficult for researchers to have an impact on the team because their work has not resulted in changes in the product.

Researchers must find other ways to create value from their work. This value may be finding more opportunities to share insights outside of their immediate stakeholder groups, or sharing with their research teams where the findings have a chance to be used in related work.

5. Spending excessive energy researching previous work

Researchers may spend a great deal of time searching for past research or data to support a particular stakeholder or research project. Because researchers must quickly jump from project to project to ensure they continue to provide value, knowledge management tasks are often relegated to a low priority during the research process.

Researchers may actively try to follow up on knowledge management activities. Since each researcher may have a different approach to tagging and storing insights, it can be difficult for other researchers to find research if they don’t know the correct search keywords.

Regardless of the format in which the researcher presents it (e.g., presentation or report), it will be stored in the same format. Inconsistent storage formats can make it difficult for future researchers to parse insights, causing researchers to have to go through each subject's report one by one to determine if there are relevant insights.

7. Bias that hinders new research

Since many stakeholders may have specific knowledge of the field they work in, they may make assumptions about the customer or product, causing them to drive product decisions based on their own experience.

Although stakeholders may interact with customers on a daily basis, they are not customers. Their underlying biases and assumptions based on their own experiences may not always align with actual customer pain points and needs.

Researchers must find ways to strategically oppose these decisions to ensure that the research provides guidance or analysis and that client needs are clearly understood.

8. Single Insight

If the stakeholders are customer-facing, or if customers are involved in the conversation, they may receive customer feedback on the product or experience. Customers may also offer solutions (i.e., provide suggestions on how to fix the product) during these conversations.

If the customer is of high value, stakeholders are more likely to make reactive decisions based on customer insights or product recommendations.

Strong stakeholder perspectives can present challenges to researchers when they attempt to propose work that might cover similar topics, as stakeholders may firmly believe that the insights they have gained in customer conversations already override the need for additional research.

9. Spend more time on operations, less time on research

The workload of research operations activities often increases dramatically before and after study execution. When research teams are small or have limited funding, they are less likely to devote resources to staffing a dedicated research operations staff.

The responsibility for all organizational and operational activities falls on researchers, leaving them with less time to focus on ensuring high-quality research throughout the process. Improvements in research operations staff focused on planning, execution, and synthesis of research can provide researchers with long-term efficiency gains.

10. It takes a lot of work to recruit participants

One of the key responsibilities of research operations is managing the recruitment of participants. Without a research panel of customers who actively choose to participate in research (either internally or through a vendor), other methods need to be used to find the right target users.

Email open rates typically average between 15-25%, so if there is a niche participant type, it is more difficult to recruit enough relevant participants to match the required sample size. Additionally, some participant types may not be active online (e.g. truck drivers, agroforestry workers), which means calling them individually or having specific relationships to find participants.

Sometimes we also have suppliers to help with operations. Suppliers mainly support research in the following two aspects:

• Recruitment & Logistics: Managing participants, including recruitment, scheduling, and incentive management. Vendors can help when it is difficult to find participant requirements, non-customer/product users, or when the study is blinded. They may also rent out research labs for researchers to conduct offline studies.
• Research Execution: Conduct a complete research project, including planning, recruitment, execution, and comprehensive analysis. This is useful when the scope of the project is clear and has low ambiguity (e.g., competitive evaluation, usability testing).

In both cases, the vendor needs to go through a procurement process to agree on the work, costs, and expected outputs. For larger or enterprise-level organizations, there are often requirements related to security, privacy, and operational processes. In these cases, researchers/research operations staff must become the middleman, working with internal teams and vendors to prevent delays to project timelines.

If the organization has strict privacy or data protection processes, procurement may be further complicated because there are strict requirements for sharing data with external parties (for example, personally identifiable information is prohibited from being disclosed).

If data researchers need for recruitment are denied access due to company policy, researchers and vendors may need to develop workarounds, which could impact participant quality or study outcomes.

Factors such as research process, knowledge management, and team structure are all obstacles to user research work. These are similar both at home and abroad. The progress of researchers' work depends on the company structure, scale, and whether product leaders can have a say in obtaining more funds and resources to break these obstacles.

Original author: Jess Nichols; Translation: Huang Suchen; Source public account: Peron User Research (ID: 330829)