User portraits, detailing my countless failed experiences

This article deeply analyzes the cases of failed user portraits, reveals the truth about the disconnect between data and business decisions, and provides practical guidance for data analysts and business strategists.

A student asked: "Teacher Chen, do you have any examples of failed user portrait projects? Share them." Answer: I have been working in data for so long and have followed so many projects. The number of failed user portrait projects is simply too numerous to mention! I will share them today.

1. Signs of User Profiling Failure

Do you often wonder: "What's the use of user portraits?" If so, congratulations, your project has failed, it's that simple! Of course, more often, when you first start working on user portraits, the business department shakes their heads and says: "We need to understand users in detail and in depth based on user portraits, such as user gender, age, region, preferences, consumption habits, ... so that we can make refined decisions." Then the data department worked hard for several months, tagged 30,000 users, and proudly reported to the leader: "We have made great progress in the construction of user portrait big data." Then at the first project report meeting, the data department proudly said:

• The male to female ratio of our users is 6:4
• South China accounts for 30%, East China 25%
• Purchase of product A accounts for 50%
• The business department rolled their eyes:
• I knew it
• Our users are all like this
• What's the point of you doing this?

Of course, there are even worse situations, that is, you put a label of "loyal user", and the business side said: Oh, since you are so loyal, don't do anything, and then you don't make any purchases or log in next month! You put a label of "A product lover", the business side promoted product A, but you didn't buy it! The business side came to settle the account angrily: "This user portrait is not accurate at all!" So the project was completely abandoned. Complaints are complaints, but what is the problem?

2. Surface reasons for failure of user portrait projects

1. Reason 1: Confusing the past and the future

Question 1: A user bought an apple yesterday, the day before yesterday, and the day before yesterday. Will he buy an apple today?

Question 2: A user bought soy sauce, chicken wings, and cola. Does he also need to buy bamboo sticks for barbecue?

Think about it for a second. You don’t need to think about it for a second. Everyone knows the answer: not necessarily, not necessarily, not necessarily. Buying apples continuously may mean that he likes to eat apples, or he may have bought a lot so he doesn’t buy anymore. Buying soy sauce + chicken wings + cola may mean that he is going to barbecue or making cola chicken wings. Past behavior does not equal future behavior. Future behavior needs to be predicted. Whether the prediction method is based on business logic reasoning or algorithm model calculation, it needs to be verified by data analysis and experiments.

Only prediction methods with stable performance can be adopted. However, when doing user profiling, the business side often confuses these two points. They often put a lot of labels on past behaviors, have no idea about future predictions, and do not invest in predictive analysis. When reading user profiling reports or setting push rules in CDP, they take it for granted that if someone bought in the past, they will buy in the future. In the end, the prediction is inaccurate, but the blame is placed on the user profiling system. The result is naturally tragic.

2. Reason 2: Confusing Behavior and Motivation

Let me ask you a simple question: If a user came to our store to buy products on one day in the past 30 days, is this user a fan of our products? What if he came to our store to buy products on two days, three days, four days... What if he came to our store to buy products on 30 days? If he came to our store to buy products every day in the past 30 days, he must be a fan, right? Answer: Not necessarily. If he came to our store to buy products every day in the past 30 days, you can call him a "high-frequency buyer" because his purchase frequency is indeed very high. But whether he really loves to use our products is not necessarily true, because you don't know whether he really loves to use them, or even whether he uses them or not.

Purchase frequency cannot be directly equated with user preference. Whether users like or dislike something requires analysis of data from more dimensions, and the analysis results must have a certain probability of stability before they can be called that. Similarly, in many companies, business parties and data analysts use terms such as "lovers" and so on very casually and roughly, basically using consumption amounts, login frequencies, etc. If it is high, it is considered "like" or "love to use", and if it is low, it is considered "marginal" or "tried". The results produced are naturally inaccurate. Needless to say, if there is a problem, such as no one buying the recommended product, it is blamed on the user portrait.

3. Reason 3: Confusing cause and effect

Q: Users who have spent more than 10,000 yuan have made purchases more than 5 times. So if we let users make purchases 5 times, their total spending will be 10,000 yuan, right?

Of course it is not right. However, the business side often does this! They take the past high-spending user behavior and apply it to the low-spending user, thinking that as long as the low-spending user simulates a certain number, he or she will become a high-spending user. There is no such thing as a "magic number". It is very likely that from the source, high-spending and low-spending users are two different types of people. We should conduct in-depth analysis to figure out what drives the behavior.

On the surface, the reasons for the failure of user profiling are: focusing on data and neglecting analysis. Too much energy is invested in detailing the behaviors that have already occurred, and too many factual labels are attached. Not enough investment is made in prediction, causal analysis is not enough, and insight into user needs is not enough. In the end, judgments are all based on business intuition.

If you ask him why he promotes products/activities based on these tags, his answers are:

• I think he has bought it so many times, he will definitely buy it this time
• I think he has bought related products before, so he will definitely buy it this time.
• I think since he bought A, he must buy B

There is no essential difference between making decisions based on user portraits and looking at reports. It is easy to create a bunch of labels based on past data, but it is difficult to precipitate user labels that are predictive and accurate. It requires not only in-depth data analysis and modeling, but also repeated, multiple rounds of comparative testing. It is not something that can be achieved overnight. So when the business department thinks they know a lot and the data department happily announces that "30,000 labels have been added", the root of the problem has already been planted. However, given the same scenario of too high business expectations and insufficient data preparation, why are user portrait projects more likely to fail than data model projects?

3. The Deep Reasons for the Failure of User Portraits

Looking deeper, because data modeling is difficult, the business side cannot participate in the intermediate process and can only comment on the results. As long as the guys who do modeling do not commit suicide, do not work in isolation, and actively lower business expectations, they can avoid problems to a large extent. Therefore, the failure of modeling projects is basically the result of a blind man riding a blind horse. But the user portrait project is just the opposite: the business side thinks they know a lot!

The guy who does data also thinks he knows a lot! Almost all business parties will say this when talking about user portraits: "For example, if I know that the user is 24 years old and female, I will recommend an XX product to her." Everyone thinks: I know a lot, the only thing missing is a number! Give me a number quickly. So the business keeps urging the data to make the past data more and more detailed, while the data is running wild on the road of labeling. No one is doing the most important three-piece set of prediction, analysis, and experimentation.

Of course, this kind of labeling based on past data is useful for some departments - it is useful for support departments such as customer service, supply chain, and logistics. For example, a customer receives a complaint from a customer, "Why hasn't the after-sales technician come to the door yet!!!" If there is no labeling, the customer has to go through several tables to confirm: what product the customer bought, when, the product body number, when the technician was scheduled, and other details. The confirmation process alone makes the customer angry. With labels, the problem can be located in a few swipes, which can greatly improve the customer experience. But the tragedy is that this usefulness only makes operations, marketing, planning, design and other departments that need to use their brains, be creative, and think about strategies more conceited. It also strengthens their feeling of "I'm really awesome, I'm just one number away!"

So the tragedies kept coming. If modeling is like a blind man riding a blind horse, then the user portrait project is like riding an electric bike while rubbing a mobile phone and running a red light in the opposite direction - the electric bike thinks it is a car, and the rider thinks he rides really well. In order to avoid this kind of problem, I often use this trick. When the business party opens his mouth and says, "If I knew that the person was 24 years old and female, I would promote product A", I will directly check the database to find out how many 24-year-old women bought product A in the past month, and then confront the business party: "No need for user portraits, I will tell you now that the purchase rate is 12%.

Why do you need to upload user portraits? Just let your guys run the numbers according to the rules. At this time, any reliable business party will immediately realize the problem here and say: such a simple splicing is not enough. We need to analyze more based on fact labels. In this way, the project will be much more stable in the future.

However, please be careful when using this trick. Your corporate environment may not be suitable for this hard-line style. In short, everyone just needs to understand the key to the problem. The key to the problem is: the prediction ability of simple fact labels is too poor, and the insight is too poor. It is not enough to meet the needs of operations, planning, sales, and marketing. Large amounts of data + in-depth analysis are the solution to the problem.

Author: Down-to-earth Teacher Chen; Source public account: Down-to-earth Teacher Chen (ID: 773891)