How can user operations accurately issue coupons by building an uplift model?

Issuing coupons is a precision marketing process. Using the uplift model to calculate the user's gain or profit helps us maximize incremental value and avoid blindly issuing coupons.

As an operator, when sending coupons to users, do you often face these two confusions:

Confusion 1: Large-scale coupon issuance leads to high marketing costs and low usage rates. Confusion 2: The usage rate of coupons is very high, but the incremental performance brought by coupons is very small and the ROI is low. This situation is often caused by issuing coupons to active users, who are likely to place orders even if coupons are not issued.

Let's explore how to solve this problem through actual cases.

Background: Send recall SMS to recall lost users. If there are 1 million users in the lost pool and we only have a budget of 100,000 SMS messages, how can we select these 100,000 users to maximize profits?

Solution:

1. Determine the recall of interest points. If the coupons are issued for 300 points or more, 50 points will be deducted.

2. Randomly select 5,000 users in the experimental group and the control group. The experimental group will receive coupons, while the control group will not receive coupons.

3. Recycle results within a conversion cycle and mark used and unused results

4. Build an uplift differential model and use the experimental group and the control group to build a regression binary response model

5. The marketing users are input into the model to calculate the uplift score, and the remaining 90,000 users are selected in descending order of uplift score to issue coupons

6. Compare the model’s targeted user usage gain value to prove the value of precision marketing

1. First, let’s introduce the uplift model principle

Uplift Modeling is a model used to estimate the Individual Treatment Effect (ITE), that is, the effect of the treatment on the user's response. The basic idea of ​​Uplift Gain Experiment is to establish a prediction model to estimate the response probability of each user with and without intervention, and then calculate the difference between the two, that is, the gain value.

The principle logic is very simple. To put it simply, it is to predict the user's repurchase rate under two circumstances: with and without coupon issuance. The difference is the gain value of the intervention. This operation can eliminate those users who will return even if coupons are not issued.

Secondly, can operations students build this kind of prediction model?

The problem faced by many enterprises is that a group of engineers are immersed in studying these so-called big data models, but these people are out of touch with the business. The essence of the prediction model is whether the user feature engineering can effectively represent the user's intention. The user features that engineers can think of are nothing more than user browsing behavior, such as the number of times the APP is opened, the browsing time, and the number of pages viewed; user purchasing behavior, such as adding to cart, favorites, purchases, consumption frequency, consumption interval time, etc. Can these features effectively predict a user's purchasing behavior? Obviously not.

For example, this set of data modeling

Modeling based solely on these shallow user behavior data will produce purely speculative results. From a data logic perspective, the more items a user collects and adds to cart, the higher the probability that the user will make a purchase. However, from a user's real-world perspective, whether a user makes a purchase is affected by multiple factors, such as price comparisons on competing platforms and whether the user is in a good mood. These characteristics are precisely what operations students, who are closest to the business and users, need to explore.

I believe that many operations students do not have the ability to mine and model data. The goal of this article is to simply introduce how operations students can use tools to implement big data mining and modeling, so as to further enhance their own value.

2. Let’s take a look at the practical construction of the model

Step 1: User feature engineering

In addition to the characteristics of user browsing behavior, consumption behavior, and preference behavior, it is more important to explore the user's trend behavior, such as the trend of consumption interval period, the number of consumption categories, the consumption amount, and so on. If a user's consumption categories change from diverse to single, or the distribution of consumer unit price changes from low to high or from high to low, or the consumption frequency changes from stable to random or from random to stable, these can all predict a user's next consumption behavior.

For example, a user on an e-commerce platform consumed products including 3C digital products, fresh food, and daily necessities in the last consumption cycle. In this cycle, the only consumption category is 3C. Moreover, the overall average order value has increased from a few hundred to thousands, and the repurchase cycle trend has increased from 30 days to 60 days. What is the probability that this user will continue to purchase fresh food?

By mining user behavior trends and combining them with AB experiment response data, a prediction model can be built.

Step 2: Build a regression response model

Uplift needs to build response models for the strategy group and the control group respectively. Finally, the target users enter the two models respectively to calculate the uplift score.

With the help of analytical mining tools, we build the above data flow, and users calculate the response rate under intervention and non-intervention conditions

First, let’s look at the prediction accuracy of the model.

70% of the modeling data is used for training and 30% for testing. The prediction accuracy of the training set is 79.53%, and the AUC value reaches 0.848; the prediction accuracy of the test set is 69.7%, and the AUC value is 0.744. The model prediction performance is acceptable.

Step 3: How to issue coupons accurately

The uplift response model can help us generate several types of prediction graphs, namely response graphs, gain graphs, and profit graphs. We can use these graphs to guide the marketing actions of coupons.

1) Select users to issue coupons based on the response graph results

The response graph shows the proportion of actual responses (such as coupon usage) among users in different percentiles. In a coupon campaign, the response graph can show what proportion of users in each percentile group actually used the coupon.

According to the cumulative response graph above, we can see that the curve gradually decreases from close to 100% and eventually tends to the 50% horizontal line. This shows that on the left side of the graph, the response rate predicted by the model is very high, and as you move to the right (i.e., cover more users), the response rate gradually decreases. In order to determine how many users to issue coupons to, several factors need to be considered:

1. Overall response rate: This is the average percentage of all users who responded to the coupon. In the response graph, this value appears to be 50%.
2. Target response rate: The minimum response rate you want to achieve by issuing coupons. This is usually higher than the overall response rate.
3. Cost and Benefit: The cost and expected benefit of issuing coupons. We need to find a balance point that maximizes the benefits while keeping the costs within an acceptable range. On the cumulative response graph, you can find a point where the response rate meets our target response rate. Starting from the left side of the graph, move to the right along the curve until you find a point whose response rate is equal to or slightly higher than our target response rate. The percentile corresponding to this point is how many users you need to issue coupons to. For example, if our target response rate is 70%, we need to find a point on the curve with a response rate of 70%. Suppose the percentile corresponding to this point is 20%, which means we should issue coupons to the first 20% of users.

2) Issue coupons based on gain graph

Gain plots show the relative gain of the model's predicted response compared to random selection. In a coupon campaign, the gain plot can indicate the additional sales boost caused by the coupon among a certain percentile of users.

In the figure, the gain value is 1.5914, which generally means that the model's prediction is 1.5914 times better than random selection. Specifically, if the response rate of random selection is a fixed value, then the response rate after using the model prediction is 1.5914 times this fixed value. This gain value is often used to evaluate the performance of a predictive model at a specific percentile, especially in marketing campaigns to determine which customer groups are most likely to respond to promotions. In practical applications, if the gain value is greater than 1, it means that the model's predictive ability is better than random selection. In the gain graph, the gain value of 1.5914 at percentile 46 shows that among the first 46% of users predicted by the model as most likely to respond, the response rate is 1.5914 times the response rate of randomly selected users, which is a relatively high improvement, indicating that the model's predictions at this percentile are effective.

3) Issue coupons based on profit graph

The profit graph shows the total profit gained by using coupons among different percentiles of users. For coupon campaigns, the profit graph can help us evaluate the overall financial benefits of distributing coupons among different user groups.

The parameter settings are as follows:

• Cost: The cost of issuing each coupon is 5 yuan.
• Revenue: The revenue for each successful response (customer using the coupon) is 10 yuan.
• Width: The width is set to 1%, which means that each data point on the profit graph represents 1% of the user population.

This provides a granular view of how profits change for every 1% increase in users. Percentile 46%, profit 630: In the figure, when the percentile of the target user group reaches 46%, the cumulative profit reaches 630 yuan. This means that if coupons are issued to the top 46% of users most likely to respond in the order predicted by the model, the expected total profit will be 630 yuan.

Author: Zhao Wenbiao, WeChat public account: User Operation Observation (ID: yunyingguancha)