To analyze indicator fluctuations, the data model should be built like this

In the world of data analytics, fluctuations in business metrics often trigger a flurry of questions and urgent demands, requiring data teams to provide immediate explanations and insights. However, not all fluctuations are equally important, and not every change requires in-depth analysis.

When business metrics start to fluctuate, people always have questions:

“Why did it increase by 5%?”

“Why did it drop another 1%?”

“Why did it go up for 2 days and then fall again?”

"Why hasn't there been any change in three days?"

There are always a hundred thousand whys coming out of the mouths of various departments, which make students who work on data busy running numbers every day. Not only are they dizzy, but they also end up complaining: "Why can't we have foresight in advance?" "You're not in-depth enough!"

What to do? Today I will explain it systematically.

1. Common Mistakes

The most common approach is to break down the indicators when they change. Various dimensions are pulled out for cross-pollination, and the factor with the largest difference is considered to be the factor causing the indicator fluctuation (as shown in the figure below).

Doing so is very brainless and inefficient.

It’s a no-brainer because the business side is concerned with specific issues. For example:

• Is it because the new product is not good enough?
• Is the opponent making a move?
• Is it that the execution is not in place?
• Is there a change in the environment?

…

These business reasons cannot be summarized by simple dimensions such as "gender, age, region, product name" in the database. Therefore, even if a cross-tabulation is pulled out, these deep-seated questions cannot be answered.

The reason for inefficiency is: it seriously wastes the time of data analysts. Many fluctuations are natural fluctuations or caused by the business themselves. Many fluctuations are simply because the developers moved the tracking points but did not say anything. These problems do not require repeated cross-tabulation. Forcing data analysts to draw cross-tabulations not only wastes time, but also misses the opportunity to summarize patterns and conduct in-depth analysis.

So, how to optimize the practice?

2. Three Key Points of Diagnostic Model

From the source, ask three soul questions:

1. Are all indicator fluctuations important?
2. Are all fluctuations caused by unknown reasons?
3. Are all fluctuations worth taking action on?

The answer is: No, no, no!

At least 3/4 of the fluctuations are planned, predictable, and not worth paying attention to. Therefore, the basic work beforehand is far more useful than rushing. It is the key to solve the problem systematically to distinguish the indicators, collect the causes in advance, and predict the results in advance. To achieve this, it depends on the support of the entire workflow, not a string of mysterious codes.

3. Differentiate between core, subsidiary, and marginal indicators

Indicators related to revenue, cost, and profit are all core indicators. Fluctuations in core indicators must be given priority attention.

Subsidiary indicators are process indicators or sub-indicators that make up revenue, cost, and profit. For example, the number of users, conversion rate, average order value, etc. Is the fluctuation of subsidiary indicators a problem? Not necessarily. It is very likely that the business development has taken a new form. Therefore, there is no need to look at changes every day, but to focus on development trends (as shown below):

Marginal indicators are indicators that are not directly related or even cannot be accurately quantified, such as satisfaction, NPS, etc. These indicators can be monitored for their long-term trends. In addition, it is more valuable to pay attention to extreme cases in word-of-mouth and public opinion (particularly dissatisfied customers or malicious attacks) than to look at statistical indicators.

Of course, the definitions of core, subsidiary, and marginal of different businesses may vary. But it is necessary to treat them differently, otherwise it is very likely that you will end up in a dilemma: "After a lot of analysis, there is no impact on performance at all."

4. Clarify the positive and negative reasons

Common positive reasons:

• Promotions
• Positive policies
• New Arrivals
• New store opening
• Peak season is coming

Common negative reasons:

• System downtime
• Policy negatives
• Old products are delisted
• Rainy weather
• Off-season is coming

Not only can these be known in advance, but quite a number of them can also be analyzed in advance to give an acceptable range.

For off-season/peak season, the periodic analysis method can be used to extract periodic fluctuation patterns from past data (as shown in the figure below).

For promotional activities, you can first label the activity types, and then calculate the input-output ratio of each type of activity based on past data.

When a new product is launched, you can first label the product type and then calculate the product LTV curve based on past data.

When opening a new store, you can first label the store type, and then calculate the store LTV curve based on past data (the principle is the same as product classification).

Through label classification + re-analysis, most fluctuations caused by natural and human factors can be quantitatively classified. Collecting these reasons in advance can greatly alleviate the nervousness caused by indicator fluctuations and focus on the issues that should be focused on.

Note that there are two types of questions that are difficult to prepare for:

1. Sudden accidents, such as system bugs, bad weather, etc.
2. Changes in external factors, such as competitor promotions and policy risks

These require communication + problem-solving mechanisms.

5. Routine communication and troubleshooting

General communication:

• From the business perspective: recent promotions, product launch and delisting plans, store opening plans, and launch plans.
• From the technical perspective: development progress, development issues
• From the outside: new policies are released and take effect; competitors have announced their actions

Troubleshooting: basic data quality, regular daily data verification.

All the information, when summarized on a timeline, can form the basic material for interpreting fluctuations, and then wait for the data to give results. After looking at the results, decide whether to go deeper.

6. Diagnosis after the result occurs

Category A: Know the reason + within expectations + positive changes. As long as the expected value is not broken, monitor the trend. If you want to know the reason for the fluctuation, just four words: normal fluctuation.

Category B: Know the reason + within expectations + negative changes. As long as the expected value is not broken, monitor the trend. If you want to know the reason for the fluctuation, just four words: normal fluctuation.

Category C: Know the reason + unexpected + positive change. For example, as shown in the figure below, the original expected promotion would increase sales, but there was no response. What's the reason? The activity was not good enough... At this time, directly cut into the details of the activity analysis, let the business side do first-hand research, and think of a life-saving solution.

Category D: Know the reason + unexpected + negative change. For example, as shown in the figure below, the bad weather was expected to last too long, causing some of the weak stores to fail. At this time, we need to split our forces into two groups.

Yi Lu: Analyze whether there are other cross-cutting factors that contribute to the evil

Another way: do benchmark analysis to see if there are emergency measures in adverse environments

Category E: Unknown reasons + positive changes. Is it a good thing to exceed expectations? Not necessarily. For example, if the business side believes the short-term sales surge and replenishes the stock, it will only cause a larger backlog. Therefore, when positive events exceed expectations, we should pay special attention to related factors, such as out-of-stock of best-selling products, backlog of slow-moving products, skyrocketing marketing costs (don't let the bargain hunters get away with it), and a surge in the number of complaints.

Category F: Unknown reason + negative change. This is something to be wary of. At this time, you should first "look at three things"

At first glance: local problem or global problem

Second look: sudden problem or persistent problem

Third look: Is there a sign of relief or is it getting worse?

(Take a simple example as shown below)

In principle, it is faster to find the cause of local and sudden problems from the inside; global and persistent problems may have profound external influences. When I shared "How to increase DAU, how to do data analysis?" before, there are more detailed explanations, which you can refer to.

In short, with sufficient basic preparation, you can quickly distinguish between the light, medium and severe levels of the problem, output analysis conclusions, and lay the groundwork for subsequent analysis, avoiding aimless cross-talk.

VII. Summary

Data analysis requires counting, but to interpret the resulting numbers, one needs to have a grasp of the rich facts, use data to quantitatively evaluate the quantifiable parts, monitor the continuously developing parts, and disassemble the ambiguous parts, so as to get closer and closer to the truth.

It should be noted that these tasks cannot be completed by data analysts alone.

• If the leader himself does not know the goal
• If we develop things on our own,
• If the business doesn’t even know what “classification” is
• If the business must say "What I do is awesome Kras! There must be other reasons interfering with me!"

…

Analysis? Bullshit! The conclusion of the analysis is: this company has too many idiots and cannot be saved.