Is the data fluctuating again? Five steps to accurately identify abnormal data fluctuations

Monitoring data and analyzing abnormal data are common tasks in data analysis. Although this requirement seems simple, it is not so easy to monitor and identify. Which changes are normal data fluctuations and which are abnormal?

As data analysis job recruitment becomes more and more inward-looking, more and more people are asking "What models have you used/built?" This question can easily give people the feeling that "interviews are like building aircraft carriers, but work is like tightening screws." In actual work, do we really need to make so many models? ? ?

This caused many students to wonder:

"Monitoring data and analyzing the reasons for indicator changes" is a common work content of data analysis. This requirement seems simple, but in practice it confuses many students: How big is the indicator fluctuation? Why sometimes the fluctuation does not seem to be big, but the business is very nervous and asks the reason, and sometimes the fluctuation is big but the business is not anxious?

The key to solving the problem correctly is not to just look at the numbers, but to understand the business meaning behind the data fluctuations. Only in this way can we know which are real changes and which are normal fluctuations.

1. Normal Fluctuations That Seem Abnormal

There are three types of fluctuations that are typical and appear abnormal but are actually normal:

1. Seasonal fluctuations in business indicators
2. Changes in the life cycle of business indicators
3. Business proactive behavior triggers change

Therefore, before identifying abnormal fluctuations, you should proactively analyze the historical trends of business indicators, discover seasonal patterns/lifecycle trends, and analyze the impact of business actions on indicators. This will help you establish basic judgment criteria and avoid making low-level mistakes (as shown in the following figure):

What we really need to pay attention to are the long-term, continuous changes that go against the normal development trend. These situations often mean that there is a problem with the business and need to be investigated in depth (as shown in the figure below).

2. Even if it is small, you should pay attention to the fluctuations

Typically, for example, there are changes in the two key indicators of revenue and cost. Revenue and cost are the KPI indicators of most companies. Increasing revenue and reducing expenditure is the core task of the company, so these two indicators have clear requirements specific to the month/week/day. Even if the fluctuation is small, it must be tracked down so that the business can be assured.

For example, if today's sales revenue KPI is 10 million, and the actual sales revenue is 9.9 million, then it must be an anomaly. As for where the missing 100,000 went, you can break it down by sales channel to find the source of the problem (as shown below).

Any KPI changes in the short term can be explained as "the business is not performing well", but if the performance is poor for several consecutive weeks, there may be deeper problems such as poor business capabilities or wrong methods, which need to be further investigated.

Hidden Problem 1: ROI Changes

The income/cost indicators can be viewed together, and more discoveries can be made. In theory, income is generated only when there is cost input, and income/cost should occur synchronously in a certain proportion.

If the relationship between two indicators is misaligned, there is a high probability of anomalies, such as:

• Investment continues to grow, but income does not increase...
• The investment has not been in place, but the income is still rising!

These are all problems.

At these times, the typical manifestation is that the business ROI is abnormal, either abnormally high or abnormally low. At this time, you have to find out the reason. An abnormally low ROI means that optimization is needed to improve business performance. An abnormally high ROI means that there is something fishy, ​​which is often related to business corruption and waste.

Attention! This problem is often overlooked by data analysts, but easily discovered by finance/management. Because when making monitoring reports, revenue/cost are often done separately, and cost classification is based on financial accounts, which data analysts cannot understand. Therefore, when making monitoring reports, try to put revenue/cost KPIs together as much as possible to make it easier to draw conclusions.

Hidden Problem 2: Trend Changes

Sometimes it’s not that the KPI is not up to standard, but that the indicator trend is wrong! For example:

• Although sales have temporarily met targets, costs are increasing.
• The trend was obviously good before, but it suddenly turned bad recently.
• Income growth is getting slower and slower, and is approaching a turning point.

These events will cause anxiety among management and business parties, who will then judge them as "unusual movements" and request in-depth analysis.

Here, some abnormal movements are caused by business, such as the sudden brake at the end of the month shown in the figure below. Many companies that rely on sales staff have this phenomenon. This is the behavior of sales staff who deliberately postpone orders and hide their performance when they see that their performance is about to be completed. If this happens too often, it will lead to deviations in product stocking, performance accounting, cost control, and business analysis, and it must be checked.

Some involve deeper issues, such as:

• The product life cycle has ended
• Potential target users are exhausted
• Marketing stimulation makes people tired

These curves show a continuous decline. Once you see signs of decline, you must alert the business side in advance. Because these long-term trends often require testing to confirm, and the handling is also very difficult. If problems really occur, it may be too late to handle them.

Hidden Problem 3: Process Changes

Many indicator changes come from process indicators, such as exposure, number of user logins, number of visits to a certain business, etc. These indicators are process indicators of KPI indicators, so once there is an abnormal change, it may affect the KPI results. Therefore, it will attract people's attention.

The first type: rise and fall together. This is the easiest to understand. For example, if the traffic is small, the income will drop accordingly. This problem can be directly observed with data and is easy to find. It should be noted that if the rise and fall are not synchronized, for example, the traffic increases by 30%, but the conversion increases by only 10%, then it is very likely that a large amount of low-quality traffic has come in, and low-quality traffic is often related to business corruption and waste. Therefore, the rise and fall are not synchronized, which is also abnormal and requires extra attention.

The second type: some ups and some downs. For example, although my traffic has decreased, the conversion rate has increased, and the total revenue target has been achieved. Is this situation considered abnormal? Of course it is. Because this change is likely to mean: the shift of user needs, changes in channel characteristics, changes in product attractiveness, and other deep-seated problems. And to confirm the deep-seated problems, complex analysis is required. Therefore, once such a change is discovered, it is necessary to promptly remind the business party and initiate in-depth analysis.

The third type: structural transformation. For example, I have three channels, ABC. I used to rely on A, but now A is getting less and less, while C is getting more and more. Is this situation considered abnormal? Of course it is. Because the actual business has deviated from the budget. At this time, we have to consider whether this represents a long-term trend and whether to adjust the budget. This is another complex issue. After discovering such signs, we must remind the business side and initiate in-depth analysis.

Hidden Problem 4: Related Changes

Related changes refer to indicators that have problems but are not directly related to the main KPI indicators, but are related to the business. This causes the business side to be very nervous. For example, when a leader wakes up in the morning and sees a popular video criticizing our products, he will definitely not be in a good mood and will ask, "Is there anything unusual?" At this time, even if the KPI indicators are fine, the leader will still be worried. Therefore, it is necessary to combine internal data for interpretation (as shown in the figure below).

Note! Related changes may also mean: "The good thing I expected did not happen." For example, I organized a training for the operation staff, and the attendance rate was 90%. After the training, the average score on the exam was 90/100. So I was overjoyed and thought that the event next month would definitely be outstanding. But after the training, there was no difference. Seeing this scene, I would also wonder: "What's going on?!" I would also ask: "Is there anything unusual?"

Therefore, when management/business asks, "Is there anything abnormal?", please ask them to explain clearly what is normal! This way, you can truly understand the issues they are concerned about and prescribe the right remedy.

summary

To accurately identify business anomalies, you need to:

1. Build a complete data indicator system
2. Clarify the position of problem indicators in the system
3. Understand the past trend of indicators
4. Understand business actions/business expectations
5. Establishing an appropriate standard of “normal”

This is the only way to avoid just looking at numbers and find out the abnormal problems in the business through data. However, many students are not good at abnormal analysis because they start with a scattered and rough monitoring indicator system, lack of correlation between indicators, and just throw numbers at the business without real monitoring.

Author: Down-to-earth Teacher Chen

Source: WeChat public account "Down-to-earth Teacher Chen (ID: gh_abf29df6ada8)"