Why are your data review reports always criticized?

Many data analysts often encounter questions from their leaders or business departments when writing review reports. This article will start with common problems, gradually guide readers on how to delve into business details, and discuss in detail how to avoid these problems, using a five-step method to improve the quality and practicality of reports.

"Reviewing activities, sales, and product performance, summarizing successful experiences, and providing optimization suggestions" is a basic work requirement for data analysis. However, many students have encountered the situation where their review reports were dissed by their leaders/business, who said:

“I knew it.”

“Are there any in-depth discoveries?”

“Any actionable suggestions?”

What should we do? Let me share with you in detail today.

1. Frequently Asked Questions

• Problem 1: The review only has result indicators. It repeats over and over again: "This year is XX more than last year" and "The activity period is XXX more than the non-activity period", but there is no deeper and more detailed content.
• Problem 2: The review lacks horizontal and vertical comparisons and only focuses on the current situation. When talking about an event, only the current event is discussed; when talking about a product, only the performance of the new features is discussed. It is very likely that these indicators have already been reported in the regular report, so naturally everyone will say, "I knew it a long time ago."
• Problem 3: The review lacks in-depth attribution, and only writes "to increase". "Too few people participated in this event, it is recommended to increase", "The retention rate of this new function is very low, it is recommended to increase"... It's the same as saying nothing.

The reasons are as follows:

• Check the regular results indicators every day. It is definitely not enough to just talk about the results.
• What the business can do after seeing the data is the key, and it needs to be associated with business actions
• Looking only at short-term performance may not be a good solution. Horizontal and vertical comparisons are very important.

In short, we need to go deep into the business details. The specific operation can be divided into five steps.

2. Step 1: Review the achievement of goals

Although it is not enough to only look at the main indicators, it is definitely not okay not to look at the main indicators. Not only should you look at them, but you should also look at them first. First, make it clear what the specific business discussed in this review is and what the main indicators of the assessment are.

Commonly used, such as:

• Sales progress review: revenue, gross profit, and payment collection
• Marketing activity review: input, incremental output
• Product launch review: revenue, turnover rate, sell-out rate
• Product function review: number of users and number of retained users

Here is a special reminder: whether to examine the total volume or the new volume, it must be clearly stated in advance. For example, sales generally look at total sales revenue, but marketing generally requires: exclude natural sales volume/exclude normal levels.

So how should we exclude them? Should we look at the time period without activities, or choose a group of people who do not participate in activities/products that do not participate in activities for comparison? We must make it clear in advance! In short, the first part of the review report is to report on the achievement of this goal as usual, and first make a judgment on the overall effect (as shown in the figure below).

Step 3: Find a benchmark for comparison

This step is particularly important! If you only look at a single business entity, you may not be able to draw a deeper conclusion, so you must label it and extract comparable benchmarks for comparison.

For example, reviewing sales progress:

• What is the sales team's personnel composition and past performance?
• How much performance and expenses does this sales team have?
• How did this sales team, a team of the same level, perform during the same period?

Marketing activity reviews are often done in more detail because marketing activities are more flexible. For example, in the same large-scale event, there are multiple traffic sources, multiple marketing rules in effect at the same time, and multiple product activity pages.

At this point, you need to comb through the process carefully and sort out the conversion of each process, because it is very likely that the overall input-output ratio is good, but:

• Some segmented channels have low traffic efficiency
• Some product categories cannot be converted
• Some products have high conversion rates but receive little traffic

These problems will be exposed in the review (as shown below)

If it is a product review, please note that product improvements may evolve through versions. For example, if you optimize the APP conversion funnel, each version may have some improvements. In this case, you need to show this evolution process during the review and tell everyone how long it took to run and what the effect was after each adjustment (as shown in the figure below).

4. Step 3: Connect business actions to find process indicators

The first step is to know whether we did well or not.

Step 2: We know whether this is better or worse than the same category.

The third step is to explain why the performance is poor and derive optimization suggestions.

In this step, remember to blindly insert indicators, because many indicators are not useful to the business. For example, if you throw a sentence like "During the event, male users participated less...", what then? So what can the business do specifically for men? This kind of things that you don't know what to do after seeing it can be completely reduced.

Attention! Business is not as complicated as you think, and many routines are fixed. For example, to increase sales, you can do the following:

• Develop more high-quality industry customers → Corresponding data: Specific types of customers have a high transaction rate
• Sell ​​more of the current best-selling products → Corresponding data: more sales of specific types of goods
• Add the number of visits → Corresponding data: When the number of visits increases, the transaction rate increases

When reviewing, pay more attention to: comparing the current action with the benchmark, the differences in indicators in terms of customers, products, and sales behaviors, and summarizing the points that can be improved/developed.

For example, marketing activities can:

• Increase the intensity of activities → Corresponding data: Activities with large discounts will increase the amount of money
• Modify the promotional materials → Corresponding data: The promotional materials are version A, read higher
• Add the number of promotions → Corresponding data: more promotions, more participants

When reviewing the event, pay more attention to: the outstanding points of the event and summarize the experience.

For example, products can do:

• Shorten the operation path → Corresponding data: Long paths have lower conversion rates
• Modify the page layout → Corresponding data: Click conversion is higher in version A than in version B
• New operation function → Corresponding data: More people use this function

When reviewing, pay more attention to: whether there are obvious improvements compared to the previous version, and summarize the experience

Step 4: In-depth understanding of differences

Note! Actually, for the review, by the third step, there are already feasible conclusions. However, sometimes, the business will want to further explore: Why was the effect good before, but not so good now? In addition, it is possible that the current environment has changed, and the previous experience can no longer be used, and new practices need to be explored.

At this point, you can first determine whether past experience is no longer effective. If you find that:

• Looking vertically, the benchmarks with good performance all occurred earlier, and those with poor performance recently
• Looking vertically, the same sales strategy/activity strategy has a worse and worse execution effect
• Looking horizontally, all types of measures are declining, and there is no better reference
• Looking horizontally, external competitors are also frequently adjusting their strategies/increasing their efforts

Well, it is highly likely that the old methods will not work... Especially in 2024, when the overall environment is not good, similar scenarios are likely to occur.

At this point, we can only accept the reality and gradually adapt the model to the current situation. When reviewing, it is recommended to start with user grouping, make a more detailed split, and observe:

• Under the current circumstances, which users are relatively loyal consumers?
• What are the characteristics of these loyal consumers and through what channels can they be obtained?
• What types of marketing activities do marginal consumers respond to?

At this point, you can only gradually optimize based on the current parameters to try to improve business indicators (as shown in the following figure).