How to build an evaluation model

This article starts from a real-life scenario and introduces the question of where to start with the evaluation model. Finally, it objectively proposes reasonable steps and concludes that data analysis should be combined with business. It is necessary not only to understand the business process but also to participate in the business. The article is full of practical information and well-organized. Students who are interested in data models are welcome to learn together.

Evaluating the effectiveness of business activities is a very common data analysis requirement. However, in many cases, leaders do not buy into the evaluation results given by data analysts.

"The evaluation is unreasonable"; "The evaluation is not comprehensive"; "There are no suggestions for improvement after the evaluation!" These three questions are heartbreaking. How should the evaluation model be built? Today, let's take a practical scenario to explain in detail.

1. Problem scenario

A company is preparing to evaluate the effectiveness of employee training and wants to know whether the employee training has produced performance results. Because there are many employee training sessions and many people participate in each session, it is not clear how to analyze it.

What would you do?

A. Calculate the correlation coefficient using training sessions and sales performance

B. Use training input and sales performance to create a regression model

C. See whether sales performance is higher or lower before and after the training.

D. Look at the performance of those who participated in the training and those who did not.

E. None of the above is correct

Take a minute to think about this.

2. Problem Analysis

Obviously, options ABCD are all bad ideas. They all make the mistake of "number-based" and piece together the few numbers at hand without considering the actual business situation.

Just a few questions:

• Is all training effective?
• Is all training for performance?
• Has everyone participated in the training?
• Are all performance improvements considered training?

Options ABCD cannot answer any of these questions. If you want to explain clearly, you have to start by sorting out the business and looking for the answer in depth.

3. The first step: business sorting

Are all trainings effective? Of course not! Daily new employee training, etiquette training, company culture training, group leader speech implementation meetings... these are all kinds of training, but they have nothing to do with performance benefits. If they are all mixed together, it will definitely be difficult to distinguish. Therefore, it is necessary to classify them in advance and filter out those trainings that may be useful to the business (as shown in the figure below).

Step 2: Identify outcome indicators

Does all training improve performance? Of course not! For example:

• New product training → Increase new product sales
• Promotional event training → Increase event participation
• Complaint response training → Reduce the number of customer returns

Different trainings have different effects, so it is necessary to design evaluation indicators based on the characteristics of the training.

Note! Training is time-sensitive. For example, new product training may be effective in the first 2 to 4 weeks after the new product is launched; promotion training may only be effective during the promotion period. Only a small number of training may have long-term effects. However, considering that human memory is also limited, it is not appropriate to expand the inspection period indefinitely. When calculating the training effect index, you must choose a limited time period.

5. Step 3: Sorting out process indicators

• Really everyone is here to listen?
• Has everyone really heard it?
• Did everyone really understand?

Obviously not! Especially for those things that really affect performance, such as product knowledge, promotion rules, and sales skills, it is very likely that they have passed by your ears and have not been heard. Therefore, data records are needed, otherwise, if you try to do everything at once, you may not be able to find the problem.

Attention! This step requires good training on site:

1. Check in → Know who is coming

2. Before the end, let the students make an evaluation on the spot → know who listened to the end

3. After the end, give an evaluation questionnaire → to know whether you understand

Especially for training on product knowledge and promotion rules, which have standard answers, testing is essential. It can not only help you understand the trainees' situation, but also help you fill in the gaps afterwards and remind them not to forget the key content.

Attention! There is a high possibility of missing data here. For example, the business department only signed in, but did not do a pre-event evaluation or an evaluation questionnaire. It may be because they did not expect it, were lazy, or were afraid of trouble. Here, data analysts need to take the initiative to suggest and increase the data collection of key training sessions. This is not only important for post-event analysis, but also for the business to prove its innocence!

Because once the performance is not good, people are likely to say: "Your training was not done well!" How can you prove it? Just rely on these data:

1. You see, your people are not actively participating and are busy with other things.

2. You see, your product is not clearly explained, and 60% of people remember it wrongly.

3. Look, your activity rules are so complicated that 80% of people don’t understand them.

Having data to speak for itself can at least protect itself from being treated as a garbage dump for shirking responsibility.

Is this the end of the story? Of course not, there is still an important task to be done.

6. Step 4: Eliminate mixed factors

Can training cure all problems? Of course not. It is very likely that some people are born talented and can do well without training. Especially in the sales department, which is directly related to performance, there is a greater possibility of polarization. Therefore, it is necessary to stratify the basic conditions of the participants in the training so that we can see clearly whether the training has played a role.

In theory, as long as training can improve new employees and low- and medium-level employees, it is already a great achievement. As for high-level employees, whether it is training that makes them do well or their own abilities, their performance output is very high anyway, so there is no need to worry about it.

Therefore, we must first classify the participants in the training. For example, the simplest method is to first distinguish between new and old employees, and then use past performance (old employees)/work experience (new employees) to distinguish who is an expert and who is a novice. In this way, you can see the actual effect clearly (as shown below)

7. Step 5: Draw conclusions from the analysis

After making sufficient preparations, you can draw conclusions from the analysis. Note! In the second step, different types of training have different assessment results and indicators, so you cannot use the same set of standards. Instead, you need to look at how to make the evaluation.

For example, when launching a new product, you need to compare it with new products of the same type or old products from last year to see whether the previous pre-launch training has played a positive role in the new product. The emphasis here is on "same type" because new products themselves have their own differences, and you can't compare a hot product with an ordinary product.

For example, in case of training on complaints, it is very likely that the number of complaints received by the company during a certain period of time remains high, and returns also increase. Therefore, when examining the effect, a time comparison should be made before and after the training. Observe whether the training has effectively reduced complaints/returns.

In this way, we can not only see the effect of training, but also find ways to optimize training, such as:

1. Insufficient number of participants → Training organization needs to be strengthened

2. Enough participants, but no one remembers → Training content needs to be optimized

3. Those who participate in the training are all experienced, and there are few newbies → Optimize the training time to ensure participation

If you find that no one understands some product descriptions or activity rules, you can also give feedback to the relevant departments and urge the marketing department to make further explanations and adjustments. This can not only achieve work results, but also avoid wrangling and blaming each other between departments, which is the best state.

8. Summary

We always say that data analysis should be combined with business. In fact, business processes are very complex, and businesses often ignore data collection. To combine data with business, we must not only understand the business process, but also participate in the business and actively optimize data collection. This will not only make the post-analysis more meaningful, but also allow the business to prove its innocence, thus achieving a win-win situation.

Author: Down-to-earth Teacher Chen

Source: WeChat public account: "Down-to-earth Teacher Chen (ID: gh_abf29df6ada8)"