Predictive Lead Scoring: Benefits and Use Case in Insurance Company

What is predictive lead scoring? It may be easy to understand the concept of lead scoring in insurance, but a bit tricky to put it into practice. Simply put,

This final “score” is calculated from multiple (and the most diverse) factors, such as demographics, behaviors, and quotes, and then compared and contrasted to the actions taken by previously converted leads.

Traditional lead scoring refers to the process where the sales team determines the lead’s quality based on a variety of factors and then assigns a score to this lead. In insurance, this is usually done by using some formula or a marketing automation tool, which can track the leads’ behavior attributes to get a fuller picture of the prospect’s desire to make a purchase.

So what’s wrong with this strategy? Traditional lead scoring in insurance is significantly limited, focusing primarily on the top of the funnel. It doesn’t take into account the complexity of today’s customer journey, which has become as diversified and non-linear as ever before.

Below is one of many other examples of what a customer journey in insurance could look like. But this might change a lot, depending on the insurance sector the company operates (car insurance, property insurance, etc.), its preferred sales and marketing tactics, technology use, and so on.

So if we briefly sum up why a traditional lead scoring model is failing your insurance business, we can mention that it’s:

1. Restricted: How many criteria could you include in traditional lead scoring? Three? Five? Ten? Regardless of the number, it’s usually not enough. And in this case, your business cannot consider complex relationships between these factors, causation, and correlation. As a result, you get an inaccurate lead score, which means lost opportunities for the insurer or, vice versa, wasted time for non-qualified leads again.

2. Labor-intensive and static: Okay, you try to diversify the criteria as much as possible, putting lots of time and effort into building an ideal traditional lead scoring system. But what are the guarantees that this system stays the same accurate and relevant in, let’s say, three months? New lead sources, metrics, and mediums could be added to an insurance customer journey at any moment. And if the insurer continues to commit to the lead scoring system that couldn’t dynamically adjust to these changes, it should expect inaccuracy in its scoring results.

3. Based on assumptions: In traditional lead scoring, the factors are usually selected based on guesses and assumptions. The management relies on the information from previously converted leads. But the analysis needs to be more comprehensive so the lead scoring was data-driven.

Easily! Use predictive lead scoring machine learning. The idea behind predictive lead scoring is to apply machine learning (ML) to calculate the lead score with the help of technology. In this scenario, an insurer benefits from:

• Using a more data-driven approach, with real-time data input analyzed through ML algorithms

• Less guesswork and more accuracy instead, as predictive lead scoring results in a more detailed and complete lead profile

• Uniqueness — since each insurance company has its unique criteria to take into account in lead scoring, model scoring in machine learning allows you to rely on them

• A high variety of scoring factors, which can help to discern previously unknown patterns in lead quality

• Getting more objective in decision-making since no humans are involved in lead scoring

• Higher speed and automation

• Improved flexibility — although it’s hardly possible to build an ideal scoring system, predictive lead scoring allows you to refine and retrain the model whenever new data appears.

When weighing over the implementation of predictive lead scoring, make sure your company considers potential challenges as well:

• Data-intensive models: For better accuracy, you need lots of quality data to feed your machine learning algorithms with. Sometimes, insurance companies don’t even realize they’re sitting on massive data, wasting its big opportunities.
• Time and money resources: Some might say that predictive lead scoring requires heavy investments, which makes it less accessible to small- and medium-sized businesses. The truth is cloud solutions and third-party partnerships provide companies with more opportunities to implement predictive models. Moreover, insurers don’t have to dive into big ML projects from the start. A better strategy is to scale predictive lead scoring step by step as soon as the project brings its first results.
• Sales team not on board: To make lead scoring using machine learning work for your company, you as an insurer need to convince the sales team of its importance and helpfulness. A good starting point is to join your efforts with ML experts and organize workshops on how to use predictive lead scoring, explaining its best practices and nuances of how ML works.

To describe the process of building an ML-based lead scoring solution, we’ll tell how the Intelliarts data science consulting team developed one for a midsize insurance company.

Business challenge

Our customer is a midsize insurance company working in property insurance. The company has a traditional workflow to process leads acquired from multiple sources, including telemarketing, partnerships, website, and others. Then, it follows up leads by cold calling, offering insurance policies.

However, a large variety of leads is hard to handle and requires the exploitation of our customer’s resources. The insurer spends extra on hiring and training agents and their wages. So, the customer’s decision to contact Intelliarts was connected to their goal of optimizing resources and building an effective ML-based lead scoring system.

Additionally, the insurance company approached us with the request to increase lead scoring efficiency using machine learning. The customer has noticed that not all leads that they contact are of the same quality. For instance, an agent could spend on cold calls three hours and sell only one policy. And the other day, the same agent could sell two or three policies during a 30-minute call.

Solution

ML-powered lead scoring solution

Having consulted the customer, we decided to build a predictive lead scoring model in machine learning that could forecast the probability of how likely the lead would buy a policy. With this system in place, insurance agents could call leads with high scores first and leave those leads with low scores for later or skip them entirely in case of lacking resources.

To build this solution, we started with a data collection process. The information we collected with the help of the insurance company could be grouped as:

• Source data: from where and how leads come to the insurer (partners, lead capture tools, company websites, and so on)

• Behavior data: how leads interact with the customer (calls, website visitors, email clicks)

• Quote data: historical estimations of how much a policy would cost

• Demographic data: all sorts of information about the age of leads, their addresses, and FICO credit score

• Property data: size calculated in square footage, the year when a property was built, coverage, and potentially dangerous factors

• Contact data: how they reached out to the lead before

In modeling, our data engineers decided to use gradient boosting techniques with this data because of three reasons: the data was tabular; it had lots of categorical features; and the target was binary (sold vs. not sold).

At this stage, we faced lots of related problems, among which were missing data and different historical factors. Since leads don’t always share their complete information, we had some gaps in the data. The challenges caused by historical changes in the company involved modifications in business processes, partners who stopped working with the customer, and the addition of new policies.

To solve these data problems, Intelliarts engineers built a Python module that contained a lot of different blocks, including:

• Extracting useful features from text and time data

• Grouping the same and similar records to prevent model uncertainty

• Cleaning data and replacing the missing values

• Cleaning anomalies and outliers in data to exclude some impossible values, e.g. negative age

• Grouping some categories and providing encoding for them to be able to work with new partners or in the new state in the future

• Intelligent replacement of the missing values

With this approach, our data scientists didn’t only manage to address the data challenges we met. We also discovered multiple insights about the insurer’s data that helped us improve the data collection logic and strengthen the ML lead scoring solution further.

The second solution we built purposed at improving lead efficiency. We used the same features as before but changed the target this time. Now our data scientists tried to predict the score continuous number, which can be described as a traditional regression task in machine learning.

But it was difficult to design correct training, validation, and testing datasets for the lead efficiency model.

Here is how we solved it:

• As said, we considered efficiency as the number of policies bought by a single lead divided by the call time spent on this lead. Importantly, there could be several calls and insurance agents who worked with this lead.

• Since there were lots of leads, the success rate wasn’t really big. Thus, we faced the problem of imbalanced data and the situation when most of our scores equaled zeros.

• For this very reason, we couldn’t include all leads to avoid lots of zeros, i.e., leads for whom agents spent some time but didn’t sell any policies. To solve this issue, we used only converted leads in training data.

• The next issue that arose was concerned with testing and validation. If we used only converted leads, we wouldn’t get a full picture. In the reality, we cannot know whether leads are sold at the time when the model is still running a prediction for lead efficiency.

• We also couldn’t leave the same distribution as in the original data because we risked getting a lot of mispredictions. The model predicts a score, but leads would get zero because no one had bought a policy.

• To solve this problem, we decided to use the previous lead scoring model we had built to run a test only on those leads that had high scores predicted by the ML model. This helped us reduce the number of non-sold leads and, thus, achieve the best results in terms of business metrics.

Business value

With this scoring model, the insurance company can group leads by the probability of being sold. Now the group with an 80% probability and more has an expected conversion rate 3.5 times bigger than the average conversion.

In contrast, the bottom group with 20% and less probability predicted by the model has 5 times lower than on average. As a result, the customer doesn’t have to waste time on 20% of all leads which wouldn’t bring any efficiency to the insurer.

If the insurer attracts low-quality leads (even if there are many of them), the conversion rate would be low. And this doesn’t bring any monetary value to the company.

Last but not least, we didn’t only build the two ML models. The Intelliarts data engineers also deployed the trained models using Amazon SageMaker. We integrated the two solutions into the customer’s system to establish connectivity and help the insurtech company avoid any system integration challenges that could arise. As it’s a full-cycle ML project, we continue to assist the company with solutions maintenance and are ready to fine-tune or retrain any model if the data changes.

As you see, predictive lead scoring is a large part of Intelliarts’ expertise. We have an experienced team of data scientists and machine learning engineers with solid expertise of ML in the insurance sector. From NLP to computer vision to predictive analytics, we build full-cycle ML solutions tailored to your specific needs.

Aside from the predictive lead scoring model, Intelliarts has also recently built an ML-powered solution to increase lead quality and agent efficiency. Our data scientists have performed data science research purposed to improve cold calling success rates in insurance.

According to Gartner, 70% of leads get lost because of poor follow-up. And every lost lead equals a missed opportunity in insurance sales. Still, an insurance company can attract hundreds of leads coming from different sources daily. How should you prioritize them?

The best idea is to incorporate a lead scoring system, which will prompt your agents whom to call first to get higher conversions and avoid wasting company resources. Even better, an intelligent machine learning lead scoring system like the one we built for the insurtech customer helps insurers adjust lead scoring based on real-time data and results. Let alone, if you decide to empower these lead scoring solutions with better lead quality achieved by implementing an ML-based lead efficiency system.

Read also how integrating an automated data extraction solution can greatly enhance the efficiency of sorting and processing large volumes of information, leading to more informed business strategies.

Are you also tired of wasting time and money on unsuccessful cold calls or want to prevent customer churn in the insurance business? Reach out to our expert Intelliarts team to implement predictive lead scoring and bring your sales effort in insurance to a new level.