How to Choose and Test an Attribution Model

When a marketer faces the challenge of attracting a certain number of conversions without exceeding the budget, they must make sure the attribution model they apply helps them make the right decisions and understand the model’s logic. Many mistakes can be made along the way, leading to losses of time and money.

In this article, we’ve put together information that will help you compare and apply modern attribution models to help you take your company’s marketing to the next level.

Since users may interact with several advertising campaigns, you need to apply an attribution model to estimate the number and value of conversions resulting from each campaign. An attribution model allows you to distribute the conversion value across campaigns a user has interacted with before converting.

What do we expect from an attribution model? It should be accurate and understandable. But the majority of known models satisfy only one of these requirements.

For example, the most popular last-touch attribution model (last click, last non-direct click) is quite clear: All value is given to the last campaign. But it has an obvious drawback: It ignores the contribution of all campaigns except the last.

Many people use associated conversions, especially to evaluate display campaigns. This model is also clear enough: It gives the conversion value to every campaign the user interacted with. But such an assessment is extremely inaccurate, since it only takes into account the presence of a campaign in the chain and ignores the degree of its influence.

For example, reach campaigns get the value of all conversions by users who see the campaign banners. As a result, the number of attributed conversions significantly exceeds the actual number of conversions.

Modern services prioritize accuracy and develop probabilistic attribution models. For example, Google promotes its Data-Driven model, Facebook promotes Conversion Lift, and Nielsen promotes Campaign Lift.

These models share a common approach: They measure to what extent a certain campaign has increased the likelihood of conversion and determine its value accordingly. This makes the assessment more objective but leaves many open questions for advertisers, as the models look like a black box and there are limited or no debugging opportunities.

At OWOX, we have been developing a probabilistic attribution model since 2015. The OWOX attribution model is based on machine learning. It evaluates ad campaigns at the user session level, takes into account their contribution to the funnel, and allows marketers to specify managed channels and the conversion window as well as connect CRM data.

We’ve recently launched a new version of the OWOX attribution model that brings together best practices and delivers important business benefits:

1. The model determines the contribution of campaigns based on the likelihood of converting and does not require manual selection of funnel steps. In the example below, in the first session from the Paid Search channel, the user had a 20% chance of converting; in the second session from the Retargeting channel, the probability increased to 70%; and in the third session, the user converted.

The value of each session is equal to the increase in the likelihood of a conversion by the user compared to the previous session. In this case:

• the first session from Paid Search will receive 20% of the conversion value, as it increased the probability from 0% to 20%
• the second session with Retargeting will receive 50% of the conversion value, as it increased the probability from 20% to 70%
• the third session with Email will receive the remaining 30% of the conversion value

In any probabilistic attribution model, the most important thing is the accuracy in predicting the likelihood of a conversion. OWOX BI machine learning algorithms are extremely accurate, which is confirmed by the results obtained by our client, who used these algorithms to achieve a 2.2x increase in ROI.

2. The second advantage of the new version of the OWOX attribution model is its ability to predict the value of even those sessions that have not yet led to a conversion:

This forecast allows you to find out how many conversions from an advertising campaign you can expect to receive in the future if you turn off the campaign today.

This allows you to make quicker decisions to disable underperforming campaigns and not disable delayed campaigns by mistake.

A model’s prediction accuracy depends not only on the algorithms but also on the size of the training sample. The increase in the speed and quality of forecasting for the OWOX model was achieved by training the model on tens of thousands of projects.

When evaluating campaigns using different attribution models, you can expect different results. Therefore, the question inevitably arises: Which attribution model should you choose, and which shows the right path?

In fact, a marketer doesn’t need a compass that points left or right but needs a full-fledged navigation system that calculates and recommends routes, taking into account traffic jams and the required arrival time. This is an attribution model, which allows a business to achieve a goal based on specific recommendations.

Moreover, managing your ad budget without taking into account channel capacity can lead you to a dead end, just like a compass that doesn’t take into account the landscape. Most marketers know that a low cost per conversion isn’t enough to make a decision to increase the budget. For example, branded campaigns may have a low cost per conversion but have depleted their capacity, so increasing your budget will not increase conversions.

The graph above shows how the number of conversions in campaigns A and B depends on the budget invested in those campaigns. Note that the current CPA of campaign A is less than that of campaign B. But the next conversion in Channel A will cost more than in Channel B!

Therefore, to apply attribution, you need to know not only the current performance of the channel but also how the overall result will change if you increase or decrease the budget for each channel, campaign, and keyword.

With such a model in their project, the marketing team can choose the appropriate growth strategy:

1. Get the same number of conversions by slashing your budget.
2. Increase the number of conversions with the same budget.
3. Increase the number of conversions with the same CPA.

This allows you to immediately get not only recommendations on redistributing the budget but also the predicted result.

The main question remains: How can you know the capacity curve for each keyword? At OWOX BI, we use market data from tens of thousands of projects to train a model that answers this question when given a project’s region and niche. Based on our experience, a month’s data even for large projects is not enough to get a high-quality forecast.

Key takeaways:

1. An attribution model evaluates advertising campaigns but doesn’t provide recommendations for redistributing the advertising budget.
2. To get recommendations, you need to know the capacity of campaigns and the promotion strategy.
3. The best attribution model is one based on recommendations that allows your business to achieve a specific goal.

Ways to increase channel capacity (media) and common mistakes in setting goals for agencies (last click) deserve special attention (and articles). Write in the comments what you’d like to read about in our next articles.

If you want to learn more about the Machine Learning Funnel Based Attribution by OWOX, sign up for a demo. We will be happy to answer all your questions.