Digital Attribution Modelling

Having a feel for your typical customer journey is an integral part of the attribution model design process.

For example, understanding the number of steps you average user takes, and over what time frame, can massively influence the assumptions you make and hence your choice of model. Fortunately almost all digital reporting platforms provide you this kind of insight as part of their standard reporting toolkit, under a name of “time lag to conversion”, “path length”, or something similar.

If most customer journeys are only a single step and a customer ‘converts’ there and then, then you need make few assumptions: for whatever model you’ve chosen there are few ways to distribute credit.

In my experience, short paths are common in insurance products. A customers’ mind set is one of speed, convenience and price – often on a renewal deadline. With the prominence of price comparison sites acting as one-stop-shops, a customer has few opportunities to be exposed to advertising and as such generates few data “touch points”.

As the research phase extends, so we collect more advertising interactions and the complexity of our path evaluation increases. Where a purchase decision may take weeks or months, so we must now decide not only on advert resonance but also how time plays as a factor.

Early on in their journey they may just be getting a feel for the marketplace; which brands are out there, what are their perceptions or previous experiences of these, do they trust the site and does it have what they are in market for?

As they progress: maybe their activity is more about deciding between a shortlist of products or brands? We expect refined search terms and specific information sought on dedicated web sites. Towards the point of conversion have they in effect made up their mind, and are now looking simply for reassurance through second opinions on social media and product review sites?

I see longer journeys more commonly with high value purchases and/or ‘desirable’ goods – homes, holidays and mobile phones for example: areas where customers are happy to spend more time browsing and researching to get a specific deal or set of product features.

From an analytical viewpoint then: does our understanding of the customer journey extend to allowing us to identify which stages are more important than others? Are they equally important? Do we believe that as the user nears decision then so the advert’s influence becomes more important? Or do we assume the early (or first) brand association step was key to the chain? Is a particular advert channel known to be strong in this sector?

In some cases we can answer these questions with data, in others we must make an educated choice.

If you’re dipped your toe already in to the Google Analytics world, you will probably have already seen their [relatively] simple modelling tools. These reports allow a simple, low-cost entry in to envisaging the results of a particular credit attribution model:

GAModels

I would say if you have fairly short conversion paths of one or two steps, and/or your online budget levels are fairly small, then these are perfectly adequate tools to begin exploring and testing models.

As a recommendation, my advice would be to try and steer clear of the ‘absolute’ models (First interaction and Last interaction). If you’ve read any of the previous posts, you’ll hopefully agree that giving all credit to just one interaction is like giving all the credit in a football match to your goal scorer. Assuming you optimise on the results of your model (if not – why bother at all?), you’d end up with a team purely of forwards which is more than likely, “a bad strategyTM”.

There are some other notable limitations to these positional models, and I would advocate you should be aware of these even though it doesn’t necessarily prevent you from drawing useful insight.

Firstly: technological limitations. Online tracking technology is not perfect. The primary tracking mechanism is a cookie, which is device specific; by that I mean tied to your mobile, tablet, laptop or computer. Many customers will use multiple devices depending on where they are and whom they are with: assuming they don’t identify themselves to the browser (such as signing in to Google+), they will generate independent cookie data on each device.

While you can see the device on which the user completes a purchases (last click), you have no idea where they really started their journey. Putting all your eggs on the first known step is, I’d suggest, at best imprecise and at worst horribly wrong.

Secondly: these models are digital domain only, and make no accounting for offline advertising such as TV and Radio, or even plain old word-of-mouth. You will with certainty be over-estimating the contribution from your online advertising, particularly through any brand name search advertising which is commonly seen as a means to site navigation, rather than advertising per se. You might be able to apply some gut feel scale-factor to adjust these numbers particularly if you’ve been monitoring offline campaigns for a while, but unfortunately at this level of complexity there’s not much more science you can apply.

Thirdly: in the GA implementation at least, the data used to feed these models only contains the paths of successful ‘converters’ (i.e. customers who complete a certain phase such as quote request or sale). Each occurrence of a particular advert is rewarded with incremental credit regardless of its realistic influence at a given step, and there is no negative feedback from failed conversions against which this incrementality may be offset. In more complex models, we observe that repeat advert appearances can in fact be an indicator of decreased effectiveness.

Still, no model is perfect and given that the results return quickly it is worth comparing a couple of these models side by side. If the results come out pretty similar for a period of a month, then it’s probably not worth getting too hung up on exactly which is best – take one, and try it out for a while (i.e. see if making the changes the results imply actually result in better returns).

When your online advertising expenditure sits in the “higher” category, or your customer journeys are more protracted, then you are probably at a stage where either a more complex model, or certainly a better fitted model, is worth investment.

I’ll get on to techniques later, but a good rule of thumb here is that these models shouldn’t just apply more complex rules, but more complex techniques. This means that rather than making more assumptions, you make fewer.

By more complex techniques; we’re really talking about utilising more complete data sets, either pushing it through statistical models (for example, logistic regression or decision trees), crunching it brute-force style with algorithmic solvers to produce weighted positional or Markov chain models, or using mathematical game theory solutions (such as obtained via Shapley Value techniques).

I’ve seen, and admittedly produced (under protest!), rule-based attribution models that apply an inordinate number of “if-then” conditions, all of which were based entirely on conjecture. These results were reported upon, but with an ever changing strategy landscape and high seasonal volatility no feasible testing structure would ever be possible. In other words, it was guess work.

Testing is a basic requirement for any model development lifecycle. If you make changes based on a model that doesn’t result in greater success (all other things being equal), then you haven’t got a good model and you’re likely wasting money. Moreover, if you can’t work back from results to understand which part of your model might be at fault, you’re doomed to repeat the failings.

The best way to test your model is, of course, to spend some money. If it looks like doing more of A is a cost effective solution, then do some more “A” and see if your sales increase and efficiency remains roughly consistent. Utilise A/B tests to help calibrate your model.

However, it is often difficult to convince a stakeholder to ‘gamble’ on a model, and frequently the market place differs significantly from one month to the next, making the ‘all other things being equal’ stipulation awkwardly un-equal. If you have a regular periodic measurement framework in place though, and maybe a second model running in parallel for comparison, you can begin to see which particular adverts in a campaign are consistently performing well, and certainly those that aren’t, and from this begin to make some campaign decisions.

Advertisements

Choosing an attribution approach that is right for you

Let me start this section by saying: there is no ‘right’ attribution model. There are no off-the-shelf solutions out there that you can buy, drop on your data, and hey presto: instant, accurate campaign evaluations.

In order to get to a view of campaign performance there are concessions you must make, the biggest being whether to split your views of the online and offline world in to independent, mutually exclusive existences, or whether to try and integrate them. No: I take that back. The biggest concession is how much time (or money) you’re willing to invest in improving your understanding of how your advertising works. More advanced techniques may take weeks to generate, and then need ongoing support and tweaks.

The underlying complexity driving this decision is a result of the vastly different data sets that arise from the offline and online worlds. The former are generally traditional media channels such as radio and TV, to which you may physically be unable respond in kind (at least until touchscreen TV technology and its ilk is widely deployed). Adverts are served at set times in the programming, and are ‘slow’ at typically 10-30seconds in duration. Whether an individual has seen your advert is an unknown.

Contrast this with online advertising where adverts are shown across most pages, at any time, are renewed with each refresh and sometimes with each scroll. A user browsing around may be exposed to tens or hundreds of adverts in a very short space in time, with every viewing and interaction tracked to the nth degree. Advert content (‘creative’) can be generated programmatically on the fly leading to a multitude of different versions and, significantly, there is (with some exceptions) a cookie record tracking the sequence of what you’ve seen and done.

In essence: you have offline data that needs (let’s say) weekly aggregations in order to provide sufficient sample volumes for a robust pattern to emerge. You then have online data that comes thick and fast, but where scale is thinned by high variety and fine detail.

So gives rise to two main techniques, Econometrics and Digital Attribution. And occasionally: an attempted mash of the two.

Econometrics attempts the holistic view, giving up some of the finer detail in order to give an all-encompassing view. Models will vary in complexity (often determined by the amount of time invested in developing the model), but the basic premise is one of linear relationships: do more of advertising type A, get a proportional increase in sales back x weeks later.

Digital Attribution foregoes explaining the offline activity and instead makes use of the online tracking technologies to try and piece together a customers’ journey. Broadly speaking, these techniques fall in to two camps. The first correlates customer interactions with sales activities, and leaves sales unexplained by patterns in the digital data in an unallocated ‘pot’. The second fully apportions the known sales across digital advertising channels. Both may choose (or not) to make sequencing and chronology a factor, but both techniques make use of case-wise customer journeys as opposed to ‘activity volumes’ as used in econometrics.

I’m not an econometrician, and my work falls in to this second set of techniques: digital attribution. In agencies this is often a complementary piece to an econometrics project, and is used to provide further, more granular insight in to the finer detail of online campaign performance.

So, having set up the background, let’s talk more about this latter field of analysis.