Engagement Scoring + Adobe Analytics Derived Metrics

Recently, I was listening to an episode of the Digital Analytics Power Hour that discussed analytics for sites that have no clear conversion goals. In this podcast, the guys brought up one of the most loaded topics in digital analytics – engagement scoring. Called by many different names like Visitor Engagement, Visitor Scoring, Engagement Scoring, the general idea of this topic is that you can apply a weighted score to website/app visits by determining what you want your visitors to do and assigning a point value to that action. The goal is to see a trend over time of how your website/app is performing with these weights applied and/or assign these scores to visitors to see how score impacts your KPI’s (similar to Marketing Automation tools). I have always been interested in this topic, so I thought I’d delve into it a bit while it was fresh in my mind. And if you stick around until the end of this post, I will even show how you can do visitor scoring without doing any tagging at all using Adobe Analytics Derived Metrics!

Why Use Visitor Scoring?

If you have a website that is focused on selling things or lead generation, it is pretty easy to determine what your KPI’s should be. But if you don’t, driving engagement could actually be your main KPI. I would argue that even if you do have commerce or lead generation, engagement scoring can still be important and complement your other KPI’s. My rationale is simple. When you build a website/app, there are things you want people to do. If you are a B2B site, you want them to find your products, look at them, maybe watch videos about them, download PDF’s about them and fill out a lead form to talk to someone. Each of these actions is likely already tracked in your analytics tool, but what if you believe that some of these actions are more important than others? Is viewing a product detail page as valuable as watching a five minute product video? If you had two visitors and each did both of these actions, which would you prefer? Which do you think is more likely to be a qualified lead? Now mix in ALL of the actions you deem to be important and you can begin to see how all visitors are not created equal. And since all of these actions are taking place on the website/app, why would you NOT want to quantify and track this, regardless of what type of site you manage?

In my experience, most people do not undertake engagement scoring for one of the following reasons:

• They don’t believe in the concept
• They can’t (or don’t have the energy to) come up with the scoring model
• They don’t know how to do it

In my opinion, these are bad reasons to not at least try visitor scoring. In this post, I’ll try to mitigate some of these. As always, I will show examples in Adobe Analytics (for those who don’t know me, this is why), but you should be able to leverage a lot of this in other tools as well.

The Concept

Since I am by no means the ultimate expert in visitor scoring, I am not in a position to extol all of its benefits. I have seen/heard arguments for it and against it over the years. If you Google the topic, you will find many great resources on the subject, so I encourage you to do that. For the sake of this post, my advice is to try it and see what you think. As I will show, there are some really easy ways to implement this in analytics tools, so there is not a huge risk in giving it a try.

The Model

I will admit right off the bat that there are many out there much more advanced in statistics than me. I am sure there are folks out there that can come up with many different visitor scoring models that will make mine look childish, but in the interest of trying to help, I will share a model that I have used with some success. The truth is, that you can create whatever model you want to use is fine, since it is for YOUR organization and not one to be compared to others. There is no universal formula that you will benchmark against. You can make yours as simple or complex as you want.

I like to use the Fibonacci-like approach when I do visitor scoring (while not truly Fibonacci, my goal is to use integers that are somewhat spaced out to draw out the differences between actions as you will see below). I start by making a list of the actions visitors can take on my website/app and narrow it down to the ones that I truly care about and want to include in my model. Next I sort them from least valuable to most valuable. In this example, let’s assume that my sorted list is as follows:

1. View Product Page
2. View at least 50% of Product Video
3. View Pricing Tab for Product
4. Complete Lead Generation Form

Next, I will assign “1” point to the least important item on the list (in this case View Product Page). Then I will work with my team to determine how many Product Page Views they feel is equivalent to the next item on the list (in this case 50% view of Product Video). When I say equivalent, what I mean is that if we had two website visitors and one viewed at least 50% of a product video and the other just viewed a bunch of product detail pages, at what point would they consider them to be almost equal in terms of scoring? Is it four product page views or only two? Somehow, you need to get consensus on this and pick a number. If your team says that three product page views is about the same as one long product video view, then you would assign “3” points each time a product video view hist at least 50%. Next you would move on to the third item (Pricing Page in this example) and follow the same process (how many video views would you take for one video view?). Let’s say when we are done, the list looks like this:

1. View Product Page (1 Point)
2. View at least 50% of Product Video (3 Points)
3. View Pricing Tab for Product (6 Points)
4. Complete Lead Generation Form (15 Points)

Now you have a model that you can apply to your website/app visitors. Will it be perfect? No, but is it better than treating each action equally? If you believe in your scores, then it should be. For now, I wouldn’t over-think it. You can adjust it later if you want, but I would give it a go under the theory that “these are the main things we want people to do, and we agreed on which were more/less important than the others, so if the overall score rises, then we should be happy and if it declines, we should be concerned.”

How To Implement It

Implementing visitor scoring in Adobe Analytics is relatively painless. Once you have identified your actions and associated scores in the previous step, all you need to do is write some code or do some fancy manipulation of your Tag Management System. For example, if you are already setting success events 13, 14, 15, 16 for the actions listed above, all you need to do is pass the designated points to a numeric Success Event. This event will aggregate the scores from all visitors into one metric that you later divide by either Visits or Visitors to normalize (for varying amounts of Visits and Visitors to your site/app). This approach is well documented in this great blog post by Ben Gaines from Adobe.

Here is what a Calculated Metric report might look like when you are done:

Using Derived Metrics

If you don’t have development resources or you want to test out this concept before bugging your developers, I have come up with a new way that you can try this out without any development. This new approach uses the new Derived Metrics concept in Adobe Analytics. Derived Metrics are Calculated Metrics on steroids! You can do much more complex formulas than in the past and apply segments to some or all of your Calculated Metric formula. Using Derived Metrics, you can create a model like the one we discussed above, but without any tagging. Here’s how it might work:

First, we recall that we already have success events for the four key actions we care about:

Now we can create our new “Derived” Calculated Metric for Visitor Score. To do this, we create a formula that multiplies each action by its weight score and then sums them (it may take you some time to master the embedding of containers!). In this case, we want to multiply the number of Product Page Views by 1, the number of Video Views by 3, etc. Then we divide the sum by Visits so the entire formula looks like this:

Once you save this formula, you can view it in the Calculated Metrics area to see how your site is performing. The cool part of this approach is that this new Visitor Score Calculated Metric will work historically as long as you have data for the four events (in this case) that are used in the formula. The other cool part is that if you change the formula, it will change it historically as well (which can also be a bad thing, so if you want to lock in your scores historically, use Ben’s approach of setting a new event). This allows you to play with the scores and see the impact of those changes.

But Wait…There’s More!

Here is one other bonus tip. Since you can now apply segments and advanced formulas to Derived Metrics, you can customize your Visitor Score metric even further. Let’s say that your team decides that if the visitor is a return visitor, that all of the above scores should be multiplied by 1.5. You can use an advanced formula (in this case an IF Statement) and a Segment (1st Time Visits) to modify the formula above and make it more complex. In this case, we want to first check if the visit is a 1st time visit and if so, use our normal scores, but if it isn’t change the scores to be 1.5x the original scores. To do this, we add an IF statement and a segment such that when we are done, the formula might look like this (warning: this is for demo purposes only and I haven’t tested this!):

If you had more patience than I do, you could probably figure out a way to multiply the Visit Number by the static numbers to exponentially give credit if you so desired. The advanced formulas in the Derived Metric builder allow you to do almost anything you can do in Microsoft Excel, so the sky is pretty much the limit when it comes to making your Visitor Score Metric as complex as you want. Tim Elleston shows some much cooler engagement metric formulas in his post here: http://www.digitalbalance.com.au/our-blog/how-to-use-derived-metrics/

Final Thoughts

So there you have it. Some thoughts on why you may want to try visitor scoring, a few tips on how to create scores and some information on how to implement visitor scoring via tags or derived metrics. If you have any thoughts or comments, let me know at @adamgreco.