The engagement metric, defined (part IV in a series)

For those of you keeping track at home, this is the fourth in what will likely be a five-part series on calculating an “engagement metric”. The first three posts are here:

• Part I
• Part II
• Part III

Originally I had postulated that an engaged visitor, at least on my web site, can be characterized as follows:

1. The visitor views “critical” content on the web site
2. The visitor has returned to the web site recently
3. The visitor returns directly to the web site some of the time
4. Some high percentage of the visitor’s sessions are “long” sessions
5. If available, the visitor is subscribed to at least one available site feed

Basically, the final calculation, one revised thanks to the valuable feedback of dozens of folks, is essentially the same with a few slight modifications. The final goals for my site, goals easily tweaked for any site, are as follows.

Well-engaged visitors will:

1. View a relatively large number of page views in a given session
2. Have visited the site in the last four weeks
3. Have relatively long sessions
4. Come directly to my site or come from a “Eric Peterson” branded search
5. Be reading my weblog in addition to non-blog content
6. Buy one or more of my books through my web site

As you can hopefully see, the first item in my original list (view “critical” content) has been softened somewhat. While the act of purchasing is necessitated by viewing critical content (my “thank you” page) ultimately I agreed with several reader comments that the a priori definition of visitor goals would skew the metric and reduce the metric’s ability to tell me about all of the content on my web site. Thanks to Victor and others for hammering this home.

Given all this, the visitor engagement metric is composed of six sub-metrics, each of which can be examined individually to provide context to the larger calculation. The six sub-metrics are:

1. Click-Depth Index: Percent of visitor sessions of “n” or more pages
2. Recency Index: Percent of visitor sessions occurring in the last “small n” weeks
3. Duration Index: Percent of visitor sessions of “n” or more minutes
4. Brand Index: Percent of visitor sessions originating directly or originating from search engine searches for terms like “eric t. peterson” and “web analytics demystified”, etc.
5. Blog Index: Ratio of blog reading sessions to all sessions
6. Conversion Index: In this case, session- or order-based conversion

Keep in mind, engagement is a visitor-based calculation, one designed to look at the lifetime of visitor sessions to the web site. So that the engagement of any visitor is a function of their lifetime of visits. Yeah, this assumes some stability in cookies so always use first-party cookies.

The final calculation is simply a summation of the component indices divided by the total number of components which yields a simple percentage:

If you’re looking across multiple visitors, you would read this as “the average visitor is just under 27 percent engaged, as defined by X, Y, and Z.” If you’re looking at a single visitor you can break engagement down on a session-by-session basis, watching for increases and decreases in the visitor’s engagement over time. In aggregate, visitor engagement becomes a very powerful but elegant key performance indicator that tells you a great deal about the make-up of your audience.

Once you decide that you need more information about the basis for an increase or decrease in visitor engagement, and assuming you have the right technology powering your analysis, you would simply visualize each of the core components over time:

As Clint commented in my last post, there is a surprising stability in each of the components, which is in my mind what you’re looking for. I want to see the variation show up when I examine engagement against my business-critical dimensions (referrer, campaign, page, search term, etc.)

When you analyze the visitor engagement calculation against all of your site visitors, you’re looking for a more-or-less normal distribution. This distribution is spiky because of the calculation, but if you’re able to drill-down, you should see something like this:

(The bars that exceed the visualization’s scale represent peaks that occur as visitors achieve 100% of sessions for 1, 2, 3, 4, and 5 of the engagement calculation’s core components. If you want to see this image at 100% scale let me know …)

Another way I can think about this is to use a scatter-plot, basically showing the same thing but easier to visualize differences as you drill-down into specific dimensions:

All of these calculations actually become relevant when you actually apply them to a dimension of data. Here, for example, is visitor engagement mapped to blog posts from my and Avinash Kaushik’s weblog:

Pretty cool, huh? I mean, it’s no great surprise that Avinash’s 2007 Web Analytics Predictions post has the highest visitor engagement score in this image when you think about all of the follow-up predictions his original post spawned. But boy-howdy, isn’t it nice to see that in a metric that you can understand and actually use?!

Here is the visitor engagement metric applied to some of the referrers to my web site:

No great surprise again that Feedburner, Technorati, and WordPress are driving visitor engagement given that they are likely to be driving visitors maxing out their blog index score. But what about the folks at ROI Revolution, sending me visitors who are on average over 30% engaged, or Blackbeak and the folks at Conversion Chronicles, sending me visitors who are as engaged as my 27 percent site-wide average?

Arrrrrrr, indeed!

Finally, and I know that I showed this already but I just think it’s damn sexy, I can map visitor engagement against any geographic dimension in my system (continent, country, city, state, zip code, DMA, etc.) to see where I might want to focus my local marketing efforts in the future:

You two people in Midland, Michigan, get ready for an onslaught of Analytics Demystified promotions!

Oh, some random notes:

• You can add non-page view events (RIAs, AJAX, Flash, etc.) into the calculation easily. I don’t have much of that on my site but I have an “Event Index” calculation that can be added for sites heavily leveraging these types of applications.
• You can add the “social media index” that I discussed in my last post just as easily as you can add content-based indices for retail, customer support, business-to-business, or content.
• You can take or leave my idea of scoring the brand index against specific search terms. Visual Site gives me a really easy way to do that and I believe that engagement is very much a function of brand awareness, something notoriously difficult to measure in any practical way.
• Visual Sciences customers interested in deploying the visitor engagement metric should contact me directly via normal company channels. I have pretty much everything you need to get up and running with this in a ZIP file and I’d be happy to talk you through the process.

As always, I welcome your comments and feedback on the engagement calculation and anything else that comes to mind. In the next (and perhaps final) installment I will cover Clint’s inevitable complaint of “What the heck happened to all the great ‘social media’ stuff?!?” as well as talk about some specific applications of the engagement metric.