Visitor Engagement + comScore = Audience Engagement!

About six months ago the management team at comScore approached me with some questions about my Visitor Engagement calculation and the Analytics Demystified engagement framework. Their Chief Research Officer, Josh Chasin, had taken an interest in my work and wondered how it may be extensible across multiple properties using the comScore dataset.

It was an excellent question, and today I’m happy to give readers a preview of what we believe to be an excellent answer. Today we’re announcing a measure of Visitor Engagement that, thanks to comScore, can be used to compare levels of engagement across multiple properties in a similar category.

Brand Marketing’s New Measure: Audience Engagement

Audience Engagement is a simple modification of Analytics Demystified’s Visitor Engagement calculation that focuses on the core site behavioral attributes, measured through the comScore panel. If you remember, the Visitor Engagement calculation is:

Σ(Ci + Di + Ri + Li + Bi + Fi + Ii)

The components of the Visitor Engagement calculation are:

• Click Depth Index: Captures the contribution of page and event views
• Duration Index: Captures the contribution of time spent on site
• Recency Index: Captures the visitor’s “visit velocity”—the rate at which visitors return to the web site over time
• Brand Index: Captures the apparent awareness of the visitor of the brand, site, or product(s)
• Feedback Index: Captures qualitative information including propensity to solicit additional information or supply direct feedback
• Interaction Index: Captures visitor interaction with content or functionality designed to increase level of Attention the visitor is paying to the brand, site, or product(s)
• Loyalty Index: Captures the level of long-term interaction the visitor has with the brand, site, or product(s)

(More information about the measure of Visitor Engagement, including the details behind the calculation and several example use cases, can be obtained by reading the white paper that Joseph Carrabis and I recently published, Measuring the Immeasurable: Visitor Engagement which is freely available on this web site.)

The Audience Engagement simplifies Visitor Engagement by applying a “zero weighting” to the Brand, Feedback, and Interaction indices. By removing these values from the core calculation we are left with Click-Depth, Duration, Recency, and Loyalty:

Σ(Ci + Di + Ri + Li)

In English:

“Audience Engagement is a function of the number of clicks a visitor generates at a site, the amount of time they spent at the site, the frequency at which they return to the site, and their loyalty to the site as a member of the category for all of the sessions to that site during the reporting period.”

We’ve selected these four indices for one very simple reason: When scored using category-level thresholds (with the exception being the Loyalty Index, see below) comScore is able to automatically generate Audience Engagement values and engagement distributions across all of the sites they track.

The result is unique view into the relationship visitors have with the thousands of web sites comScore tracks around the globe. Now, for the first time ever, marketers and advertisers are able to gain insights into the level of engagement using a much more robust measure than session duration, page views, or recency alone.

Using Audience Engagement we can say with a high level of certainty that a greater percentage of Internet users find CNN more engaging than MSNBC and Yahoo! News:

More importantly we can also say that CNN has a larger population of “highly engaged” visitors to their site (22.5% of visitors at CNN versus 15% at MSNBC and less than 10% at Yahoo! News.) We believe that assessment of the audience distribution will provide advertisers an entirely new way to evaluate sites, focusing on audience quality over more simplistic measures of quantity.

This same type of analysis applied to popular network sports sites yields similarly interesting insights:

Here we can see that ESPN, while trailing Yahoo! Sports across all traditional measures (page views, sessions, minutes spent, active days) dominates Yahoo! from an Audience Engagement perspective. A closer examination of these two sites shows that ESPN’s dominance is driven largely by the frequency at which their audience members return to the site (Recency Index of 47.2% versus Yahoo! Sports at 27.0%) — an insight that has clear value to advertisers looking to create brand awareness and drive brand impressions across a sports-minded audience.

While comScore and Analytics Demystified are still working on how this data will be packaged and presented, another way of visualizing the relationship between two sites or a site and the category average is using a spider chart:

This chart visually tells the same story as the table above — ESPN has a higher level of Audience Engagement (bigger footprint) that is largely driven by Loyalty and Recency.

We believe that brand advertisers, advertising planners, and marketing managers will be able to use this data to make better decisions during the ad planning and media buying process. The whole debate over the definition of engagement manifest largely from advertisers desire to find more engaged audiences juxtaposed against a lack of faith in the simple measures being proposed as proxies for engagement. Thanks to comScore, these simple measures are about to become a thing of the past, giving way to a significantly more robust measure of the level of Attention audiences are paying at advertising powered sites around the world.

Interpreting Individual Data Points

In case you don’t want to spend the time reading the 50 page white paper I wrote recently on the subject with the mathematician and cultural anthropologist Joseph Carrabis, I’ll provide a brief summary of how the data comScore is reporting can be used.

Here is a sample of sites from comScore’s automotive category:

The first line in this table says that 42.8% of the audience to KBB.com is appreciably engaged with the web site. Engagement at KBB.com is largely driven by visitors clicking deeply into the site and spending an appreciable amount of time doing so, with nearly 85% of audience members exceeding the category Click Depth threshold and over 60% exceeding the duration threshold. Finally, using the distribution data, we can also see that 63% of the audience is highly engaged versus less than 3% who are only poorly engaged.

Audience Engagement data provided by comScore can also be used in a comparative context. Looking at the most and least engaging sites in this group, the data suggests that the audience going to KBB.com is over 400% more engaged than the audience going to About.com Autos (42.8% versus 8.5%.)  This is not to say that advertising at About.com Autos is a bad idea — over 90 percent of the site’s audience appears to be moderately engaged and in some instances a moderate level of engagement may be exactly what the campaign is looking for.

A Technical Note about Audience Engagement’s Loyalty Index

In the Audience Engagement calculation, the Loyalty Index is calculated differently than in the Visitor Engagement calculation because of an advantage conferred by the comScore system. Instead of simply counting the number of times a visitor has returned to the site as we’re forced to do using a site-centric data model, comScore allow us to better approximate loyalty as more commonly used: a measure of your likelihood to prefer a single site or brand over all others in the category. This model is essentially a “share of requirements” model used traditionally in the brand advertising industry and is calculated as:

Li(AE) = Visits to Site / Visits to All Sites in the Category

So, for example, if a comScore panelist is going only to eBay in comScore’s “Auctions” category, their Loyalty Index for eBay would be 100%:

Li(AE) = 10 visits to eBay / 10 visits in the “Auctions” Category

Conversely, if another visitor goes to eBay half the time and Bidz.com half the time, their Loyalty Index for eBay would be 50%:

Li(AE) = 5 visits to eBay / 10 visits in the “Auctions” Category

The result is a distribution of Loyalty Index scores for auction sites tracked by comScore in September that looks like this:

As you can see, eBay’s Audience Engagement component indices are higher than those of their competitors, but their Loyalty Index is much higher and tells us that nearly visitors in this category strongly prefer eBay to their competitors.

One of the challenges comScore and Analytics Demystified face regarding the Loyalty Index is the refinement of categories. Some categories like “Auctions” are well defined and represent logical competitors in a sector; others, like “News/Information” include diverse sites like Weather.com, Discovery.com, and Court TV Online. Over time we hope to refine these categories in partnership with comScore clients to provide the most accurate view of category loyalty possible. If you’re interested in participating in this work, please contact me directly.

Next Steps for comScore and Analytics Demystified

This is the first time we’ve been able to apply the Analytics Demystified Engagement construct to a syndicated audience data base.  We’re just announcing this work today, but we can already see possibilities for the measure’s evolution. Potential next-generation enhancements could include:

• Allowing comScore clients to provide a set of branded search terms to support the inclusion of Visitor Engagement’s Brand Index (Bi)
• Allowing comScore clients to provide a set of key site interactions designed to promote visitor Attention, supporting the inclusion of Visitor Engagement’s Interaction Index (Ii)
• Incorporating third-party data sources measuring more qualitative aspects of the audience relationship with the site, supporting the inclusion of Visitor Engagement’s Feedback Index (Fi)
• Allowing comScore clients to define their own competitive set in order to drill down into a more specific engagement profile in support of the advertising sales process
• Providing comScore clients access to the details behind the Audience Engagement calculation for their site and category
• Providing comScore clients custom access to Audience Engagement data, to provide a measure of Visitor Engagement in situations where the web analytic technology deployed does not support direct measurement

These are just a handful of examples of where this data offering can go. We’re presenting this model and starting the conversation because we want to hear from you. Regardless of whether you’re a current comScore or Analytics Demystified client, we would love your feedback regarding the calculation, the data, and the type of insights Audience Engagement is likely to provide to your organization.

Want to Know More about Audience Engagement?

Any reader of this blog knows that I have a passion for talking about the new measures of success on the Internet. I’m tremendously excited about this announcement and happy to talk if you’re interested in how you might be able to leverage Audience Engagement data.

Also, don’t hesitate to contact us if you have concerns about how we measure Audience Engagement or, in the extreme case, don’t think engagement can be measured at all. I firmly believe that the measures of Visitor and Audience Engagement I have proposed and the work I’ve done with Mr. Carrabis and now with comScore are only the beginning of the search for more useful measures of success on the Internet. Because these measures attempt to approximate something we agree is difficult to quantify, we believe that these measures will evolve over time; nothing is set in stone.

But we also believe that Visitor and Audience Engagement are better measures than “page views” and “average time spent” and far more useful to the measurement industry as a whole than simply sticking our head’s in the sand and exclaiming “engagement is an excuse” or worse, taking a Luddite’s view and declaring that complex measures are destined to fail.

For the time being, comScore is previewing additional details on the measure of Audience Engagement with their clients selectively.  If you’d like more information about how to be added to comScore’s list, or would like to discuss the measure of Audience Engagement with me, please email me directly and we can arrange a time to chat.