Have you read Lee Sherman's piece on Actionable Analytics?
I’ve never met Lee Sherman, Senior Vice President and General Manager for Global Solutions at AvenueA/Razorfish but I had a chance to chat with him recently about the AvenueA document “Actionable Analytics” that the company recently began distributing. Gary Angel’s eloquent rant on the “myth of actionability” aside, I found that I agreed with much of what Lee had to say in his document.
I say “much” but not all, especially not his comments about web analytics applications not being able to track advertising from end-to-end, nor his comments about web analytics applications not being good at tracking Rich Internet Applications. To his credit, Lee admitted that he had not had a very good look at Visual Site and Platform 4, but Seattle is only a short drive from Portland so hopefully I can fix that before long.
Some of the things that really interested me in the document were Lee’s discussion of RIAs. We talked about his assertion that “rich internet applications are the best vehicle to engage users online and build brand loyalty,” an assertion that was made in a PPT that AvenueA uses to support the document. I was really interested in this statement, especially the use of the word “best” since I’ve never seen data to confirm a broad statement like that, and also the use of the word “engage” since I’m really into figuring out how companies will measure engagement online.
Lee admitted that “best” was perhaps too assertive of a term but that anecdotally, in AvenueA tests, in almost every case the RIA version of a process outperforms the HTML version of the process. When I asked about the performance criteria, Lee said they were looking for differences in subsequent perception of brand, intent to purchase, and actual transactions. Clearly, Lee is from Avinash’s school of the trinity and believes in asking visitors directly to augment observed behavioral data.
We then talked about how RIA’s should be measured, a conversation I hoped to promote in the Web 2.0 Measurement Working Group. Lee said he felt that that are no standards for RIA measurement yet and that most companies he works with are more worried about what they can measure than what they should measure. He believes we’re more likely to see “custom” measures, perhaps supplemented by online surveys designed to gather attitudinal data after visitors interact with RIAs.
Not a bad idea, really, but I sure would feel better if we could just agree on a handful of easily understood measurements for success to apply to RIAs. Wouldn’t you?
Regarding Lee’s definition of “engagement”, he cited a few measurements that I haven’t really heard from anyone other than AvenueA: Page IQ and scrolling analysis. The former is basically the percentage of visitors interacting with a page at all (i.e., not closing the browser, not clicking the back button) and the latter is an assessment of visitor’s scrolling down a page. Both measures are clearly page-centric, not that that is bad, but neither seem to easily lend themselves to the concept of visitor engagement.
All-in-all the report is worth a read if you haven’t already seen it. You have to send Avenue A some personal information to get the download but hey, few good things in life are truly free. You can request the download here.
Thanks to Lee for taking the time to chat with me and to David Deal for setting up the call.
[…] To expand a bit on #2, Eric Peterson highlighted a paper from the folks at AA|RF which I had the chance to read. Oddly enough, the paper directly mirrors many of our own studies and research and in other cases parallels a massive volume of work I’m involved in from day to day. Like Eric, I didn’t agree with all the points in the paper but I’d strongly encourage everyone to read it. However, I did agree with Lee Sherman’s assertion that web analytics packages are limited as holistic media measurement tools (at least today). Having worked in digital media and web analytics for 10 years, I’ve gained an intimate level of knowledge about the strengths and weaknesses of tracking, reporting, testing, analysis and optimization between these related industries. Without going into great detail, there are a few noteworthy tidbits between web analytic and digital media tools: A. Web analytic tools are superb for website tracking and reporting, and are – generally speaking – still growing in their data mining, classification and segmentation abilities. B. Web analytic tools are not media serving, media testing or media optimization platforms, including OMTR/TouchClarity. They are not ad tag management solutions and they have very, very limited visibility into media reporting and media measurement. To date their media measurement has predominantly been relegated to circumstances where a click parameter can be passed, although view attributions are now being accounted for in a limited manner with select vendors. C. Select web analytic tools allow some “integration” with ad server data for reporting purposes (e.g. Omniture and DoubleClick) and provide visibility into both click-based and view-based traffic and conversions, but they do not replace the need for ad server tracking or the sophisticated media controls those ad server tools provide (e.g. sequencing, creative and message management, media planning and campaign management, targeting and other media controls such as recency, frequency and more). […]