Have you read Lee Sherman's piece on Actionable Analytics?

I’ve never met Lee Sherman, Senior Vice President and General Manager for Global Solutions at AvenueA/Razorfish but I had a chance to chat with him recently about the AvenueA document “Actionable Analytics” that the company recently began distributing. Gary Angel’s eloquent rant on the “myth of actionability” aside, I found that I agreed with much of what Lee had to say in his document.

I say “much” but not all, especially not his comments about web analytics applications not being able to track advertising from end-to-end, nor his comments about web analytics applications not being good at tracking Rich Internet Applications. To his credit, Lee admitted that he had not had a very good look at Visual Site and Platform 4, but Seattle is only a short drive from Portland so hopefully I can fix that before long.

Some of the things that really interested me in the document were Lee’s discussion of RIAs. We talked about his assertion that “rich internet applications are the best vehicle to engage users online and build brand loyalty,” an assertion that was made in a PPT that AvenueA uses to support the document. I was really interested in this statement, especially the use of the word “best” since I’ve never seen data to confirm a broad statement like that, and also the use of the word “engage” since I’m really into figuring out how companies will measure engagement online.

Lee admitted that “best” was perhaps too assertive of a term but that anecdotally, in AvenueA tests, in almost every case the RIA version of a process outperforms the HTML version of the process. When I asked about the performance criteria, Lee said they were looking for differences in subsequent perception of brand, intent to purchase, and actual transactions. Clearly, Lee is from Avinash’s school of the trinity and believes in asking visitors directly to augment observed behavioral data.

We then talked about how RIA’s should be measured, a conversation I hoped to promote in the Web 2.0 Measurement Working Group. Lee said he felt that that are no standards for RIA measurement yet and that most companies he works with are more worried about what they can measure than what they should measure. He believes we’re more likely to see “custom” measures, perhaps supplemented by online surveys designed to gather attitudinal data after visitors interact with RIAs.

Not a bad idea, really, but I sure would feel better if we could just agree on a handful of easily understood measurements for success to apply to RIAs. Wouldn’t you?

Regarding Lee’s definition of “engagement”, he cited a few measurements that I haven’t really heard from anyone other than AvenueA: Page IQ and scrolling analysis. The former is basically the percentage of visitors interacting with a page at all (i.e., not closing the browser, not clicking the back button) and the latter is an assessment of visitor’s scrolling down a page. Both measures are clearly page-centric, not that that is bad, but neither seem to easily lend themselves to the concept of visitor engagement.

All-in-all the report is worth a read if you haven’t already seen it. You have to send Avenue A some personal information to get the download but hey, few good things in life are truly free. You can request the download here.

Thanks to Lee for taking the time to chat with me and to David Deal for setting up the call.