Sad to say, I partially agree with Brandt Dainow

Readers who are enthusiastic members of the web analytics community are by now familiar with Brandt Dainow and his sometimes antagonistic missives published at iMediaConnection. While I try pretty hard to follow the old “if you can’t say something nice” rule I occasionally fail in my efforts. Perhaps the best evidence of my failing was my calling Brandt Daniow insane when he suggested that Google Analytics version 2.0 was “simply a quantum leap above any other analytics product on the planet.”

While I firmly believe that Google Analytics is a great, valuable, and appropriate application for a wide range of needs, I think that Dainow’s “quantum leap” claim and statements like “What Google has done is simply take every feature in every product on the market and put them all into one system, and then make it available for free” are so obviously hyperbolic that they beg criticism (which Mr. Dainow got in spades from many within the analytics community.)

Dainow has since turned on Google Analytics, more recently pointing out what he describes as “disturbing inaccuracies behind Google Analytics” and again getting our  attention with irresponsible statements like “Google Analytics is different from other products in that it has been intentionally designed by Google to be inaccurate over and above the normal inaccuracies that are inevitable.“

Oddly enough, his rant about Google Analytics included some statements that rubbed members of the Web Analytics Association the wrong way.  When folks like Jodi McDermott commented on the article and questioned some of Dainow’s assertions, Brandt did what any normal person would do …

… he wrote a nasty follow-up piece critical of the Web Analytics Association and the WAA Standards Committee!

I will let you read his piece yourself, but the two-sentence summary of Dainow’s opinion is that “the work of the WAA standards committee is a disaster for the web analytics community. It will take years to undo the damage and create proper precise standards that can be implemented in software. The WAA “standard” is not a standard, it’s just second-rate muttering.“

Clearly Dainow is not worried about making friends in the web analytics industry.

I personally am a big fan of the Web Analytics Association.  I am pretty loyal to some of the current Board of Directors, I’ve done a bunch in the past to support the WAA and am about to announce more of the same, and I’ve even gone out of my way to help promote the work of the Web Analytics Association Standards Committee.  So it is was with great trepidation I wrote this article’s title … but I find myself agreeing with one small part of Dainow’s otherwise unnecessary rant.

Towards the end of his article, right before he declares that some pretty nice people’s work has been little more than time wasted, he says this:

“The WAA should be setting the agenda, not following the crowd. The task of the WAA standards committee should be to determine how web analytics metrics should be calculated in order to achieve the highest degree of precision possible. The WAA should be laying out the roadmap for the way things should be. It then falls to the vendors to bring their software into line.”

I more or less made this same comment, although I like to believe I used a great deal more tact, when I commented on the original Web Analytics Association Standards published under the direction of former Director Avinash Kaushik back in August 2007.

At the time I preferred to focus on the reality of the situation–the fact that the WAA had proposed a set of standard definitions that, for good or ill, were better than anything else out there.  Instead of being openly critical of the definitions as written, I preferred to ask the question, “Now that we have these definitions, what are we going to do with them?”

While my call for a web analytics standards compliance matrix has since been answered by all of the major vendors except for Omniture, I personally don’t believe that the Standards process is serving the needs of our community as best possible.  We all continue to be vexed by a lack of standard definitions, a situation that will likely get worse with the decline of the web analytics economy.

Not having participated in the process of drafting the WAA Standards I can only express gratitude towards those members of the community who have volunteered  their valuable time for this work.  In my humble opinion, people like Jason Burby and Angie Brown are to be congratulated for their efforts, not denigrated and accused of having set our industry back into the dark ages.

But, in the spirit of having an open mind and building consensus, I would be interested in hearing my reader’s collective thoughts on Dainow’s point that the WAA should be setting standards without regard to their practicality today. Put another way, should the Association have written definitions that would be robust and useful in an analytics context and then presented that guidance to the entire community–vendor, consultant, and practitioner alike–saying “this is the result we should all be working towards.”

For example, should the WAA have been more explicit in their definition of a “visit” and proclaim that a visit is terminated after 30 minutes of inactivity, instead of saying “if an individual has not taken another action (typically additional page views) on the site within a specified time period, the visit will terminate by timing out.”  Being explicit about the timeout duration would make a clear statement about our collective expectation for the definition of a visit, and any technology or analysis that choses to use a timeout other than 30 minutes would also need to justify their decision to eschew the WAA Standard for another value.

I know that the WAA is doing the best they can, and I am enthusiastic about the work Angie, Judith and their fellow volunteers have all been doing.  But I do think Dainow’s assertion that standards should be set based on overall value to the community in the long run, not necessarily the near-term practicality, is worth exploring.  Taking this approach would definitely penalize some vendors and reduce their self-generated “compliance score” but it does kind of make sense to be working collectively towards a more precise set of definitions we can all work from.

Doesn’t it?

These are the kinds of conversations that aren’t just magically resolved and so I’m sure we’ll have to add this to the list of issues worthy of discussion the next time we all meet.  I’m sure it will come up at some of the upcoming vendor events, in San Jose at Emetrics, and likely at our own web analytics conference, the X Change (where last year Forrester analyst John Lovett led a conversation on the topic.)

As always I welcome your thoughts, feedback, open disagreement, pointing out flaws in my logic, etc.  I consider myself fortunate to have such thoughtful and experienced analytics practitioners among my most loyal readers and sincerely hope Dainow’s otherwise disturbing rant will lead to something of value for our community.