The Coming Bifurcation in Web Analytics Tools

When John was with Forrester Research last year he had the opportunity to do some work for Google that published some pretty bold claims. Among these was his reporting that “a staggering 53% of enterprises surveyed currently use a free solution as their primary Web analytics tool, and 71% use free tools in some capacity (PDF from Google). At the time I commented:

“When [John] first told me that over half of Enterprise businesses were using free solutions I have to admit I didn’t believe him. In a way I still don’t, but perhaps that’s only because I work with a slightly different sample than he presents. Regardless, John’s report paints a picture of an increasingly challenging market for companies selling web analytics and a new sophistication among end users.”

Increasingly my new partner is looking like some kind of prescient seer, although perhaps not for the reason some of you expect. Without a doubt Google is pushing hard to improve their analytics application, and by nearly all measures they are doing a phenomenal job. As I said back in November I personally believe their “Analytics Intelligence” feature is brilliant, and I have little doubt that we’ll continue to see little improvements here and there over the coming year.

But as much as I love Google Analytics for what it does, I am also willing to be honest about what it does not do and what it is not. Google Analytics alone is simply not enough for truly sophisticated web analytics.

Despite John’s findings at Forrester, and despite the fact that Google Analytics is easily the most widely deployed web analytics solution ever built, there are clearly limits to what Google Analytics is capable of today. What’s more, there is nothing wrong with having limits … what is wrong is trying to be all things to all people, which is what this post is really about.

At Analytics Demystified we have been talking over the last six months to an increasing number of companies that are considering dropping their historical vendor, almost always in favor of Google Analytics. And at Analytics Demystified we don’t do that much work with small, mom-and-pop shops … these are global organizations, name brands, and market leaders in their respective categories. Most of these companies are spending well-over $500,000 per year on analytics technology, and a handful are spending double that.

What’s more, all of these companies have multiple dedicated resources for web analytics. These companies, in many cases, are freaking awesome at putting their tools to work, and in all cases understand that web analytics is a “people” thing, not a “technology” thing. So what the heck is driving them towards Google’s waiting arms?

Limits.

It turns out there are limits to the amount your average business user is willing to invest in learning web analytics tools. As more companies begin to truly take a strategic approach towards web analytics, many of them are realizing that their business users are simply not “getting” the fee-based solution they’ve invested so heavily in. The business users have more or less found their limit, and hit the wall, and are balking at the amount of time it actually takes to learn and become proficient with these tools.

Apparently in an effort to further differentiate themselves from Google Analytics, the paid guys have inadvertantly made their technology so complex that few people in the business are actually willing to use it.

Oops.

Having worked at WebSideStory in the past I have to admit I cringed when one business user complained that [market leading vendor X] “was just too complicated” and that she “really, really missed using HBX because it was simple.” But this is a story we are hearing over and over and over … to the point where I am having to revise my entire opinion about Google Analytics place in the true Enterprise, which I’m happy to do …

… except.

A problem with the wholesale shift to GA arises when we go to the dedicated analysts and consulting teams who actually do get and use the paid solutions, pretty well in most cases. Suggesting to them that they might try and get by solely on Google Analytics is kind of like telling LeBron James that he needs to do his job with only one leg, one arm, and a blindfold on if he wants to keep playing ball.  He’d probably still be able to drop 30 on the Knicks, but he certainly wouldn’t be happy about it.

I don’t personally know a single analyst worth their weight in salt who would be happy and willing to standardize completely on Google Analytics, at least not today. Despite awesomeness galore, the decreasing list of things that GA doesn’t do is pretty important to these sophisticated users. True visitor-level segmentation, real flexibility in reporting on custom data, the ability to define custom metrics and dimensions, true data integration … it’s not an infinite list, but it’s still pretty long by my estimation.

And while the list of third-party applications to provide additional functionality to Google Analytics — for example ShufflePoint and their wonderful use of the GA APIs — we have still not seen a solution emerge that confers all of the necessary functionality that “professional” web analysts need to do their jobs well. In my experience there is nothing worse than knowing how to answer a question but not have the tools in place that you need to make the necessary connections.

Notice that I’m not saying that the alternatives to Google Analytics necessarily have these features. You don’t have to spend much time following the Web Analytics Forum or the Twitter #measure tag to see complaints about how hard it is to do stuff that should be pretty simple. But for the most part this rich functionality can be found in the add-on ad hoc exploration tools (e.g., Omniture Insights, Coremetrics Explore, Webtrends Visitor Intelligence, Unica NetInsight, etc.), and it turns out that when you’re competing on web analytics these features are pretty important.

So what am I saying, and what did I mean by “bifurcation” in this post’s title?

I believe that we are about to see an increasing number of companies in the coming year drop their paid vendor’s “basic solution” in favor of Google Analytics and, at the same time, seriously consider adding their vendor’s high-end offering. More specifically thanks to advances in “universal” (sic) tagging, the increasing cry from business users to “get us something simple that we can use”, and the true and present need for experienced operators to have a robust data exploration tool at their disposal, I think we’ll see an increasing number of “Google Analytics + Omniture Insights” implementations.

Some caveats:

• I am not saying that all companies will drop their paid vendor in favor of Google Analytics, mostly because “all companies” never do anything;
• I am not saying that all companies should drop their paid vendor in favor of Google Analytics, or even that companies should drop their paid vendor at all, especially if you have a pretty solid web analytics strategy in place;
• I am definitely not saying that I believe companies can manage a sophisticated web analytics operation using Google Analytics alone, although this statement hinges on the definition of “sophisticated”;
• I am not evangelizing for Omniture Insights, even thought I used to work at Visual Sciences and continue to use OI thanks to the good graces of Omniture/Adobe;
• I am not evangelizing for Google Analytics, even thought I do think the GA team has made amazing advances over the last 12 months;

The final caveat is that I am only using Omniture Insights in the description below as an example — you can substitute any of the solutions I listed above just as easily, or even use SAS if you’re ready for the coming revolution in web analytics. Heck, if you’re super-motivated, you can take Hiten Shah of KISSmetrics suggestion and build your own clickstream data warehouse and analyze the results using Tableau.

Regardless of the technology you choose, the bifurcated solution looks kind of like this:

• For your business users you simply do an awesome job implementing all the great new functionality present in Google Analytics;
• Using very simple Javascript libraries you “copy” the data you’re sending to Google Analytics and pass it along to Omniture Insights (OI);
• At the same time you add whatever other information you need to pass to OI, either because GA can’t handle it or you’ve filled all your custom variables;
• In OI you transform the data to match what GA is doing as closely as possible, knowing full well the data will never match because of GA sampling and the reality of our industry;
• In OI you add to the data with whatever you need, either via transformation, lookup tables, custom metrics and dimensions, whatever …

With these technologies in place you now have two things:

1. A very appropriate solution for your internal business users, one they will likely embrace thanks to it’s simplicity, it’s beauty, and it’s Googliness;
2. A very powerful solution for your web analysts that is largely based on what your business users are looking at.

The way the solution set works practically within the business:

• Business users get training on Google Analytics, which is surprisingly easy to provide, and if you’re big enough the rumor is that Google’s own evangelist will come visit with you (fun!)
• Business users get used to the idea that the numbers in GA are not 100% perfect, especially in high-volume situations where GA is sampling;
• Business users follow the age old advice to “manage based on trends” and use some of the slickness that is GA to identify problems and opportunities;
• When the business finds something interesting they ask the analytics group to work with them to look more closely and provide analysis (not reports);
• When the business needs “more accurate” numbers they ask the analytics group to provide reporting from the complete set of data (normal accuracy and precision caveats still apply);
• With their newly gained free time, the analytics group can become more of a proactive analytics service organization and less of a barrel full of “report monkeys”;

Yes this involves some internal education, but c’mon people, all web analytics involves internal education. You’ll need a clear explanation about the “goodness” of the GA data in high-volume (e.g., sampling) situations; you’ll need to provide training on Google Analytics, but there are some amazing people out there who can help you; and you’ll need to manage two vendor relationships … although if John’s data from Forrester is correct, 71% of you are already doing that!

Clearly this solution is not without risks, but from where I sit, I am having more and more trouble putting together a viable and workable alternative. Web analytics is becoming an increasingly critical function across the Enterprise and awareness of the solution set is bubbling up more rapidly than ever. As this happens, an increasing number of internal stakeholders are starting to ask for direct access to web data.

But the fee vendors, again for pretty good and obvious reasons, have evolved their base solutions to an unprecedented level of complexity, especially when you look across many vendor’s “complete” base solution (e.g., Webtrends 9 plus the requirement to use Webtrends Live for a lot of stuff, Omniture SiteCatalyst plus Omniture Genesis, etc.) Nobody is blaming them for the push upstream … especially since nobody I know could think of an alternative to differentiate their solutions from the 8,000,000 lb gorilla that Google Analytics has become.

At the end of the day in many, many cases you end up with business users frustrated by their inability to effectively and efficiently self-serve, and analytics professionals frustrated by the amount of time they spend pushing out basic reports. Quickly the situation becomes what is politely described as “inefficient” or,  in more colloquial terms, FUBAR. You choices are  then to A) lump it and suffer or B) do something about it.

I’m not a big fan of suffering.

The bifurcated solution, if you think about it, is actually pretty awesome. You get the best of both worlds, and one of the solutions costs you nothing and so (hopefully) frees up budget to hire more and better people to manage your web analytics efforts. I’d rather see a company put $500,000 equally towards an ad hoc analysis engine and smart people to run it than the case I see too commonly today where the lion’s share of that $500,000 going directly to buy a solution that is not meeting the needs of the business.

What do you think?

If you’re a company of any size and history of investment with any of the big U.S. or European vendors, and if you’ve been considering something similar, we’d love to hear from you. While we’re already providing guidance to some pretty large clients making this move we are always eager to collect additional data as an input to our thinking. We’re also happy to hear from consultants and vendors who have a clearly vested interest in the outcome I’ve described. And yes, if you need to bitch at me for suggesting that Google Analytics is anything short of manna from heaven, I suppose I’ll approve those comments as well.