Matt Belkin of Omniture: Web Analytics is Easy!
Matt Belkin of Omniture recently posted on a few of the pitfalls companies fall into when deploying web analytics. I was pretty surprised to see Matt, someone who was worked in this field nearly as long as I have, make the following statement:
“Analytics success is all about building a baseline for performance (your KPI trend), and trying new things to improve on this baseline. That’s it! That’s why I think it’s easy. I know other bloggers have argued that analytics is hard, but I’ve done this for a living and I can tell you that it’s not.”
Ironically enough I have been meeting many of Omniture’s largest customers recently, none of whom seem to think web analytics is easy. They universally have some difficulty associated with technology, people, or process—the triumvirate that is truly “web analytics”—and I suspect many of them had the same response I did when I read Matt’s statement above.
I quickly scribbled out the following response late last night but for some odd reason it has not been approved yet. I figured I’d post my comment here so that Matt and his customers would have a chance to read an opposing point of view.
“Hysterical! I talk to Omniture customers constantly who complain about how hard it is to do the most basic things like calculate bounce rate, integrate data using your Genesis platform, make sense of your reports, and even just get the data they need when they need it.
Perhaps the problem that you and people like Stephane Hammel are having with my statement is something called the echo chamber effect. You say something for so long, and your buddies all repeat it, that eventually you ignore the reality of the situation and begin to believe something that is clearly not true. Seth Godin accused me of doing this once (he was wrong, it turned out, people are deleting cookies … you’ve said so yourself!)
But you’re wrong, Matt. Web analytics is hard. Ask your customers, they’ll tell you.
It’s not just hard to improve your baselines, it’s hard to implement code properly, it’s hard to understand reports and definitions, it’s hard to find qualified staff to run these applications, it’s hard for HR to stomach the salaries we are asking for, it’s hard to train newbies, it’s hard to produce quality analysis based on only quantitative data, it’s hard to get management to listen, it’s hard to make management understand, it’s hard to select a good vendor when so many are failing, it’s hard to know if and when to migrate off of HBX, it’s hard to know which low hanging fruit to pick, …
You get the picture.
You make my point yourself in your post. If there are multiple versions of the truth, it’s hard to know who to trust. If there are multiple systems, it’s hard to know which system’s “click” is the right click to count. If you yourself have had to spend “countless hours trying to reconcile differences” in data, how is that “easy?”
In a way I’m happy you wrote this post because it reinforces everything I say when I travel the globe and meet with your biggest customers. They say “Our vendor says this is easy … there must be something we’re not getting.” I say, “Why would you expect your vendor to tell you that web analytics is hard? Would that make the sales process move forward more quickly? Would that make you more likely to buy their ever-expanding series of offerings? Would it make you think you won’t end up spending more money counting events, creating custom reports, or adding ad hoc segmentation tools?”
No. If you told the truth about web analytics, your prospects and customers would think twice about their investment. But that is exactly what companies need to do to be successful, really successful, with web analytics — take the science of audience measurement seriously!
When I say “web analytics is hard” I’m not saying that it is impossible, I’m not saying it’s not complex, I’m not saying that it is best left to the experts, and I’m not saying that companies should give up and go home. I’m saying that vendors, consultants, and customers should set their expectations regarding web analytics appropriately.
In my humble opinion, your customers need to know that web analytics is hard so they can:
- Plan to spend a reasonable amount of time determining their needs
- Allocate resources appropriately for implementation and deployment projects
- Set expectations with management about when results will begin to appear and what will need to be done with those results
- Make the case to management when they need additional resources, more software, or more time
- Have an appropriate relationship with their vendor, based on clear expectations
When people are told that “web analytics is easy” they take their investment for granted. They expect that a “standard implementation” or something that comes from a cut-and-paste template will serve their needs, that a 0.25 FTE will be enough to produce analysis, that results will be available in a matter of days, that the software they have will solve all their problems, and that they won’t need their vendor’s support from time to time.
In a way it’s ironic that you say “web analytics is easy” given Omniture’s obvious commitment to their customer’s satisfaction — Larry Freed of ForeSee Results taught me that satisfaction is a function of expectation; when you say “I’ve done it, it’s easy brah” then as soon as they realize the truth, you’ve failed to set their expectation correctly and thusly they’re unsatisfied.
With the increasing numbers of your customers experimenting with less costly tools, I would think that customer satisfaction would be your #1 priority.
I doubt you’ll publish this comment and I suspect you’ll be pissed off at me (again) for voicing an alternative viewpoint but consider this: I’m not saying anything bad about Omniture or any of the companies you guys are buying. I think Omniture is a great organization full of incredible talent. I think the market position you’ve carved out is enviable. I think you guys have tremendous potential to advance the market, driving adoption of Web Analytics 2.0, Web Analytics 3.0, and beyond.
“Web analytics is hard” isn’t about any vendor technology or any one person. “Web analytics is hard” is about your customers and their ability to use your technology and your guidance to their greatest advantage.
When you say “web analytics is easy” you’re oversimplifying what is involved in being successful with web analytics. When you say “I’ve done [web analytics] for a living and I can tell you it’s not [hard]” you’re not paying attention to what your customers are going through. When you say “from your perspective, it’s just not that hard” you’re demonstrating your intelligence but not your wisdom. In fact, your statement “Analytics success is all about building a baseline for performance (your KPI trend), and trying new things to improve on this baseline. That’s it! That’s why I think it’s easy” really says it all.
Suffice to say I was bummed to see this hyperbole and tired rhetoric in an otherwise insightful post.
Sincerely,”
I think it’s one thing when people evangelize for free products as an easy-to-learn entry point into the market, and another entirely when one of the market leading vendors makes such bold and (in my opinion) unfortunately misleading statements.
Our collective ability to be successful depends on having clear expectations, not false ones, and our satisfaction is a function of our expectations. I think Matt is setting the wrong expectation with his comments. What do you think?
