The Myth of the "Data-Driven" Business

You may have noticed I have been pretty quiet in my blog lately aside from sharing news about our ACCELERATE event in San Francisco in November. It’s partially because honestly I’ve been swamped with new clients, existing work, and the never-ending effort to be a good husband, dad, and friend in the midst of Demystifying web analytics …

But being busy is no excuse to stop sharing ideas and encouraging conversation so let’s dive into something that has increasingly become a pet-peeve of mine: the notion leveraging web analytics to create a “data-driven” business.

I’m sure I have used this phrase in the past in an effort to describe the transformation that companies need to go through in the digital world, relying less on “gut feel” and more on cold, hard data to guide business decision making. Hell, a lot of smart of people have, including Omniture’s Brent Dykes and Google Analytics Evangelist Avinash Kaushik who has gone so far as to describe creating a data-driven culture as the “holiest of holy grails.”

Becoming “data driven” is the way to silence the HIPPO and to more firmly establish the value of our collective investments in digital measurement, analysis, and optimization technology. It sounds great, except for one thing:

A “data-driven business” would be doomed to fail.

I think that perhaps what people mean when they talk about being “data-driven” is the need for a heightened awareness of the numerous source of data and information we have available in the digital world, enough so that we are able to take advantage of these sources to create insights and make recommendations. On this point I agree — better use of available data in the decision making process is an awesome thing indeed.

My concern arises from the idea that any business of even moderate size and complexity can be truly “driven” by data. I think the right word is “informed” and what we are collectively trying to create is “increasingly data-informed and data-aware businesses and business people” who integrate the wide array of knowledge we can generate about digital consumers into the traditional decisioning process. The end-goal of this integration is more agile, responsive, and intelligent businesses that are better able to compete in a rapidly changing business environment.

Perhaps this is mere semantics — you say “potato” I say “tuberous rhizome”  — but given the sheer number of consultants, vendors, and practitioners talking about creating, powering, and working in the mythical “data-driven business” I have started to worry that we’re about to shoot ourselves in the collective foot. We (meaning the web analytics industry as a whole) have done this before, first by claiming that web analytics was easy, then by insisting that cookies were harmless … and personally I’d prefer we avoid yet another self-imposed crisis of credibility if possible.

And while this may be semantics, I do disagree with Brent Dykes assertion that in the absence of carrot-and-stick accountability that web analytics breaks down and fails to create any benefit within the business, although I do understand fully where Mr. Dykes is coming from. I simply have not seen nearly enough evidence that eschewing the type of business acumen, experience, and awareness that is the very heart-and-soul of every successful business in favor of a “by the numbers” approach creates the type of result that the “data-driven” school seems to be evangelizing for.

What I do see in our best clients and those rare, transcendent organizations that truly understand the relationship between people, process, and technology — and are able to leverage that knowledge to inform their overarching business strategy — is a very healthy blend of data and business knowledge, each applied judiciously based on the challenge at hand. Smart business leaders leveraging insights and recommendations made by a trusted analytics organization — not automatons pulling levers based on a hit count, p-value, or conversion rate.

Kishore Swaminathan, Accenture’s chief scientist, in his discussion on “What the C-suite should know about analytics” outlines how an over-dependence on data can lead to “analysis-paralysis”, stating:

“Data is a double-edged sword. When properly used, it can lead to sound and well-informed decisions. When improperly used, the same data can lead not only to poor decisions but to poor decisions made with high confidence that, in turn, could lead to actions that could be erroneous and expensive.”

Success with web analytics and optimization requires a balance, and business leaders who will be successful analytical competitors in the future will need to develop a top-down strategy to govern how their businesses will leverage both digitally-generated insights and the collective know-how of their organizations. Conversely, being “driven” implies imbalance and over-correction — going out of your way to devalue experience, ignore process, and eschew established governance in favor of a new, entirely metrics-powered approach towards decision making.

You can do this, but to Swaminathan’s point, what if the numbers you’re using are wrong?

I think that creating a “data informed” business is a huge victory and for most companies a major step in the right direction. What’s more, working to create a “data informed” business shows respect for the hard work, commitment, and passion your employees have for their jobs and your company and products.

Rather than walk in and “embarrass the boss” with your profound and amazing knowledge of customer interactions, you can actively work with your management team by providing insights and recommendations that reflect your knowledge of how the entire business works, not just your amazing talent as web analytics implementer (or analyst, whatever …)

But I digress.

I’m interested in your collective thoughts here people. Am I over-reaching after a blogging hiatus and unnecessarily sniping in hopes of an early Fall dust-up in Google+? Or have you had the same thoughts and/or concerns, that by insisting that everyone needs to do exactly what the data tells them that we risk alienating (again) the very consumers of our efforts? Do you work at a truly “data driven” business and do what the numbers tell you each and every time? Or are you working to create a practice where otherwise smart, hard-working, and passionate marketers, merchandisers, and business leaders can benefit from the type of information and insights you are uniquely able to provide as a digital measurement, analysis, and optimization specialist?

While you consider your response I’ll leave you with a story that has shaped some of my thinking about web analytics over my career. Years ago my good friend Shari Cleary brought me into CBS News in New York to train her editorial team on Hitbox (yeah, Hitbox, I told you it was years ago!) Most of my clients at the time were “new school” but not these guys — they were hardcore news editors from the TV side of the business who had been tasked with making digital news work.

I talked and talked and talked about how powerful Hitbox was and how real-time analytics was going to power the content they put out there in the world. The editors were polite and showed real interest in the training until at one point the oldest and most grizzled of the group stopped me.

“Son, we’re not going to let the data make the decisions for us regarding editorial content,” he said with all sincerity. I was, of course, shocked to hear this — I mean, hell, that is what Hitbox was for! Figuring out which stories generated page views and which needed to be rolled off the page into obscurity.

“Umm, why is that?” I asked, figuring he’d lay into me about the inaccuracy of the system or how painful it was to use Hitbox …

“Because if we let the data drive editorial, all you will read about at CBS News is Paris Hilton’s breasts and Lindsay Lohan’s drinking problem.”

Needless to say, I stopped talking about real-time, data-driven changes to editorial content.

As always I welcome your comments, criticism, and feedback.