Four Ways that Media Mix Modeling (MMM) Is Broken

Many companies rely on some form of media mix modeling (or “marketing mix modeling”) to determine the optimal mix of their advertising spend. With the growth of “digital” media and the explosion of social media, these models are starting to break down. That puts many marketing executives in a tough bind:

1. Marketing, like all business functions, must be data-driven — more so now than ever
2. Digital is the “most measurable medium ever” (although their are wild misperceptions as to what this really means)
3. Ergo, digital media investments must be precisely measured to quantify impact on the bottom line

For companies that have built up a heavy reliance on media mix modeling (MMM), the solution seems easy: simply incorporate digital media into the model! What those of us who live and breathe this world recognize (and lament over drinks at various conferences for data geeks), is that this “simple” solution simply doesn’t work. Publicly, we say, “Well…er…it’s problematic, but we’re working on it, and the modeling techniques are going to catch up soon.”

My take: don’t hold your breath that MMM is going to catch up — even if it catches up to today’s reality, it will already be behind, because digital/social/mobile will have continued its explosive evolution (and complexity to model).

Believe it or not, I’m not saying that MMM should be completely abandoned. It still has it’s place, I think, but there are a lot of things it’s going to really, really struggle to address. I’d actually like to see companies who provide MMM services weigh in on what that is. At eMetrics earlier this month, I attended a session where the speaker did just that. Skip ahead to the last section to find out who!

Geographic Test/Control Data

Both traditional and digital marketing have a mix of geo-specific capabilities. The cost of TV, radio, print, and out-of-home (OOH) marketing provides an imperative to geo-target when appropriate (or simply to minimize the peanut butter effect of spreading a limited investment so thinly that it doesn’t have an impact anywhere). Many digital channels, though, such as web sites and Facebook pages, are geared towards being “available to everyone.” Other channels – SEM, banner ads, and email, for instance – can be geo-targeted, but there often isn’t a cost/benefit reason to do so. Without different geographic placements of marketing, the impact on sales in “exposed areas” vs. “unexposed areas” cannot be teased out:

Cross-Channel Interaction

While marketers have long known that multi-channel campaigns produce a whole that is greater than the sum of the parts, the sheer complexity that digital has introduced into the equation forces MMM to guess at attribution. For example, we know (or, at least, we strongly suspect) that a large TV advertising campaign will not only provide a lift in sales, but it will also produce a lift in searches for a brand. Those increased searches will increase SEM results, which will drive traffic to the brand’s web site. Consumers who visit the site can then be added to a retargeting campaign. Those are four different marketing channels that all require investment…but which one gets the credit when the consumer buys?

This is both data capture and a business rules question. Entire companies (Clearsaleing being the one that I hear the most about) have been built just to address the data capture and application of business rules. While they provide the tools, they’re a long way from really being able to capture data across the entire continuum of a consumer’s experience. The business rules question is just as significant — most marketers’ heads will explode if they’re asked to figure out what the “right” attribution is (and simply trying different attribution models won’t answer the question — different models will show different channels being the “best”). Is this a new career option for Philosophy majors, perhaps?

Fragmentation of Consumer Experiences

This one is related to the cross-channel interaction issue described above, but it’s another lens applied to the same underlying challenge. Consumer behavior is evolving — there are exponentially more channels through which consumers can receive brand exposure (I picked up the phrase “cross-channel consumer” at eMetrics, which is in the running for my favorite three-letter phrase of 2010!). Some of these channels operate both as push and pull, whereas traditional media is almost exclusively “push” (marketers push their messages out to consumers through advertising):

We’re now working with an equation that has wayyyyyyyy more variables, each of which has a lesser effect than the formulas we were trying to solve when MMM first came onto the scene. HAL? Can you help? This is actually beyond a question of simply “more processing power.” It’s more like predicting what the weather will be next week — even with meteoric advancements in processing power and a near limitless ability to collect data, the models are still imprecise.

Self-Fulfilling Mix

Finally, there is a chicken-and-egg problem. While there are reams of secondary research documenting the shifting of consumer behavior from offline to online consumption…many brands still disproportionately invest in offline marketing. It’s understandable — they’re waiting for the data to be able to “prove” that digital marketing works (and prove it with an unrealistic degree of accuracy — digital his held to a higher standard than offline media, and the “confusion of precision with accuracy” syndrome is alive and well). But, when digital marketing investments are overly tentative (and those investments are spread across a multitude of digital channels), the true impact of digital can’t be detected because it’s dwarfed by the impact of the massive — if less efficient — investments in offline marketing:

If I shoot a pumpkin simultaneously with a $1,500 shotgun and a $30 BB gun and ask an observer to tell me how much of an impact the BB gun had…

So, Should We Just Start Operating on Faith and Instinct?