Campaign Measurement Planning — Columbus WAW Recap Part 1

We tried a new format at last week’s Columbus Web Analytics Wednesday, in that we had three completely unrelated presentations, and we kept the entire presentation period to right at a half hour. Mathematically, that gave us 10 minutes per presentation, and we split the time between formal presenting and Q&A. The event was sponsored by Resource Interactive (population 320-ish and growin’; SA…LUTE! </heehaw>), and it was our first “presentation included” WAW since last November. Apparently, we had some pent-up WAW demand, as we had right around 45 attendees.

The three presentations of the evening were:

• Social Search by Dave Culbertson of Lightbulb Interactive
• Campaign measurement planning by Bryan Cristina of Nationwide Insurance
• Web analytics platforms comparison by yours truly of Resource Interactive

Dave’s presentation was the most informal and focussed on the various developments across Facebook/Bing and Google when it comes to incorporating social graph and social profile data into search results. The Google video he showed was pretty interesting, and he illustrated how rapidly the space is evolving. But, overall, I took lousy notes, so I don’t know that I’ll manage to get a full blog post up on the subject.

As for Bryan’s presentation, I had the benefit of previewing the material and, as such, getting to have a mini-Q&A with Bryan via e-mail.

Campaign Measurement Planning

Bryan and I are both pretty passionate about measurement planning. His presentation really nails some key points about the topic and has a fantastic list on slide 8 as to elements to consider including in a measurement plan:

In addition, Bryan provided a (click to download) measurement plan example Word document (it’s an auto insurance company example, so it’s obviously grounded in reality, but he worked it over pretty thoroughly on several fronts in preparation for the presentation, so it is an entirely fictional example).

I asked Bryan a couple of questions offline about his approach prior to the event:

Q: Under “Targets and Benchmarks,” you note, “Don’t be afraid to put ‘TBD’ or ‘No Data’ for some benchmarks.” If that is the case, do you support not setting a target, or should you still try to set a target (even noting that it is a bit of a swag) in the absence of a benchmark?

Bryan’s response: I try to set a target no matter what because it gets people at least thinking about it and TRYING to set up some kind of expectation.  It makes sure that people are at least estimating.  Maybe they don’t know the CPC for the search terms yet, aren’t sure on the demand, and aren’t sure on the completion rates, but it’s at least a start.  We were completely off for one of our last campaigns because we had no idea on all those factors.  It still gave the agency something to report on for their % of goal and it drove an informed discussion mid-campaign.

[I, of course, loved this answer…because I totally agreed with it]

Q: You’re at a company that uses agencies for much of the campaign execution, and, clearly, you have put in a process whereby you develop this sort of plan partly as a tool to drive clarity and alignment with the agencies and their work. As an agency analyst, we are increasingly including “measurement planning” as a non-optional part of the scope of our engagements. In those cases, we (the agency) actually do the discovery and documentation of the measurement plan (which clients provide input to, review, and approve). I actually would love to have clients coming to us with this level of forethought, but, in the absence of that, what are your thoughts on having the accountability for the creation of a measurement plan reside with an agency?

Bryan’s response: I think this varies on the relationship between the agency and company.  For us, we’re very capable, we all know how to do the campaign execution, but we just don’t have the time or bodies to do it.  I’m sure there are many companies that have no clue how to do it, so the agency does both the execution and the strategy, or the execution, strategy, tracking, plus reporting, or whatever else. It really depends on the analytics maturity of the client as to whether it makes more sense for the agency or the client to own the creation of the plan.  If it’s the agency, you’d have to be absolutely be sure to talk to the client in depth about all of it and make sure they’re on board with all the points.  In the end, the outcome should be the same, the only difference really being the author of the document would be the agency instead of the business, and I’m sure some of the reporting responsibilities would change based on that.

Bryan joked during his presentation about how “exciting” the topic of measurement planning is. Obviously, it can seem like a pretty dry topic, but, in both of our experiences, measurement planning can drive some tough and interesting discussions. More importantly, it’s a foundational element of marketing — without it, you wind up looking back after the fact and wondering if what you executed was successful, whether you captured the right data, and whether you learned anything that can be meaningfully applied to the next initiative.

Hey…I also cleaned up my “sharing” options on my blog this weekend. Go ahead. Give it a try! See how easy it is to Like or Tweet (or…er…whether I really got those implemented and functioning correctly). Who knows, maybe Facebook Insights will start giving me some interesting web site data!