Reporting: You Can't Analyze or Optimize without It

Three separate observations from three separate co-workers over the past two weeks all resonated with me when it comes to the fundamentals of effective analytics:

• As we discussed an internal “Analytics 101” class  — the bulk of the class focusses on the ins and outs of establishing clear objectives and valid KPIs — a senior executive observed: “The class may be mislabeled. The subject is really more about effective client service delivery — the students may see this as ‘something analysts do,’ when it’s really a a key component to doing great work by making sure we are 100% aligned with our clients as to what it is we’re trying to achieve.”
• A note added by another co-worker to the latest updated to the material for that very course said: “If you don’t set targets for success up front, someone else will set them for you after the fact.”
• Finally, a third co-worker, while working on a client project and grappling with extremely fuzzy objectives, observed: “If you’ve got really loose objectives, you actually have subjectives, and those are damn tough to measure.”

It struck me that these comments were three sides to the same coin, and it got me to thinking about how often I find myself talking about performance measurement as a critical fundamental building block for conducting meaningful analysis.

“Reporting” is starting to be a dirty word in our industry, which is unfortunate. Reporting in and of itself is extremely valuable, and even necessary, if it is done right.

Before singing the praises of reporting, let’s review some common reporting approaches that give the practice a bad name:

• Being a “report monkey” (or “reporting squirrel” if you’re an Avinash devotee) — just taking data requests willy-nilly, pulling the numbers, and returning them to the requestor
• Providing “all the data” — exercises of listing out every possible permutation/slicing of a data set, and then providing a many-worksheeted spreadsheet to end users so that they can “get any data they want”
• Believing that, if a report costs nothing to generate, then there is no harm in sending it — automation is a double-edged sword, because it can make it very easy to just set up a bad report and have it hit users’ inboxes again and again without adding value (while destroying the analyst’s credibility as a value-adding member of the organization)

None of these, though, are reasons to simply toss reporting aside altogether. My claim?

If you don’t have a useful performance measurement report, you have stacked the deck against yourself when it comes to delivering useful analyses.

Let’s walk through a logic model:

1. Optimization and analysis are ways to test, learn, and drive better results in the future than you drove in the past
2. In order to compare the past to the future (an A/B test is a “past vs. future” because the incumbent test represents the “past” and both the incumbent and the challenger represent “potential futures”), you have to be able to quantify “better results”
3. Quantifying “better results” mean establishing clear and meaningful measures for those results
4. In order for measures to be meaningful, they have to be linked to meaningful objectives
5. If you have meaningful objectives and meaningful measures, then you have established a framework for meaningfully monitoring performance over time
6. In order for the organization to align and stay aligned, it’s incredibly helpful to actually report performance over time using that framework, quod erat demonstrandum (or, Q.E.D., if you want to use the common abbreviation — how in the hell the actual Latin words, including the correct spelling, were not only something I picked up in high school geometry in Sour Lake, TX, but that has actually stuck with me for over two decades is just one of those mysteries of the brain…)

So, let’s not just bash reporting out of hand, okay? Entirely too many marketing organizations, initiatives, and campaigns lack truly crystallized objectives. Without clear objectives, there really can’t be effective measurement. Without effective measurement, there cannot be meaningful analysis. Effective measurement, at it’s best, is a succinct, well-structured, well visualized report.

Photo: Greymatterindia