Tag: John Lovett

Reporting, Social Media

The New Facebook Insights — One More Analyst's Take

Posted on by Eric Peterson

Facebook released its latest version of Facebook Insights last week, and that’s kicked off a slew of chatter and posts about the newly available metrics. Count this as another one of those. It’s partly an effort to visually represent the new metrics (which highlights some of the subtleties that are a little unpleasant, although, in the end, not a big deal), and it’s partly an effort to push back against the holy-shit-Facebook-has-new-metrics-so-I’m-going-to-combine-the-new-ones-and-say-we’ve-now-achieved-measurement-nirvana-without-putting-some-rigorous-thought-into-it posts (not linked to here, because I don’t really want to pick a fight).

Basically…We’re Moving in a Good Direction!

At the core of the release is a shift away from “Likes” and “Impressions” and more to “exposed and engaged people.” There are now a slew of metrics available at both the page level and the individual post level that are “unique people” counts. That…is very fine indeed! It’s progress!

Visually Explaining the New Metrics

As I sifted through the new Facebook Page Insights product guide (kudos to Facebook for upping the quality of their documentation over the past year!) with some co-workers, it occurred to me that a visual representation of some of the new terms might be useful. I settled on a Venn diagram format, with one diagram for the main page-level metrics and one for the main post-level metrics.

Starting with page-level metrics:

Defining the different metrics — heavily cribbed from the Facebook documentation:

  • Page Likes — The number of unique people who have liked the page; this metric is publicly available (and always has been) on any brand’s Facebook page.
  • Total Reach — The number of unique people who have seen any content associated with a brand’s page. They don’t have to like the page for this, as they can see content from the page show up in their ticker or feed because one of their friends “talked about it” (see below).
  • People Talking About This — The number of unique people who have created a story about a page. Creating a story includes any action that generates a News Feed or Ticker post (i.e. shares, comments, Likes, answered questions, tagged the page in a post/photo/video). This number is publicly available (it’s the “unique people who have talked about this page in the last 7 days”) on any brand’s Facebook page.
  • Consumers — The number of unique people who clicked on any of your content without generating a story.

A couple of things to note here that are a little odd (and likely to be largely inconsequential), but which are based on a strict reading of the Facebook documentation:

  • A person can be counted in the Total Reach metric without being counted in the Page Likes metric (this one isn’t actually odd — it’s just important to recognize)
  • A person can be counted as Talking About This without being included in the Reach metric. As I understand it, if I tag a page in a status update or photo, I will be counted as “talking about” the page, and I can do that without being a fan of the page and without having been reached by any of the page’s content. In practice, this is probably pretty rare (or rare enough that it’s noise).
  • Consumers can also be counted as People Talking About This (the documentation is a little murky on this, but I’ve read it a dozen times: “The number of people who clicked on any of your content without generating a story.” Someone could certainly click on content — view a photo, say — and then move on about their business, which would absolutely make them a Consumer who did not Talk About the page. But, a person could also click on a photo and view it…and then like it (or share it, or comment on the page, etc.), in which case it appears they would be both a Consumer and a Person Talking About This.
  • A person cannot be Consumer without also being Reached…but they can be a Consumer without being a Page Like.

Okay, so that’s page-level metrics. Let’s look at a similar diagram for post-level metrics:

It’s a little simpler, because there isn’t the “overall Likes” concept (well…there is…but that’s just a subset of Talking About, so it’s conceptually a very, very different animal than the Page Likes metric).

Let’s run through the definitions:

  • Reach — The number of unique people who have seen the post
  • Talking About — The number of unique people who have created a story about the post by sharing, commenting, or liking it; this is publicly available for any post, as Facebook now shows total comments, total likes, and total shares for each post, and Talking About is simply the sum of those three numbers
  • Engaged Users — The number of unique people who clicked on anything in the post, regardless of whether it was a story-generating click

And, there is a separate metric called Virality which is a simple combination of two of the metrics above:

That’s not a bad metric at all, as it’s a measure of, for all the people who were exposed to the post, what percent of them actively engaged with it to the point that their interaction “generated a story.”

The Reach and Talking About metrics are direct parallels of each other between the page-level metrics and the post-level metrics. However (again, based on a close reading of the limited documentation), Consumers (page-level) and Engaged Users (post-level) are not analogous. At the post-level, Talking About is a subset of Engaged Users. It would have made sense, in my mind, if, at the page-level Talking About was a pure subset of Consumers…but that does not appear to be the case.

KPIs That I Think Will Likely “Matter” for a Brand

There have been several posts that have jumped on the new metrics and proposed that we can now measure “engagement” by dividing People Talking About by Page Likes. The nice thing about that is you can go to all of your competitors’ pages and get a snapshot of that metric, so it’s handy to benchmark against. I don’t think that’s a sufficiently good reason to recommend as an approach (but I’ll get back to it — stick with me to the end of this post!).

Below are what I think are some metrics that should be seriously considered (this is coming out of some internal discussion at my day job, but it isn’t by any means a full, company-approved recommendation at this point).

We’ll start with the easy one:

This is a metric that is directly available from Facebook Insights. It’s a drastic improvement over the old Active Users metric, but, essentially, that’s what it’s replacing. If you want to know how many unique people are receiving any sort of message spawned from your Facebook page, Total Reach is a pretty good crack at it. Oh, and, if you look on page 176 of John Lovett’s Social Media Metrics Secrets book…you’ll see Reach is one of his recommended KPIs for an objective of “gaining exposure” (I don’t quite follow his pseudo-formula for Reach, but maybe he’ll explain it to me one of these days and tell me if I’m putting erroneous words in his mouth by seeing the new Facebook measure as being a good match for his recommended Reach KPI).

Another possible social media objective that John proposes is “fostering dialogue,” and one of his recommended KPIs for that is “Audience Engagement.” Adhering pretty closely to his formula there, we can now get at that measure for a Facebook page:

Now, I’m calling it Page Virality because, if you look up earlier in this post, you’ll see that Facebook has already defined a post-level metric called Virality that is this exact formula using the post-level metrics. The two are tightly, tightly related. If you increase your post Virality starting tomorrow by publishing more “engage-able” posts (posts that people who see it are more like to like, comment, or share), then your Page Virality will increase.

There’s a subtle (but important this time) reason for using Total Reach in the denominator rather than Page Likes. If you have a huge fan base, but you’ve done a poor job of engaging with those fans in the past, your EdgeRank is likely going to be pretty low on new posts in the near term, which means your Reach-to-Likes ratio is going to be low (keep reading…we’ll get to that). To measure the engage-ability of a post, you should only count against the number of people who saw the post (which is why Facebook got the Virality measure right), and the same holds true for the page.

Key Point: Page Virality can be impacted in the short-term; it’s a “speedboat measure” in that it is highly responsive to actions a brand takes with the content they publish

This is all a setup for another measure that I think is likely important (but which doesn’t have a reference in John’s book — it’s a pretty Facebook-centric measure, though, so I’m going to tell myself that’s okay):

I’m not in love with the name for this (feel free to recommend alternatives!). This metric is a measure (or a very, very close approximation — see the messy Venn diagram at the start of this post) of what percent of your “Facebook house list” (the people who like your page) are actually receiving messages from you when you post a status update. If this number is low, you’ve probably been doing a lousy job of providing engaging content in the past, and your EdgeRank is low for new posts.

Key Point: Reach Penetration will change more sluggishly than Page Virality; it’s an “aircraft carrier measure” in that it requires a series of more engaging posts to meaningfully impact it

(I should probably admit here that this is all in theory. It’s going to take some time to really see if things play out this way).

Those are the core metrics I like when it comes to gaining exposure and fostering dialogue. But, there’s one other slick little nuance…

Talking About / Page Likes

Remember Talking About / Page Likes? That’s the metric that is, effectively, publicly available (as a point in time) for any Facebook page. That makes it appealing. Well, two of the metrics I proposed above are, really, just deconstructing that metric:

This is tangentially reminiscent of doing a DuPont Analysis when breaking down a company’s ROE. In theory, two pages could have identical “Talking About / Page Likes” values…with two very fundamentally different drivers going on behind the scenes. One page could be reaching only a small percentage of its total fans (due to poor historical engagement), but has recently started publishing much more engaging content. The other page could have historically engaged pretty well (leading to higher reach penetration), but, of late, has slacked off (low page virality). Cool, huh?

What do you think? Off my rocker, or well-reasoned (if verbose)?

0
Analysis, Reporting, Social Media

"Demystifying" the Formula for Social Media ROI (there isn't one)

Posted on by Eric Peterson

I raved about John Lovett’s new book, Social Media Metrics Secrets in an earlier post, and, while I make my way through Marshall Sponder’s Social Media Analytics book that arrived on bookshelves at almost exactly the same time, I’ve also been working on putting some of Lovett’s ideas into action.

One of the more directly usable sections of the book is in Chapter 5, where Lovett lays out pseudo formulas for KPIs for various possible (probable) social media business objectives. This post started out to be about my experiences drilling down into some of those formulas…but then the content took a turn, and one of Lovett’s partners at Analytics Demystified wrote a provocative blog post…so I’ll save the formula exploration for a subsequent post.

Instead…Social Media ROI

Lovett explicitly notes in his book that there is no secret formula for social media ROI. In my mind, there never will be — just as there will never be unicorns, world peace, or delicious chocolate ice cream that is as healthy as a sprig of raw broccoli, no matter how much little girls and boys, rationale adults, or my waistline wish for them.

Yes, the breadth of social media data available is getting better by the day, but, at best, it’s barely keeping pace with the constant changes in consumer behavior and social media platforms. It’s not really gaining ground.

What Lovett proposes, instead of a universally standard social media ROI calculation, is that marketers be very clear as to what their business objectives are – a level down from “increase revenue,” “lower costs,”and “increase customer satisfaction” – and then work to measure against those business objectives.

The way I’ve described this basic approach over the past few years is using the phrase “logical model,” – as in, “You need to build a logical link from the activity you’re doing all the way to ultimate business benefit, even if you’re not able to track those links all the way along that chain. Then…measure progress on the activity.”

Unfortunately, “logical model” is a tricky term, as it already has a very specific meaning in the world of database design. But, if you squint and tilt you’re head just a bit, that’s okay. Just as a database logical model is a representation of how the data is linked and interrelated from a business perspective (as opposed to the “physical model,” which is how the data actually gets structured under the hood), building a logical model of how you expect your brand’s digital/social activities to ladder up to meaningful business outcomes is a perfectly valid  way to set up effective performance measurement in a messy, messy digital marketing world.

No Wonder These Guys Work Together

Right along the lines of Lovett’s approach comes one of the other partners at Analytics Demystified with, in my mind, highly complementary thinking. Eric Peterson’s post about The Myth of the “Data-Driven Business” postulates that there are pitfalls a-looming if the digital analytics industry continues to espouse “being totally data-driven” as the penultimate goal. He notes:

…I simply have not seen nearly enough evidence that eschewing the type of business acumen, experience, and awareness that is the very heart-and-soul of every successful business in favor of a “by the numbers” approach creates the type of result that the “data-driven” school seems to be evangelizing for.

What I do see in our best clients and those rare, transcendent organizations that truly understand the relationship between people, process, and technology — and are able to leverage that knowledge to inform their overarching business strategy — is a very healthy blend of data and business knowledge, each applied judiciously based on the challenge at hand. Smart business leaders leveraging insights and recommendations made by a trusted analytics organization — not automatons pulling levers based on a hit count, p-value, or conversion rate.

I agree 100% with his post, and he effectively counters the dissenting commenters (partial dissent, generally – no one has chimed in yet fully disagreeing with him). Peterson himself questions whether he is simply making a mountain out of a semantic molehill. He’s not. We’ve painted ourselves into corners semantically before (“web analyst” is too confining a label, anyone…?). The sooner we try to get out of this one, the better — it’s over-promising / over-selling / over-simplifying the realities of what data can do and what it can’t.

Which Gets Back to “Is It Easy?”

Both Lovett’s and Peterson’s ideas ultimately go back to the need for effective analysts to have a healthy blend of data-crunching skills and business acumen. And…storytelling! Let’s not forget that! It means we will have to be communicators and educators — figuring out the sound bites that get at the larger truths about the most effective ways to approach digital and social media measurement and analysis. Here’s my quick list of regularly (in the past…or going forward!) phrases:

  • There is no silver bullet for calculating social media ROI — the increasing fragmentation of the consumer experience and the increasing proliferation of communication channels makes it so
  • We’re talking about measuring people and their behavior and attitudes — not a manufacturing process; people are much, much messier than widgets on a production line in a controlled environment
  • While it’s certainly advisable to use data in business, it’s more about using that data to be “data-informed” rather than aiming to be “data-driven” — experience and smart thinking count!
  • Rather than looking to link each marketing activity all the way to the bottom line, focus on working through a logical model that fits each activity into the larger business context, and then find the measurement and analysis points that balance “nearness to the activity” with “nearness to the ultimate business outcome.”
  • Measurement and analytics really is a mix of art and science, and whether more “art” is required or more “science” is required varies based on the specific analytics problem you’re trying to solve

There’s my list — cobbled from my own experience and from the words of others!

0
Reporting, Social Media

Have You Picked Up a Copy of "Social Media Metrics Secrets" Yet?

Posted on by Eric Peterson

John Lovett’s Social Media Metrics Secrets hit the bookshelves (Kindle-shelves) earlier this month, and it’s a must-read for anyone who is grappling with the world of social media measurement. It’s a hefty tome as business books go, in that Lovett comes at each of the different topics he covers from multiple angles, including excerpting blog posts written by others and recapping conversations and interviews he conducted with a range of experts.

As such, it’s simply not practical to provide an effective recap of the entire book. Rather, I’ll give my take on the general topics the book tackles, and then likely have some subsequent posts diving in deeper as I try to put specific sections into action.

Part I

The first three chapters of the book are foundational material, in that they lay out a lot of the “why you should care about social media,” as well as set expectations for what isn’t possible with social media data (calculating a hard ROI for every activity) as well as what is possible (moving beyond “counting metrics” to “outcome metrics” to enable meaningful and actionable data usage). Early on, Lovett notes:

Analytics solutions and social media monitoring tools are often sold with the promise that “actionable information is just a click away,” a promise that an increasing number of companies have now realized is not usually the case.

That encapsulates, by extension, much of the theme of the first part of the book — that it requires that a range of emerging tools, skills, processes, and organizational structures to come together to make social media investments truly data-driven activities. In addition to the social analytics platforms that Lovett discusses in greater detail later in the book, he makes a case for data visualization as a key way to make reams of social media data comprehensible, and he paints a picture of a “social media virtual network operations center” — a social media command center that harnesses the right streams of near real-time social media data, presents that data in a way that is meaningful, and has the right people in place with effective processes for putting that information to use.

Part II

In Part II of the book, Lovett starts with some basics that will be very familiar to anyone who operates in the world of performance measurement — aligning key metrics to business objectives, using the SMART (Specific, Measurable, Attainable, Relevant, Times) methodology (although Lovett extends this to be “SMARTER” by adding “Evaluate” and “Reevaluate) for establishing meaningful goals and objectives, understanding the difference between accuracy and precision, and so on. This material is presented with a very specific eye towards social media, and then extended to provide a list common/likely business objectives for social media, which each objective drilled into to identify meaningful measures.

These objectives build directly on the work that Lovett did with Jeremiah Owyang of Altimeter Group in the spring of 2010 when they published their Social Marketing Analytics: A New Framework for Measuring Results in Social Media paper. In the book, Lovett substantially extends his thinking on that framework — broadening from four common social media objectives to six, laying out the “outcome measures” that apply for each objective, and then providing pseudo-formulas for getting to those measures (pseudo-formulas only because Lovett emphasizes the need for social media strategies to not be premised on a single channel such as Facebook or Twitter, and he also didn’t want the book to be wholly outdated by the time it was published — the formulas are explicitly not channel-specific, but anyone who is familiar with a given channel will be well-armed with the tools to develop specific formulas that ladder up to appropriate outcome measures). In short, Chapter 5 is one area that warrants a highlighter, a notepad, and multiple reads.

Part III

Part III of the book really covers three very different topics:

  • Actually demonstrating meaningful results — looking at how to get from the ask of “what’s the hard ROI?” to an answer that is satisfactory and useful, if not a “simple formula” that the requestor wishes for; Lovett devotes some time to explaining the now-generally-accepted realization that the classic marketing funnel no longer applies, and then extends that thinking to demonstrate what will/will not work when it comes to calculating social media ROI
  • Social analytics tools — while Lovett makes the point repeatedly that there are hundreds of tools out there, which can be overwhelming, he nonetheless managed to narrow down a list of seven leading platforms (Alterian SM2, Converseon, Cymfony, Lithium, Radian6, Sysomos, and Trendrr) and conducted an extensive evaluation of them. He includes how that evaluation was organized and the results of the analysis in Chapter 8. While the information is sufficiently detailed that a company could simply take his list and choose a platform, the evaluation is set up as an illustration of what should go into a selection process, so it’s a boon to anyone who has been handed the task of “picking the best tool (for our unique situation).”
  • Consumer privacy — this is a very hot topic, and it’s a messy area, so Lovett tries to lay out the different aspects of the situation and what needs to happen to get to some reasonably workable resolution over the next few years. It’s a portion of the book that I’ve already referenced and quoted internally, as it is very easy for marketers and vendors to get caught up in the cool ways they can make content more relevant…without thinking through whether consumers would be okay with those uses of the data

After reading the book once, I’ve already found myself flipping back to certain sections to the point that I’ve got Post Its coming out of it to mark specific pages. Overall, the book is sufficiently modular that individual chapters (and even portions of chapters) stand alone.

Buy it. Buy it now!

 

0