Facebook Engagement (aka, Facebook Rhetoric Facebook Reality)

Oh, Facebook.

Facebook, Facebook, Facebook.

Ours is a tumultuous relationship of unrequited frustration, is it not? I am an analyst, therefore (apparently), you scorn me. And, by “scorn,” I mean “ignore.”

You never responded to my letter last year. You don’t return my calls. (Well, that’s not entirely true: you put salespeople on my calls whose general response to any question is, “Buy Facebook media.” I get it. That’s their job, but they act like they’ve parachuted straight out of Mad Men and are pushing traditional mass-blast advertising. Ironic, no?)

Facebook, I’ve dug into the data. Your own documentation states:

Posting regularly with engaging content gets more people to talk about your business with their friends. As a result, you end up reaching more people overall.

Yet, the data you provide us tells a very different story. We debunked this particular claim — that getting people to talk about your content leads to greater reach — a month ago.

So, What Can We Debunk This Month?

Lately, I’ve been digging into a more basic mystery: you claim that, the more someone engages with a page’s content, the more likely that person is to get presented with more of that page’s content in the future. That seems pretty reasonable. Of course, you hedge at the same time:

No matter how engaging your Page posts are, not all of your fans will see them in their News Feed. In order to make sure that more of your fans see your posts, you should create a Page Post Ad

Can we quantify that “not all of your fans…” statement? AllFacebook.com did just that when they published a pretty alarming article last week based on Edgerank Checker data. Their study showed that, on average, across 4,000 pages, only 17% of total fans were being reached per individual post by the brand. “Zoiks!” were the cries that echoed through the halls of community managers the world over!

To be fair, not everyone is on Facebook all the time, and, while that number matches data we’re seeing overall, it also leaves out the fact that these don’t appear to be the same 17% day in and day out. When it comes to looking at the 28-Day Total Reach from Page Posts measure you provide, we see numbers that are more in the neighborhood of half of a page’s Lifetime Total Likes (when there is no Facebook media running — it’s much higher than that if that exposure is being purchased from Facebook).

Is 17% really all brands can expect, or is it all they can expect if they’re doing a lousy job posting content?

Are Brands Simply Not Publishing Engaging Content?

We’ve been working pretty hard to learn what kind of content our clients’ fans like, as well as how often and when to post. That put us in a good position to dig into the data to see how we were doing, especially in light of the drop we felt we were seeing in the Reach of posts across a range of our clients’ pages.

We looked at data from a half-dozen pages. These pages were all devoted to major consumer brands, had Lifetime Total Likes ranging from the low 100,000s to multiple millions, and cut across a range of different verticals. Is “6 pages” on the order of the “4,000 pages” from the Allfacebook.com study? Well, no, but we were working with over 600 status updates, and it quickly became apparent that we’d dug in enough to draw some pretty sound conclusions..

For the chart below, we removed the handful of posts that were clearly data anomalies (skewing both wildly high and wildly low) and then, for each post, took the Lifetime Engaged Users for the post (the number of unique people who clicked anywhere in the post within 28 days of it being posted, regardless of whether the click generated a story or not) and divided it by the Total Reach for the post.

It’s not the cleanest of graphs, but it seems pretty clear that, if anything, these pages are, overall, making some headway when it comes to producing more engaging content.

The idea here is that the only people a post has a chance of engaging are people that it reaches. So, we have Total Reach as the denominator. This is similar to the Post Virality calculation that you, Facebook, generate for me…but we’re looking at a lower level of engagement than “generated a story” — just looking to see if fans are interacting with the post in any way. Because, in theory, if they are, then you will be more likely to present them with subsequent posts from the same page.

So, Engagement Isn’t Dropping. Presumably, Reach Isn’t, Either?

In the post engagement chart, there’s nothing all that shocking. What does get alarming, though, is when we look at the average Organic Reach (unique users who saw the post directly as a result of the page posting it — not because a friend talked about it, and not because the brand ran paid media to extend the reach of the post). We divided that organic reach by the Lifetime Total Likes for the page to see what % of the total fans were reached by the post organically.

Again, outliers (high and low) were removed (this included locally-targeted posts, where the reach, obviously, was very low relative to the total likes for the page). Each point on the chart represents all of the status updates on that day from our sample:

Wow. I’m not a data scientist, so the above doesn’t have any true statistical rigor applied to it. Rather, it is an exercise in what a stats professor once preached to me: “Start off by plotting the data! That’s going to tell you a lot!”

It’s pretty conclusive, I think, that a Facebook algorithm change (and related UI changes — but the algorithms drove what content appears anywhere for a user, regardless of the UI) in late September gave brands a temporary ability to reach a higher proportion of their fans. That, undoubtedly, led to any number of community managers thinking they had been listening and learning and publishing more engaging content.

Then, (alas!) November arrived. And, suddenly, Reach plummeted.

WTF?

It’s not that I’m opposed to paying you for reach, Facebook. I’m totally okay with paid media being part of my social media mix. But, if I have to pay you each time I want to reach someone, the numbers start to get hard to justify. If someone likes my page, and then they engage with my content, why don’t they keep getting my content for some period of time?

Here’s what I think happened (and, frankly, I’d respect you a bit more in the morning if you just came out and admitted it):

1. You put some sharp people in a room and told them to come up with a good EdgeRank/GraphRank algorithm
2. While you have “a lot of data,” that algorithm still was largely driven by that team’s instincts around what weighting should be given to different factors
3. There was a fair amount of teeth-gnashing, and the team even tried to do some testing of the algorithm before rolling it out. But, that’s a taller order than it sounds.
4. The algorithm got rolled out.
5. You had no idea what was going to happen. What looked good on paper looked, well, different in practice.
6. For various reasons — none of which have been openly stated — the algorithm has been quietly tweaked a couple of times. In one case, it was related to the Timeline rollout, but, by this time, the algorithm had become the red-headed stepchild of Palo Alto. No one really wants to own it, because no one can really figure out what will make it “work.” After all…the algorithm-heads are all just down the street in Mountain View! (zing!)

How close am I with the above speculation? I don’t have inside knowledge (as noted earlier, you don’t call, you don’t write), but I’m not sure what other explanation makes sense.

Know that you’re killing us — the analysts who are trying to drive learning and optimization! At least set up some sort of open dialogue. We don’t need to see the full formula. But, we need to have useful information about how to do things better. And we need to know when you’re tinkering with the algorithm and what the likely result of that tinkering will be. Otherwise, we can’t trust the data, which means we can’t learn from it. Without data we can use, it’s hard to justify investment and action.