Dear Facebook: As an Analyst, It’s Hard to Be Your Friend

Update: More Facebook Insights updates rolling out, and, so far, they are buggy: Facebook Quietly Updates Insights to Show Real-Time Data on Page Posts, Bugs Appear.

Dear Facebook,

I really want to stay your friend, you see, but I’m an analyst. I’m someone who daily gets asked by marketers: “How do I know if my Facebook investment is paying off?” They want to be your friend, too, but you sure don’t make it easy.

For starters, I couldn’t figure out where to send this note. And, honestly, I’ve never been able to figure out how to actually contact you. It seems kinda’ silly — you do a fantastic job of helping people interact with each other, and you do a lot to enable brands to engage with consumers. Yet, you don’t make it very easy for us data types to engage with you. It would be one thing if there were a handful of uber-analysts who had an “in” with you, and if those folk were out there chiming in to the myriad aborted threads of frustrated analysts trying to extract meaningful data from your systems, that would be one thing. But there aren’t.

Alas! This note is destined to be a bit of a pissy rant. You can accuse me of using social media in all the wrong ways, of launching a Festivus-style airing of grievances. All I can say is that I’ve been around long enough to know that I should not post in anger, but, rather, should pen this note and then let my heels cool overnight. I did. Over a couple of nights, actually. And it still seemed like the right thing to do.

You see, I’ve been doing this web analytics thing for a while now. I lived through the maturing of the industry from “counting hits” all the way to “measuring conversion, segmenting traffic, and testing and optimization experiences.” As an industry, we’ve learned a lot on that front, but, Facebook, you seem hell-bent on reinventing the wheel, and, so far, your wheel looks like a square drawn by a drunken monkey. I want to be able to cleanly measure who’s interacting with my brand within Facebook and how they are engaging with my content. I don’t want to know names and e-mail addresses, but I sure would like to know how first-timers with my brand engage with me as compared to long-time fans. I want to be able to segment my fans and analyze their behavior by segment. Each successive Facebook Insights rollout gets a lot of buzz, but that buzz tends to turn out to be a swarm of horseflies…circling a fresh cow pie. And that stinks.

I don’t know if you know this about me, Facebook, but I was a technical writer early in my career. That’s made me a few things: 1) a pretty fast typist, 2) a guy who occasionally does RTFM, and 3) someone who expects formal documentation to be pretty pristine and comprehensive. With that in mind, let me show you what happens when I go to Facebook Insights and click the Export button. This is the box that pops up:

Seperated? SepErated?!! Maybe you don’t have copy editors on your dev team, but that’s just embarrassing, and, it turns out, an indication of deeper issues. I killed several hours trying to get Excel 2010 + PowerPivot + OData working to “sync data with Excel,” and I overcame several hurdles before running into a brick wall a hundred feet from the finish line. One of these days, maybe I’ll fully crack the code and can write a nice guide on how to make that work — it would be a handy capability — but your single page of documentation simply blithely points to dead ends! Never mind that Excel 2010 is far from a mass-adopted tool (and let me point out that all major web analytics vendors, including, Google Analytics, which is every bit as free as Facebook, have Excel integrations that are well-documented and work with multiple versions of Excel). I’m an analyst — not a programmer. That means I’m reasonably technically savvy, I can generate, hack up, and even do a little debugging of VBA and Javascript here and there. But, I’m not really equipped to jump into a poorly documented API to start extracting data. I need a little help, and you simply do not provide it.

Let’s say I just export an Excel file manually, though. At least, finally, you provide daily data for a few more metrics so that I can do some roll-ups and trending. Now, mind you, I still can’t get trended data for individual custom tab traffic without jumping through a painful number of scroll-and-click hoops, and that’s a pretty run-of-the-mill need. But I digress. I’ve exported my Excel file and I’m checking out the Key Metrics tab. I’m not going to even bother to quibble with how on earth you could know what my key metrics are, or the fact that you provide 18 “key” metrics. Let’s put that aside and, instead, just take a close look at the first three columns of data and the metric names and descriptions provided:

• Daily Active Users — Daily 1 day, 7 day, and 30 day counts of users who have engaged with your Page, viewed your Page, or consumed content generated by your Page (Unique Users)
• Weekly Active Users — Weekly 1 day, 7 day, and 30 day counts of users who have engaged with your Page, viewed your Page, or consumed content generated by your Page (Unique Users)
• Monthly Active Users — Monthly 1 day, 7 day, and 30 day counts of users who have engaged with your Page, viewed your Page, or consumed content generated by your Page (Unique Users)

Wha…?!!! Keeping in mind that each of these metrics has a single column of data that has a value for the metric for each date…what the HECK is a “Monthly 1 day count of users?” I guess I can make an assumption that this was just some of the sloppiest bit of documentation ever written (maybe it was those drunken monkeys again?), and that Daily Active Users are, for each day, the number of unique users who “engaged with the Page” (more on that in a minute) on that one day; that Weekly Active Users are, for each day, the number of unique users who engaged with the page over the prior 7 days (so it’s a rolling 7-day count); and that Monthly Active Users, for each day, are the number of unique users who engaged with the Page over the prior 30 days (so it’s a rolling 30-day count).

Unfortunately, that’s not what the definitions say. What the definitions say is…gibberish.

But wait! There’s more! Let’s look at “…or consumed content generated by your Page.” That’s, like, three multi-syllable words put back to back, which, seemingly, indicates a coherent command of the language. Alas! It’s actually a pretty vague statement. Again, I have to make an assumption that this means, “any user who generated an impression by having a status update by the page render in their news feed.” If that’s what it means, then why not say so? And, if that’s what it means, should that really count as an “active” user? Sure, “engaging with your Page” (my assumption being that that is a Like of or a Comment on content from my page) is a sign of “Active,” as is visiting the page itself (“viewed your Page”), but an impression? Hardly. Unfortunately, I can’t carve that out and use a metric definition that makes sense for me.

The vagueness of the documentation points to a larger issue of transparency as to the mechanics of how you capture and report data. With Google Analytics, Sitecatalyst, Coremetrics, Webtrends, Twitalyzer, Localytics, Flurry, and other analytic tools, I can roll up my sleeves, dive into the documentation and the interwebtubes, do a little experimentation, and wind up with a fundamental understanding of how bits and bytes are flying around to capture the data. While these sorts of underlying mechanics aren’t something that the  business users I support need to understand, it’s critical for my ability to translate the business questions they ask into the interpretation of the reporting and analysis I do. If I had a nickel for every time I had to say, “Well, it’s pure guesswork as to how Facebook is actually capturing and counting that (video views is a biggie there),” I’d have a nice chunk o’ change that I could transfer to an offshore account and then buy a little piece of Facebook. I don’t have those nickels, though, so I’ll settle for just having you pull back the covers a bit and share your data capture mechanisms and data model.

That actually leads to a real head-scratcher on some data you don’t provide. Call me crazy if you must, but I actually care if people are spreading my content to their social graph. You know how they do that? Of course you do! You were instrumental in bringing the concept of “Share” into the mainstream! Yet, you provide no native reporting on share volume (much less segmentation of who shares, or any indication of the lifecycle of a share)! I can get basic content Share counts for content that I manage through Vitrue, but I’m not running Vitrue on all of the pages I work with.

Don’t even get me started on the random nonsensical holes in your data — “my page plummeted from hundreds of thousands of fans to zero fans for two days and then mysteriously returned to its pre-plunge levels!” — or the firm commitment you’ve made to have data available within 48 hours <choke!> of the activity occurring. 48 hours? It’s a real-time world, baby, and, even if “real-time” doesn’t truly need to mean absolute zero latency, 48 hours is ridiculous.

Now, a workaround that occurred to me back in late 2009 was to simply give up on Facebook when it came to getting the data I wanted and, instead, to just deploy web analytics code on my pages. But, you made it clear from the get-go that you had no interest in me bringing anyone else into this relationship, even if they could totally offer something that you’re not interested in providing. Javascript only runs in the narrowest of circumstances, your image caching stymies many workarounds to that limitation, and, even when I am successful, you manage to make me feel vaguely dirty about my success, like I’m doing something wrong. I’m not. I just want to understand how people are engaging with my pages!

I could go on and on. Unfortunately, I don’t have time to make this note shorter, and I do apologize for that. I’m going to go hang out with some Twitter data for a bit to calm down. Maybe, while I’m out, you could take a good hard look at the way you’ve been treating me? A few months ago, Brian Clifton predicted that, in order to survive, Webtrends needs to get acquired, and he suggested that Facebook would be be a good suitor. When I initially read that, I thought it was a pretty “out there” idea. I don’t think that any more. You need to get help. You need a friend, and having some seasoned web analysts and web analytics developers sharing their thoughts and ideas with you would really help your and my relationship with each other.

Facebook, as a user, I am your friend. And I’m loyal. You give me a lot. It’s as an analyst that I’m being forced to remain your friend, even though I soooo Unlike how you reciprocate.

Best regards,

Gilligan on Data