Twitter Analytics — Turmoil Abounds, and I'm a Skeptic
Last week was a little crazy on the Twitter front, with two related — but very different — analytics-oriented announcements hitting the ‘net within 24 hours of each other. Let’s take a look.
Selling Tweet Access
On Wednesday, Twitter announced they would be selling access to varying volumes of tweets, with 50% of all tweets being available for the low, low price </sarcasm> of $360,000/year. It appears there will be a variety of options, with “50%” being the maximum tweet volume, but with other options in the offing to get 5% of all tweets, 10% of all tweets, or all tweets/references/retweets that are tied to a specific user. All of these sound like they’re going to come with some pretty tight usage constraints, including that they can’t be resold and that the actual tweet content can’t be published.
Twitter has made an API available almost from the moment the service was created. That’s one of the reasons the service grew so explosively — developers were able to quickly build a range of interfaces to the tool that were better than what Twitter’s development team was able to create. But, the API came with limitations — a very tight limit on how often an application could get updates, and a tight limit on just how many updates could be pushed/pulled at once.
As various Twitter analytics-type services began to crop up, Twitter opened up a “garden hose” option — developers could contact Twitter, show that they had a legitimate service with a legitimate need, and they could get access to more tweets more often through the API. Services like Twitalyzer, TweetReach, and Klout jumped all over that option and have built out robust and useful solutions over the course of the last 6-12 months. Now it looks like Twitter is looking to coil up the garden hose, which could spell a permanent end to the growing season for these services. This will be a shame if it comes to pass.
For a steep price, these paid options from Twitter will have limited use: limited to some basic monitoring/listening and some basic performance measurement. Even with the $360K/year option, providing half of the tweets seems problematic when you consider Twitter from a social graph perspective — in theory, half of the network ripple from any given tweet will be lost, or, more confusingly, will crop up as a 2nd or 3rd degree effect with no ability to trace it back to its source because the path-to-the-source passes through the “unavailable 50%!”
This data also won’t be of much use as a listen-and-respond tool. Imagine a brand that has a fantastic ability to monitor Twitter and appropriately engage and respond…but appears schizophrenic because they’re operating with one eye closed (and paying a pretty penny to do even that!). To be clear, for any given brand or user, only a tiny fraction of all tweets are actually of interest, but that tiny fraction is going to be spread across 100% of the Twitterverse, so only having access to a 5%, 10%, or even 50% sample means that relevant tweets will be missed.
Online listening platforms — Radian6, SM2, Buzzmetrics, Crimson Hexagon, Sysomos, etc. — may actually have deep enough pockets to pay for these tweets to improve their own underlying data…but they will have to significantly alter the services they provide in order to comply with the usage guidelines for the data.
Ugh.
Twitter Analytics
On Thursday, Mashable reported that Twitter Analytics was being tested by selected users. Unfortunately, I’m not one of those users (<sniff><sob>), so I’m limited to descriptions in the Mashable article. Between that article and Pete Cashmore’s (Mashable CEO) editorial on cnn.com, I’ve got pretty low expectations for Twitter Analytics.
Both pieces seem somewhat naive in that they overplay the value to brands that Facebook has delivered with Facebook Insights, and they confuse “pretty graphs” with “valuable data.” All I can think to do is rattle off a series of reactions from the limited information I’ve been able to dig up:
- Replies/references over time: um…thanks, but that’s always been something that’s pretty easy to get at, so no real value there.
- Follows/unfollows: this seems to be taking a page directly from Facebook Insights with it’s new fans/removed fans reporting (which, by the way, never agrees with the “Total Fans” data available in the same report, but I digress…); this has marginal value — in practice, unless a user is really pissing off followers or baiting them to follow with a very specific promotional giveaway (“Follow us and retweet this and you’ll be entered to win a BRAND NEW CAR!!!”), there’s probably not going to be a big spike in unfollows, and it isn’t that hard to trend “total followers” over time, so I can’t get too excited about this, either
- Unfollows (cont’d.): “tweets that cause people to unfollow” is another apparent feature of Twitter analytics. Really? Was that something that someone living on planet Earth came up with? This sounds nifty initially, but, in practice, isn’t going to be of much use. If a user posts offensive, highly political (for a non-political figure user), or obnoxiously self-promoting tweets…he’s going to lose followers. I don’t think “analytics” will really be needed to figure out the root cause (if it was a single tweet) driving a precipitous follower drop. Common sense should suffice for that.
- Retweets: this is like references, in that it’s not really that hard to track, and I wouldn’t be surprised at all if Twitter Analytics only counts retweets that use the official Twitter retweet functionality, rather than using a looser definition that includes “RT @<username>” occurrences (which are retweets that are often more valuable, because they can include additional commentary/endorsement by the retweeters)
- Impressions: I’m expecting a simplistic definition of impressions that is based just on the number of followers, which is misleading, because most users of Twitter see only a fraction of the tweets that cross their stream. Twitalyzer calculates an “effective reach” and Klout calculates a “true reach” — both make an attempt to factor in how receptive followers are to messages from the user. None of these measures is going to be perfect, but I’m happier relying on companies whose sole focus is analytics trying to tinker with a formula than I am with the “owner” of the data coming up with a formula that they think makes sense.
With the screen caps I’ve seen, there is no apparent “export data” button, and that’s a back-breaker. Just as Facebook Insights is woefully devoid of data export capabilities (the “old interface” enables data export…but not of some of the most useful data, and API access to the Facebook Insights data doesn’t exist, as best as I’ve been able to determine), Twitter looks like they may be yet another technology vendor who doesn’t understand that “their” dashboard is destined to be inadequate. I’m always going to want to combine Twitter data with data that Twitter doesn’t have when it comes to evaluating Twitter performance. For instance, I’m going to want to include referrals from Twitter to my web site, as well as short URL click data in my reporting and analysis.
Ikong Fu speculated during an exchange (on Twitter) that Twitter may also, at some point, include their internal calculations of a user’s influence in Twitter Analytics:
I didn’t realize that Twitter was calculating an internal reputation score. It makes sense, though, that that would be included when they make recommendations of who else a user might want to follow. I found a post from Twitter’s blog back in July that announced the rollout of “follow suggestions,” and that post indicated these were based on “algorithms…built by our user relevance team.” The only detail the post provided was that these suggestions were “based on several factors, including people you follow and the people they follow.” That sounds more like a social graph analysis (“If you’re following 10 people who are all following the same person who you are not following, then we’re going to recommend that you follow that person”) than an analysis of each user’s overall influence/quality. Again…I’m more comfortable with third party companies who are fully focussed on this measurement and who make their algorithms transparent providing me with that information than I am with Twitter in that role.
So, Where Does This Leave Us?
Maybe, for once, I’m just seeing a partially filled glass of data as being half empty rather than half full (okay, so that’s the way I view most things — I’m pessimistic by nature). In the absence of more information, though, I’m forced to think that, just as I was headed towards analytics amour when it came to Twitter data, Twitter is making some unfortunate moves and rapidly smudging the luster right off of that budding relationship.
Or, maybe, I’m unfairly pre-judging. Time will tell.
