Analytics Strategy, Reporting, Social Media

A Framework for Social Media Measurement Tools

Fundamental marketing measurement best practices apply to social media as much as they apply to email marketing and web site analytics. It all begins with clear objectives and well-formed key performance indicators (KPIs). The metrics that are actually available are irrelevant when it comes to establishing clear objectives, but they do come into play when establishing KPIs and other measures.

In a discussion last week, I grabbed a dry erase marker and sketched out a quick diagram on an 8″x8″ square of nearby whiteboard to try to illustrate the landscape of social media measurement tools. A commute’s worth o’ pondering heading home that evening, followed by a similar commute back in the next morning, and I realized I might have actually gotten a reasonable-to-comprehend picture that showed how and wear the myriad social media measurement tools fit.

Here it is (yep — click on the image to view a larger version):

‘Splain Yourself, Lucy

The first key to this diagram is that it makes a distinction between “individual channel performance” and “overall brand results.” Think about the green box as being similar to a publicly traded company’s quarterly filing. It includes an income statement that shows total revenue, total expenses, and net income. Those are important measures, but they’re not directly actionable. If a company’s profitability tanks in any given quarter, the CEO can’t simply say, “We’re going to take action to increase profitability!”  Rather, she will have to articulate actions to be taken in each line of business, within specific product lines, regarding specific types of expenses, etc. to drive an increase profitability. At the same time, by publicly announcing that profitability is important (a key objective) and that it is suffering, line of business managers can assess their own domains (the blue boxes above) and look for ways to increase profitability. In practice, both approaches are needed, but the actions actually occur in the “blue box” area.

When it comes to marketing, and especially when it comes to the fragmented consumer world of social media, things are quite a bit murkier. This means performance measurement should occur at two levels — at the overall ecosystem (the green box above), which is akin to the quarterly financial reporting of a public company, and at the individual channel level, which is akin to the line of business manager evaluating his area’s finances. I use a Mississippi River analogy to try to explain that approach to marketers.

Okay. Got It. Now, What about These “Measurement Instruments?”

Long, long, LONG gone are the days when a “web analyst” simply lived an breathed a web analytics tool and looked within that tool for all answers to all questions. First, we realized that behavioral data needed to be considered along with attitudinal data and backend system data. Then, social media came along introduced a whole other set of wrinkles. Initially, social media was simply “people talking about your brand.” Online listening platforms came onto the scene to help us “listen” (but not necessarily “measure”). Soon, though, social media channels became a platform where brands could have a formally managed presence: a Facebook fan page, a Twitter account, a YouTube channel, etc. Once that happened, performance measurement of specific channels became as important as performance measurement of the brand’s web site.

When it comes to “managing social media,” brand actions occur within a specific channel, and each channel should be managed and measured to ensure it is as effective as possible. Unfortunately, each of the channels is unique when it comes to what can be measured and what should be measured. Facebook, for instance, is an inherently closed environment. No tool can simply “listen” to everything being said in Facebook, because much of users’ content is only available to members of their social graph within the environment, or interactions they have with a public fan page. Twitter, on the other hand, is largely public (with the exception of direct messages and users who have their profile set to “private”). The differing nature of these environments mean that they should be managed differently, that they should be measured differently, and that different measurement instruments are needed to effectively perform that measurement.

Online listening platforms are not a panacea, no matter how much they present themselves as such. Despite what may be implied in their demos and on their sites, both the Physics of Facebook and the Physics of Twitter apply — data access limited by privacy settings in the former and limited by API throttling in the latter. That doesn’t mean these tools don’t have their place, but they are generalist tools and should be seen primarily as generalist measurement platforms.

Your Diagram Is Missing…

I sketched the above diagram in under a minute and then drew it in a formal diagram in under 30 minutes the next morning. It’s not comprehensive by any means — neither with the three “social media channels” (the three channels listed are skewed heavily towards North America and towards consumer brands…because that’s where I spend the bulk of my measurement effort these days) nor with the specific measurement instruments. I’m aware of that. I wasn’t trying to make a totally comprehensive eye chart. Rather, I was trying to illustrate that there are multiple measurement instruments that need to be implemented depending on what and where measurement is occurring.

As one final point, you can actually wipe out the “measurement instrument” boxes and replace those with KPIs at each level. You can swap out the blue boxes with mobile channels (apps, mobile site, SMS/MMS, mobile advertising). I’m (clearly) somewhat tickled with the construct as a communication and planning tool. I’d love to field some critiques so I can evolve it!

Social Media

New Research on Social Marketing Analytics

I’ve been awaiting this day with eager anticipation for some time now because we are finally releasing our paper on Social Marketing Analytics. Several months ago Eric Peterson and I started talking with Jeremiah and Altimeter Group about the issues facing social marketers. Despite the red hot flames trailing anything with the word social in it, the outlook for effectively measuring the effects of social marketing initiatives in a meaningful way was somewhat grim.

A lack of standardization, reckless experimentation, and unanswered calls for accountability were plaguing businesses who were working in earnest to embark on their social marketing activities. After some intense discussions and some creative thinking, we decided to collaborate on a research project that would leverage social media strategy from Altimeter Group and digital measurement rigor from Analytics Demystified. The result is a framework for measuring social media that we’re happy to share with you today. If you’re into this stuff, please drop us a note or give a call to get involved in the conversation.

Introducing the Social Media Measurement Framework

There’s no denying that social media is the hottest sensation sweeping the globe today. Yet, marketers must see past the shiny object that is social media and start applying a pragmatic approach to measuring their efforts in the social space. We developed a framework that starts with a strategy that requires solid business objectives. From there, specific measures of success – that we call Key Performance Indicators – provide a standardized method for quantifying performance.

 

Mapping Business Objectives to KPIs

Our report identifies four social business objectives that include: Foster Dialog, Promote Advocacy, Facilitate Support and Spur Innovation. While, there may be others that apply to your business, we view these as a solid foundation for beginning the measurement process. In the report we align KPIs to these social business objectives and offer real formulas for calculating success.

    The Social Business Objectives and Associated KPIs are:

  • Foster Dialog: Share of Voice, Audience Engagement, Conversation Reach
  • Promote Advocacy: Active Advocates, Advocate influence, Advocate Impact
  • Facilitate Support: Resolution Rate, Resolution Time, Satisfaction Score
  • Spur Innovation: Topic Trends, Sentiment Ratio, Idea Impact

We encourage you to download the full report here to get the complete context and actual formulas for these KPIs.

Note: This report was a collaborative effort by Analytics Demystified and Altimeter Group and as such there are two versions of this report. The content is identical, yet we each published under our own letterhead.

This is Open Research…

We made a conscious decision not to accept sponsors for this research and to produce it entirely at our own expense so that we could offer a genuine launching pad for social media measurement to the industry. However, this research would not have been possible without numerous contributions from social media and measurement visionaries. We thank them in the report, but it’s worth mentioning that these contributors helped illuminate the big picture of the challenges and opportunities associated with measuring social. We’re publishing this work under a Creative Commons License Attribution-Noncommercial-Share Alike 3.0 United States and encourage practitioners, vendors and consultants to adopt our framework and use it in measuring social media.

We also want to be realistic about this body of work and acknowledge that it does not answer all questions regarding the measurement of social activities. Our hope is that it offers a solid jumping off point for getting started and that each of you in the community will modify and make contributions to improve this method of measurement. We can assure you that we’ll be listening to to your feedback and will continue to update our knowledge based your feedback on this work.

Want to Contribute or Learn More?

Jeremiah and I will be conducting a webcast on June 3rd to reveal the gritty details behind the strategy and framework. Join us by registering here: Attend the no-cost webinar on Social Media Measurement.

We encourage you to embed the Slideshare link into your own sites and I’ll link to others that extend the conversation here as well.

For more white papers from Analytics Demystified, click here. Or to instantly download a PDF of this Social Marketing Analytics report click this GET IT NOW link.

Related Links:

Jeremiah Owyang, my co-author, on the Web-Strategist blog

The Altimeter Group blog posting

Social market analytics: the dark side? Posted by Dennis Howlett

Shel Holtz also recognizes “The haphazard means by which we are monitoring and measuring social media…”

cjlambert’s posterous gives us a “like” but remains skeptical on the concept that media can be measured comprehensively

Geoffroi Garon takes the report to his French speaking audience.

Marshall Sponder includes us in his Social Analytics Web Journal write-up. Marshall is also working to implement our KPIs with one of his clients. Here’s Part 1 of his multi-part series.

Kenneth Yeung delivers a fantastic synopsis of our 25 page report with once concise post on his blog, The Digital Letter, with a follow-up post here.

Research Live offered a brief write-up of the research.

@Scobleizer tweeted us! Woot!

Chelsea Nakano references our research in her ambitious post titled Everything You Need to Know About Social Media Marketing.

Lisa Barone of Outspoken Media gives us a shout out. Lisa was also an influencer that we interviewed for the research.

Christopher Berry delivers great questions and thoughts on the research in his Eyes on Analytics blog.

Social Media

Three Classification Genres for Measuring Twitter

I think it’s safe to say that Twitter has progressed from frivolous novelty to productivity tool for thousands of consumers, professionals and businesses alike. Projections from eMarketer have active Twitter users pegged to reach 18 million by the close of this year. I’ll admit that I was skeptical of the value of Twitter at first. I even went as far as to publish Conscientiously Objecting to Twitter, because I couldn’t see the value. But upon experiencing delirium tremors after being locked out of my Twitter account for 18 hours last week, I’m beginning to see things differently.

Frivolity leads to efficient information intake. There’s been a lot written about how and why people use Twitter for business purposes. I won’t rehash since my fellow Forrester alum, Jeremiah Owyang has published a great blog on the do’s and don’t of Twitter usage. For me, Twitter has become my go-to source for industry news and information. My evolution to this point began circa 2000 when jumping between bookmarks to troll major news sources was my common practice. A few years later, it evolved to aggregating RSS feeds (via iGoogle) in a single portal that I personalized to meet my news interests and needs. Now in 2009 I’m firing up TweetDeck to review the latest buzz; to gather news; and to quickly find information from people I feel are relevant in understanding technology, marketing and to the measurement industry. By using Twitter in this way, it creates visibility for things that I’m curious about and calls my attention to what’s new.

A new mode of discourse comes to light. It used to be that striking up a conversation with someone meant looking them in the eye and asking them what they thought about a particular topic. Twitter (and social media as a whole) has shattered the geographic limitations of conversation. This medium offers both individuals and brands the chance to pose their questions to thousands of potential listeners and receive feedback at scale. The conversation, which may have involved just a handful of people, now includes many. This offers great potential for understanding sentiment, sharing ideas and generally interacting with multiple people in an efficient way. European brands including Cadbury and Vodafone are leveraging Twitter as a new means of interacting with their customers through promotions and clever events. This exchange of information is fueling ideas, products, and adding incremental value (as well as entertainment) in an exponential way.

Twitter ushers in a new era of consumerism. The New York Times recently wrote, “America’s first Twitter Christmas got underway in earnest on Friday”. The article speaks of Black Friday shoppers using Twitter to reveal bargains, lodge complaints and even disclose parking availability at the Mall of America. Examples in the article illustrate that consumers are circumventing traditional channels and turning to social media. This poses significant threats for organizations because service issues are aired for all to see and the appropriateness of response can likely ripple extensively to shape the opinions of thousands of listeners. Best Buy is one organization that is out in front of the social networking craze and has developed “Twelpforce”, an employee driven service that responds to Twitter inquiries. According to the NYTimes article, the Twelpforce answered 25,000 questions even before Thanksgiving demonstrating substantial resolution in an efficient manner.

So what! How can a business use all this? Well, of course it all centers around measurement. As with all marketing initiatives, I recommend that businesses begin with a strategic approach to measurement by clearly understanding goals and objectives (this works for personal brands as well). It requires an introspective look at your motivations for getting involved with Twitter in the first place and then applying a matrix of key performance indicators that will indicate progress toward your goals. These KPIs should be specific to business objectives that pivot depending on the tactic and the social media channel.

The way I see it, there are three classifications of Twitter objectives that can be used by organizations and individuals alike. I’ve broken these classifications into genres because each contains myriad possibilities that will undoubtedly expand and grow as this medium matures. Yet, for individuals, marketers and the brands they represent, each Tweet falls into a genre that can be measured and evaluated with specific indicators of success. Everything else is just noise.

The Twitter genre’s are:

Visibility – This genre includes specific objectives such as building awareness, driving public relations, new customer acquisition efforts, dissemination of news and so on. Visibility is the “hey look over here” function that Twitter offers to get people to read your blog, visit your Web site, learn about your new initiative or simply turn towards that shiny new object you have to offer.

Exchange – Herein lies the catalyst to interaction between individual Twitter users, organizations and brands. The opportunity to pose questions, drive inquiries and elicit feedback within communities opens a new discourse that’s amplified through the channel. It’s the truly collaborative aspect of Twitter, where parties interact with one another in a meaningful way.

Resolution – This is the genre that provides answers. Resolution includes Twitter’s ability to provide service and support in a rapid and widespread manner. It demonstrates to the population that brands are listening to their customers and actually solving problems. It gives consumers a megaphone for expressing either satisfaction or displeasure and places them in the drivers’ seat.

I advise mapping specific measures of success (in the form of KPIs) to these genres in order to better understand the ways in which you’re providing value as an organization (or as an individual) to your following.

Categorization and measurement leads to understanding. Regardless of whether you use my genres above or develop your own, the ability to classify Tweets leads to a systemic method for measurement. The one thing that I love about social media is that there is an opportunity for measurement to transcend the mistakes we’ve collectively made while measuring web sites using Web Analytics. In most cases, Twitter doesn’t carry the baggage of legacy measures. Measurement analysts and marketers have a chance to work should-to-shoulder to establish metrics that align with the goals of their social media efforts. This requires understanding that topline metrics like follows and followers are generally meaningless without context and even with some context they aren’t actionable measures. Thus, more in depth metrics like influence, velocity and clout and the tools we use to measure these activities are required to recognize value from Twitter. For the typical digital measurement analyst this means starting from a place that’s more enlightened than visits and page views. Halleluiah!

Get on the bus and develop a measurement plan. The explosive growth of Twitter (be it healthy or unhealthy) begs the question…How are these 18 million users measuring the value of their efforts? I’ll venture a guess and say that 0.0001% is actually doing any kind of meaningful measurement on their Twitter efforts today. But, if you’re a business immersed in social media then you’d better be measuring – or – if you’re just toe-dipping into the social media arena there’s no better time to start.

But, I want to hear what you think. Are you measuring Twitter using anything like my proposed genres today? Would this method work at your organization? Could it lead to a strategy for developing your personal brand? I welcome your comments in an exchange of ideas here!

Reporting, Social Media

Social Media Measurement: A Practitioner's Practical Guide

Connie Bensen has a Social Media Measurement post that is worth a read. While the post is focussed on measuring social media specifically, she hits on some areas that, all too often, are overlooked when it comes to developing metrics and then reporting on them over time.

The post includes a lot of resources for measuring social media — going well beyond simply web analytics data — as well as a list of examples of things that can be measured. What really struck me, though, was the list at the end of the post of what a community manager’s monthly report should include. First, the fact that it is a monthly report is somewhat refreshing — real-time on-demand reports are way overrated, and really are not practical when it comes to providing the sort of context that Connie describes.

On to Connie’s list of report elements — the bold text is from her list, and the non-bold description is my own take on the item:

  • Ongoing definition of objectives — the framework of any recurring report should be the objectives that it is attempting to measure, so I love that this is the first bullet on the list. I would qualify it just a bit — it does not seem right to be making the defining of objectives an ongoing exercise; rather, objectives should be established, reiterated on an ongoing basis (so that everyone remembers why we’re tackling this initiative in the first place), and revisited periodically (objectives can and should change).
  • Web analytics — this is the “easy” data to provide on a recurring basis, it’s data that most people are getting comfortable with, and, even though there is a lot of noise in the data, it is still reasonably objective; the key here is to focus on the web analytics data that actually matters, rather than including everything.
  • Interaction – Trends in members, topics, discovery of new communities — this is a somewhat community-specific component, but it’s a good one; the “discovery of new communities” actually implies an objective regarding the role of a community manager; what a great metric, though, to drive behavior within the role.
  • Qualitative Quotes – helpful for feedback & marketing — to broaden this list to beyond reporting for social media, let’s change “Quotes” to “Data;” make the report real by providing tangible, but qualitative, examples of what is going well (or not); reporting on lead generation activity, for instance, can include selected comments that were made by attendees at a webinar — highlighting what resonated with the audience (and what did not).
  • Recommendations – Based on interactions with the customers — recommendations, recommendations, recommendations! What is the point of pulling all of this information together if nothing gets done with it? I sometimes like to include recommendations at the beginning of a report — they’re a great way to engage the report consumer by making statements about a course of action right up front.
  • Benchmark based on previous report — my preference is to use stated targets (where it makes sense) as the benchmark, rather than simply looking for the delta of the data over a prior reporting period. But, sometimes, that is simply not feasible. Including “here’s the measurement…and here’s the direction it is heading” is definitely a good thing. But, it’s also important to not look at a 2-month span and jump to “we have a trend!”

Having recently relaunched the Bulldog Solutions blog, I’ve got a good opportunity to put Connie’s post into practice. Oh, dear…that’s going to require re-opening the, “What are our objectives for this thing…clearly stated, please?!” Stay tuned…