Social Media

Facebook Insights — My Favorite KPIs (as of Dec-2011)

This is the last post in an informal 3-part series covering what Mike Amer, a fellow analyst at Resource Interactive, and I have arrived at when it comes to understanding and using the latest release of Facebook Insights. In this post, I’ll cover what metrics we’re generally gravitating towards as effective ways to measure the performance of a Facebook page.

As many, many, many people pointed out before the latest update to Facebook Insights, Page Likes (or “fan count”), while easy to measure, is not a particularly meaningful metric. As John Lovett would say, it is simply a “counting metric.”

Below are the metrics I’m gravitating to these days as KPIs for a page:

  • Reach and Impressions – pick one or the other, but, if one of your goals for Facebook is to gain exposure for your brand, these are much better measures of exposure than Page Likes. If you’re running Facebook media, you may want to use Organic Reach (or Impressions) to measure the exposure you’re generating through non-paid means while the ads or Sponsored Stories are running (this will undercount the overall exposure slightly, as some of your viral reach is from non-paid activity, but there simply is no way to really tease that out)
  • Engaged Users – if one of your goals for Facebook is to foster dialogue with users, then engaged users is a good measure, because it is a measure of how many people took any actual action related to your page (regardless of whether it “generated a story”); again, if you’re running paid media, you may want to adjust this metric by subtracting out New Page Likes from Ads.
  • Average Post Engagement Rate – this is a second potential KPI for the goal of fostering dialogue with users; you have to get this from the post-level data, but it is simply a matter of dividing the number of engaged users by the total reach of the post and then averaging this for all posts in the reporting period.  This metric does not need to be “adjusted” when paid media is running. It is also a metric for which a page owner really can take direct action to affect by analyzing the virality of the individual posts in the reporting period and developing hypotheses as to what made the posts with the highest/lowest engagement rates different from each other (type of post, time of day, day of week, content, etc.). Those  hypotheses can then be tested with subsequent posts to see if they are validated.
  • People Talking About or Stories Generated – if you are aiming for your users to spread the word about your page through their social graph, then these are KPIs to consider. Keep in mind that a person who generates a story by liking your page is producing a much broader reaching “story” than a person who simply comments on a page post. And, as with Engaged Users, subtracting out New Page Likes from Ads when you’re running paid media will give you a better picture of the non-paid results from the page in the same time period (although there will still be some spillover impact that is not currently possible to eliminate).
  • Average Post Virality – Facebook reports the “virality” of any single post as the number of people talking about the post divided by the reach of the post. It’s a good metric, if something of a misnomer, because “Virality” is really “potential virality but minimal real virality due to Facebook’s EdgeRank algorithms…unless the post is a Facebook Question.”

It’s pretty easy to engineer much more involved metrics by diving into the organic, viral, and paid breakouts…but then you wind up with metrics that are hard for the typical business user to understand.

That’s our take. What metrics are you finding most useful with the new Facebook Insights? What measures are you least able to get that you wish Facebook would add (for me, it’s the ability to break out “viral” metrics into “triggered by paid media” and “not triggered by paid media”)?

Social Media

Facebook Insights — “Viral” Measures and EdgeRank

In my last post, I provided an update as to how to interpret the primary measures and dimensions (organic/paid/viral) that are available in the latest iteration of Facebook Insights. While digging into those dimensions, my fellow Resource Interactive analyst, Mike Amer, stumbled across some mild unpleasantries that don’t quite square with how Facebook talks about brand pages in their formal documentation.

On the one hand, Facebook would have us thinking that it’s all about virality. That’s one of the reasons they’ve made “Friends of Fans” such a prominent (if laughable) metric!

To recap, the viral reach of a page or a post is the number of unique people who were exposed to content as a result of another user generating a story (“talking about” the page or post – liking, sharing, commenting, etc.). This differs from organic reach, which is the number of unique people who visited the page or saw an item in their news feed or ticker as a direct result of the page posting the content.

Here are a couple of dirty little clarifications and secrets about virality, though:

  • The most common type of viral reach is from someone liking your page despite Facebook’s insinuations that getting people to like and comment on your page posts will tap into that ginormous “friends of fans” number…those user actions tend to go nowhere. When someone likes your page, though, that generates a story that has a meaningful viral reach (unfortunately, that is a one-time viral exposure — that same user may comment on 10 page posts over the next week and the viral reach generated from those actions will be virtually nil).
  • A page’s virality is dramatically impacted by paid media – If a Facebook Ad for the page is run and a user is exposed to the ad, then that exposure counts as 1 person towards the page’s Paid Reach. If the person clicks the Like button, Facebook will record that as a Like Source of “ads” (why they don’t have that data field name capitalized bothers my OCD, FWIW). But, a good chunk of their friends are going to get an item in their ticker that the person liked the page. All of those friends being exposed get counted as viral reach and impressions.
  • Oh…yeah…and Facebook Questions – Facebook Questions are the single type of Facebook page post that appear to drive meaningful viral reach (presumably, because the Ask friends action is more valued by Facebook than other actions such as standard likes, comments, and shares). Questions are good for that! Unfortunately, we’ve seen several cases over the last month across different pages where the Organic Reach of Facebook Questions was reported as dramatically lower than the typical reach for a status update on the page. It’s unclear whether those lower numbers reflect reality or whether they are simply a Facebook Insights glitch

All this is to say that viral reach is messy (…and don’t take what Facebook espouses at face value).

In my last post in this unofficial series, I’ll provide a list of the KPIs we’ve been gravitating towards with our clients and why.

Analytics Strategy, Social Media

Counting ROI in Pennies with Social Media

“Goddam money. It always ends up making you blue as hell.” ~ Holden Caufield, The Catcher in the Rye

That is…if you let it.

During our webinar yesterday Activating Your Socially Connected Business, Lee Isensee (@OMLee) and I caused a minor flurry on Twitter when I Tweeted about the results Lee showed from the IBM/comScore social sales data from Cyber Monday. The findings revealed that $7 million dollars captured on Cyber Monday 2011 in online sales was directly attributable to social media. This makes up 0.56% of all online sales on Cyber Monday 2011.

The skeptics were quick to pounce on the paltry figure, with #WhoopDeeFrigginDo’s and “rounding error” rhetoric (see the Storify.com synopsis). And I agree, that half a percentage point, by anyone’s count isn’t a whole lot of impact. Even when it equates to $7 million bucks in a $1.25 billion dollar day of digital shopping. However folks, remember that all online sales last year represented just 7.2% of holiday cha-chingle in retailers’ pockets. According to comScore’s numbers that’s $32.6B in digital business over the 2010 holiday shopping season. Yet, how many of the total $453B in last year’s holiday sales…or this year’s forecasted $469B in holiday sales…were/will be ***influenced*** by online channels? The answer is a lot.

According to research firm NPD, 30% of all holiday shoppers plan to buy online this year, with the numbers even larger for high income households. Further, a full 50% of shoppers will turn to the Internet to research products prior to buying this year. And this that doesn’t include another 20% that will rely on consumer reviews and 4% who will turn to social media for their pre-buying intel. As we know, many of these shoppers will hit the stores with smartphones in hand, ready to get info or tap into their social networks as necessary.

My point is that if you’re so narrowly focused on social media that the only reason you’re in it is for the money…then you’re missing the point. Social media is today – and will be tomorrow – an enabler. It’s a method to engage with people on a meaningful level and to allow them to engage with one another. As a brand, if you can’t see this then you’re totally missing the point. It’s not all about the Benjamin’s. Social media ROI is important, but trying to pin everything down to bottom line metrics will have you “blue as hell” when it comes time to tally the numbers.

Instead, work to identify other Outcomes for your social media objectives that ***don’t have*** direct financial implications, but that ***do have*** business value. Demonstrating that your social channels reduce call center costs, elevate customer satisfaction, or simply drive awareness of your in-store promotions will deliver value deep within the business.

I’m all for generating ROI from social media activities and making direct revenue correlations when they exist. Yet, in today’s world, social media isn’t just about the bucks. It’s a means to deliver better experiences for the many people who turn to that channel.

If you’re interested in learning more about Activating Your Socially Connected Business, download Chapter 3 from Social Media Metrics Secrets, courtesy of IBM.

Social Media

Understanding Facebook Insights Terminology Redux

When the latest Facebook Insights was released, I quickly put up a post that both tried to explain the new metrics that became available and proposed some probable KPIs.

Well, a few months have passed, Facebook has quietly rolled out some changes to Facebook Insights, and we’ve gotten a chance to actually dive into some of these metrics. This post and the next two are the result of some digging that Mike Amer and I have done on behalf of Resource Interactive and our clients.

Note: This is minimally a post about the web-based Facebook Insights interface. Rather, it is focused on the slightly deeper data that is available behind that interface, which is available by exporting page-level and post-level data or through the Facebook API.

Understanding the Basics – Reach, Talking About, Engaged Users, Consumers

I get a little depressed when I think about the number of times I have read and re-read the same one-line Facebook Insights definitions for various metrics, which have the illusion of being crystal clear on an initial reading, and then get increasingly confusing with each subsequent cycle of trying to actually interpret the data.

I continue to think that the best way to understand the main new metrics is via a Venn diagram. But, the page-level Venn diagram has evolved a bit since my initial post, as Facebook quietly added a page-level Engaged Users metric, which is the union of People Talking About and Consumers. I also think that Facebook changed the definition of Consumers to include clicks that generated a story, but I haven’t tracked down old printouts to fully confirm.

Below is an updated Venn diagram for page-level Facebook metrics.

And, below is an (unchanged) Venn diagram for post-level metrics:

What About Paid/Organic/Viral (especially Viral!)?

At both the page level and the post level, Facebook breaks down a number of metrics by “paid,” “organic,” and “viral” Here’s how I’ve been describing these when it comes to page-level reach:

  • Organic – unique people who visited the page or saw an item published by the page in their news feed or ticker
  • Viral – unique people who were exposed to content as a result of another user generating a story (“talking about” the page – liking the page, sharing a post, etc.)
  • Paid – unique people who saw a Sponsored Story or Ad pointing to the page

A single user can be reached by multiple ways in a given time period (e.g., they saw a post from the page that they’re a fan of in their news feed – organic – and then saw that a friend of theirs responded to a question on the page in their ticker – viral – and then was exposed to a Facebook Ad – paid), so, when it comes to reach, the sum of organic reach plus viral reach plus paid reach is greater than the total reach. Reach measures are always de-duped to be a count of unique users.

When it comes to impressions, though, there is no de-duping, so the sum of the different types of impressions equals the total impressions.

In my next post, I’ll dig into “virality” a little deeper (it turns out to be a bit of a bugaboo metric, but it’s also one that turns out to reveal some sneaky little unpleasantries about Facebook’s EdgeRank algorithm).

Conferences/Community

The Evolution of Web Analytics Wednesday

I’ve been thinking a lot about some of the community events that my partners and I have had the opportunity to create over the years lately. While a lot of the focus recently has been on ACCELERATE — the web analytics industry’s first free conference series — our efforts more will turn back to Analysis Exchange and Web Analytics Wednesday as we roll into 2012.

I wanted to discuss the latter event.

Since co-founding the event with June Dershewitz in 2005, Web Analytics Wednesday has impacted web analytics practitioners, consultants, and vendors around the globe. Since January 1, 2009, over nearly 12,800 individuals around the globe have attended 524 different events … all free, almost all sponsored, and all designed to create local community value for web analytics professionals.

The best thing about Web Analytics Wednesday, at least in my opinion, is that nobody owns the event series! I get calls all the time from vendors asking about having an event in a city or on a date, and I have to admit I cannot really help them because we are only the brand steward for Web Analytics Wednesday, not the owners, and Web Analytics Wednesday ONLY HAPPENS because of the generosity and commitment of the broader web analytics community.

I think this is amazing.

Dozens of sponsors, hundreds of hosts, and thousands of participants, all coming together to make something happen. The list of hosts is too long to write out, but 99% of them are generous, selfless, and incredibly hard-working individuals who spent their free time organizing these events without any thought of compensation or recognition. When they could be with their families, they are working on behalf of the community. When they could be relaxing, they are organizing.

I think this is humbling.

Web Analytics Wednesday has become a nearly frictionless system, one that anyone, anywhere can help to make happen, and one that has helped people find jobs, find employees, find connections, and find new friends.

I think this is freaking awesome.

Sure, we have guidelines … we ask that hosts use our system for registration, we ask that events not charge money, and we ask that sponsors be treated fairly and appropriately at events, and we ask that when Global Funds are used that hosts take pictures for our Flickr Photo Group so that everyone can share in the fun. We expect Web Analytics Wednesday hosts to be cool, to be honest, and to do what they do for “the community.”

So few people have trouble with this model, the exceptions just become noise in the background.

What’s more, we have big plans for Web Analytics Wednesday in the coming year! Where markets have started to languish, Adam, John, and I have started stepping in and offering willing hosts help to reinvigorate their events. Where smaller events have started to grow, the Global Fund has been providing more and more money for reimbursement, and where we see synergies between our other efforts and those of associations and brands we respect and trust, we have been working to organize larger and more diverse events.

And we are just getting started.

If you’re new to Web Analytics Wednesday, here are the five most important things you should know about getting an event started in your town or community:

  1. Web Analytics Wednesday is FREE and OPEN. By design, Web Analytics Wednesday events are open to all practitioners of web analytics and related disciplines and, thanks to the generous support of IQ Workforce and dozens of other companies, always free!
  2. Web Analytics Wednesday belongs to everyone. We do not own Web Analytics Wednesday events, we are only shepherds of the brand, working to ensure consistency across a diverse global analytics community. Anyone willing to follow our very simple guidelines can establish a WAW chapter in their town.
  3. Web Analytics Wednesday is what you make it. Because everyone owns Web Analytics Wednesday, the event is whatever the local community wants it to be. In some cities, WAW happens over lunch. In others, in nightclubs. Sometimes there are presentations, sometimes not.
  4. Web Analytics Wednesday is a state of mind. These events are about local practitioners gathering together, not about a day of the week. Any day can be “Web Analytics Wednesday” … if you’re willing to put in the effort.
  5. Web Analytics Wednesday is a profitless system. Again by design, and with specific intent, nobody makes money off of Web Analytics Wednesday. Regardless of who buys the drinks, nobody — including Analytics Demystified — makes a single, solitary penny off of these events.

This last point is important — if only because some people simply don’t seem to understand.

Every year generous sponsors like IQ Workforce, Coremetrics/IBM, SiteSpect, and dozens more agree to help pay for Web Analytics Wednesday events around the world. And every year my firm (Analytics Demystified) contributes hundreds of hours to ensure that these events go off smoothly. Tens of thousands of dollars are spent to entertain web analysts in great cities like Boston, Chicago, San Francisco, Hong Kong, Sydney, London, and hundreds more. But nobody working on these events — from the mightiest sponsor to the most humble host — gets any compensation in return.

Why do we do this? Why give our money and time to something that won’t make us money? Why did we bother to help create an event series that wouldn’t line out pockets and pay our hourly consulting rate? Simple …

Because we truly care about the web analytics community.

We created Web Analytics Wednesday with June Dershewitz because there was a need back in 2005. We created Web Analytics Wednesday because our community was growing in a strangely fragmented way. We created Web Analytics Wednesday because we could.

I sincerely hope that all of you who have sponsored, hosted, and participated in a Web Analytics Wednesday over the last seven years will continue to do so for years to come. At Analytics Demystified, our commitment is to what is right and just when it comes to this event series and, more importantly, to continue to help evolve and improve Web Analytics Wednesday to ensure that analysts everywhere are able to enjoy and appreciate the same community spirit that we enjoy every time we attend one of these events.

I welcome your comments.

Adobe Analytics, Social Media

Google’s New Social Data Hub

Google’s Eric Schmidt appeared today at LeWeb 2011 and dropped some notable quotes during his interview with conference organizer Loic Le Meur (@loic), including this prescient perspective: “It’s reasonable to say that in the future, the majority of cars will be driverless or driving-assisted.” Foreshadowing perhaps? Could be…but closer to reality:

Google’s Executive Chairman also quipped, “It’s easier to start a revolution and more difficult to finish it.” Google should know. They’ve been revolutionizing the way in which consumers interact on the Web since their inception and news posted today following the LeWeb chat follows suit.

The news reveals a new initiative launching today called the Social Data Hub. What’s even more exciting is the Google Analytics Social Analytics reporting to appear sometime next year. While the details were somewhat vague, I got the inside scoop and what was published should be enough to incite a minor frenzy in the Social Analytics circles.

The “Social Data Hub” is a data platform that is based on open standards allowing Google to aggregate public social media posts, comments, tags, and a plethora of other activities using ActivityStream protocol and Pubsubhubbub hooks. (Yea, that’s a real thing…I had to look it up too.) Early partners in the initiative include social platforms such as Digg, Delicious, Reddit, Slashdot, TypePad, Vkontakte, and Gigya among others. Of course Google’s own social platforms, Google+, Blogger, and Google Groups are included as well. Noticeably absent from the list are social media moguls like Facebook, Twitter, and LinkedIn who have yet to buy into the new Googley idea of a Social Data Hub.

So What…?

If you’re scratching your head wondering how this is different than Google just trying to get more of the world’s data, you’re not alone. At first glance this may seem like yet another big enterprise ploy to get more data (and oh yeah, Don’t be evil). Well, I see this as a huge win for marketers, bloggers, publishers and anyone else trying to discern the impact of social media marketing across the multitude of channels and platforms available today. Currently, most marketers are forced to evaluate their social media activities through the lens that the platform (or their social monitoring tool) offers. Typically this yields low-hanging counting metrics which can be of some value, but more often than not end up as isolated bits of information that don’t provide business value.

Getting at this all important business value in many cases requires wrangling the metrics into another system, processing data and just generally working hard to gain some incremental insight. This is laborious work for the average marketer, so it’s no wonder that eConsultancy just reported that 41% of marketers surveyed had no idea what their return on investment was for social media spending in 2011. Yikes!

Google’s new Social Data Hub – coupled with Google’s Social Analytics reporting – has the potential to knock the socks off these unknowing marketers. By aggregating data from multiple social platforms into the Social Data Hub, they have the ability to make comparisons across platforms to show which channels are driving referrals, which are generating the most interactions, and which are potentially not worth investing in. It’s not that big of a stretch to imagine Google linking this information to data within their Google Analytics product such as Adwords, Goal completion rates and cool new flow visualizations. If/when Google applies the lens of their analytics tool to this new aggregated data set, look out marketers — you just hit the jackpot! Of course, I’m speculating here, but the possibilities are intriguing for a Social Analytics geek like me. That is of course, if platforms open their APIs to the Social Data Hub. A big if…

So Why Would a Platform buy into the Social Data Hub?

Well, it’s questionable if Facebook ever will opt in for this system so I wouldn’t hold your breath on that one. However for other social platforms, being part of the hub has some distinct advantages. They get to prove their value by partnering up with one of the only solutions on the Web that is capable of providing real comparative data on the performance of social channels.

This is a no-brainer for fledgling platforms that want to increase their visibility and even for established players, opting into Google Social Hub could mean the difference in gaining advertising dollars from skeptical marketers. While the big dogs in social media may take a while to come around, I see this new Hub as a potentially great equalizer for understanding the impact of social media as it relates to referrals for on-site activities which can ultimately lead to conversions and bottom line impact.

While today’s announcement may be just a small ripple in the social media pond, I see big waves building for Marketers. But that’s just my take on the disruptive and revolutionary force that is Google…

If you want in on the action, here’s a link to request access to the private beta for Google’s Social Analytics Reporting: https://services.google.com/fb/forms/socialpilot/

And here’s one to for platforms to join the Social Data Hub: http://code.google.com/apis/analytics/docs/socialData/socialOverview.html

Adobe Analytics

Date Stamp Variable [SiteCatalyst]

I was recently working with a client that had a unique situation arise. This client is well-versed in the usage of the Adobe Discover product and frequently takes advantage of its ability to segment by date. For those unfamiliar with this feature, you might use it to address the following scenario: “I’d like to build a segment of people who filled out a form in the third week of January 2011, but I want to see their behavior for the months of February, March and April.” Here is how this segment could be built using Discover:

This functionality is cool since you can use it to limit your population to folks who took some action in a specific time period and then observe their subsequent behavior across a future time period. Another example might be the desire to see purchase behavior of people in Q4 who looked at products in Q3.

However, the challenge facing this client is that very few people in the organization had access to Discover so they wanted to have the ability to apply this date-based segmentation to their SiteCatalyst reports to which everyone had access (and take advantage of the new v15 segmentation capabilities). I hadn’t thought about doing this in SiteCatalyst due to its segmentation limitations (see below), but after contemplating a bit, I came up with a cool trick that should allow SiteCatalyst users to take advantage of this Discover functionality. If this is of interest to you, please read on…

Date Stamp Variable

In order to build a segment that crosses multiple visits, the obvious starting point is the Visitor container within SiteCatalyst’s Segmentation tool. If you want to select a Visit in one time frame, but look at data for another time frame, you will need to use a Visitor container and nest a Visit container and/or Success Event container within it. In the preceding example, we would want to create a Visitor container, but nest a Visit container within it in which the visitor had a Visit where a Form was completed in a specific week of the month of January. Sounds easy right?

Unfortunately, it isn’t as easy as you’d think, because there is no way to segment on a date or month within SiteCatalyst like you can in Discover. Therefore, the trick is to pass the date to a SiteCatalyst variable within each Visit. I suggest you add one new eVar and one new sProp and set the date on every page. In addition, you can easily create a SAINT Classification for each date which rolls these dates up into weeks, months or years as needed.

Once we have set the date to a variable, let’s see an example of how we would create the aforementioned segment from within SiteCatalyst. First, we grab the Visitor container, then we nest a Visit container and within that Visit, we nest a Form Completion Success Event. To narrow down the Form Completion to a specific week in January, we can use our new Date Stamp variable (eVar or sProp version):

Of course, as I mentioned earlier, it may be easier to classify these variables and segment on them by week or month. This process would be identical to the segment shown above, but instead, would use a Classification of the Date Stamp variable. Here is an example of a SAINT Classification of the Date Stamp variable:

If you’ve read my past blog posts, you will soon realize that this trick is similar to the Time-Parting plug-in I described years ago. In fact, it is really just a variation on that, but without the time of the day. However, limiting the values to just the date makes the data much more manageable and more easily classified. The use of this, plus segmentation allows you to mimic what has been possible in Discover for a while so if you have lots of SiteCatalyst users, give this workaround a whirl…Enjoy!

Adobe Analytics, Reporting

v15 Segmentation vs. Multi-Suite Tagging [SiteCatalyst]

With the arrival of SiteCatalyst v15, one of the most intriguing questions is whether or not clients should take advantage of segmentation and replace the historic usage of multi-suite tagging. This is an interesting question so I thought I’d share some of the things to think about…

Multi-Suite Tagging Review

As a quick refresher, if you have multiple websites, it has traditionally been common to send data to more than one SiteCatalyst data set (known as report suites). The benefits of this multi-suite tagging were as follows:

  1. You could have different suites for each data set (i.e. see Spain data separately from Italy data)
  2. If you sent data to many sub-suites and one global (master) report suite, you could see de-duplicated unique visitors from all suites in the global report suite
  3. If you wanted to, you could see Pathing data across multiple sites in the global report suite to see how people navigate from one website to another
  4. You could create one dashboard and easily see the same dashboard for different data sets in SiteCatalyst or in Excel
  5. You want to see metrics at a sub-site level, but also roll them up to see company totals in the global report suite

As you can see, there are quite a few benefits of multi-suite tagging and most large websites tend to do this as a best practice. Of course, where there is value, there is usually a cost! Since you are storing twice as much data in SiteCatalyst, our friends at Omniture (Adobe) have always charged extra for doing this, but normally these “secondary server calls” are charged at a dramatically reduced rate.

Along Comes Instant Segmentation

However, once SiteCatalyst v15 came out, it brought with it the ability to instantly segment your data. Suddenly, you have the capability to narrow down your focus to a specific group of visitors. Therefore, many smart people started asking themselves the following question:

“If I track the website name on every page of every one of my websites, why can’t I just send all data to one global report suite and build a segment for each website instead of paying Omniture extra money to collect my data twice through multi-suite tagging?”

If you look at the list of multi-suite tagging benefits above, you can see that you can accomplish pretty much all of them by simply creating a website segment. For example, if you currently pass data to a global report suite and an Italy report suite, you could simply pass the phrase “Italy” or “it” on every page and build the following Italy segment:

Doing this would narrow the data to just Italy traffic and you don’t have to pay Omniture any extra money! Most clients I have spoken to are very interested in this concept since it will allow them to move some budget to other things they might need (like more analysts or A/B Testing). I think many companies are taking a “wait and see” attitude to this while they get comfortable with SiteCatalyst v15. However, I expect that in the next twelve months, many large enterprises will decide to go this route in order to save a little money and simplify their implementations (one can only dream about not having to keep 50-100 report suites consistent in the Admin Console!). To date, I have not heard Omniture’s stance on this, but I expect that they are not opposed to companies doing this, but will probably not broadcast this concept too loudly since they will lose some recurring revenue as a result.

Any Downsides?

While it is still early days for SiteCatalyst v15, I have tried to think about what, if any, the downsides might be from throwing away multi-suite tagging in favor of an instant segmentation approach. While I hate to rain on the parade of those who want to move forward with this, I have found a few potential downsides that I think you should consider. I don’t think any of these will dissuade you, but I like to present both sides of the story so you can make an informed decision!

The first downside I can see is that moving to one global report suite will make the creation and usage of segments inherently more difficult. For example, let’s say that you create an Italy segment as shown above. That works well if you are in Italy and want to see all Italy traffic. But what if you are in Italy and want to see all first time visitors from a specific list of keywords who have abandoned the shopping cart. That is a semi-complex segment and you have to be careful to include the Italy part of the segment at the same time! Creating segments is tricky enough, but if you use segments to split out countries (or brands), you have to build even more complex segments to take these into account. Should you use an AND clause, an OR clause, combine Visit containers, use a Visitor container, etc? These are tricky questions for everyday end-users, while having a separate report suite (data set) for each country allows you to simplify your segments and just segment within that report suite and not worry about the additional country container. For advanced SiteCatalyst users, this nuance shouldn’t be a showstopper, but it can definitely trip up novice users and is something that should be considered.

Another downside is a lack of security around your data. While you can add security controls to report suites, you cannot do the same when it comes to segments within one master report suite. This means that if you use the one-suite approach, anyone who has access to that suite can see any data within it. You can lock down success events and sProps in the Admin Console, but that is the limit of what you can do. Security remains one of the key reasons why companies continue to use multiple report suites.

Lastly, if you work for a multi-national company, individual report suites allow you to use a different currency type for each suite. This means that a german-based site can use Euros, while a British site can use Pounds. When you send data to a global report suite, these currencies are translated into the one used for the global report suite (i.e. US Dollars). However, if you use only one suite and segmentation, you lose the ability to see data in different currencies. You can use the report settings feature to translate what you see in the interface into your own native currency, but this is much different than seeing the data collected in a native currency. The former simply translates historical data using today’s exchange rate, while the latter uses the currency rates associated with the date that currency was collected. Obviously, the latter is the more accurate approach.

Final Thoughts

So there you have it. Some of my thoughts on this monumental decision that many large SiteCatalyst customers will have to make over the next year. What do you think? Will you take the plunge? Have you thought of any other benefits and/or downsides of making the switch? If so, leave a comment here…

Conferences/Community, General

Are you in Google+? We are!

Just a quick note at the end of the Thanksgiving holiday to encourage those of you who are still using Google+ to go and circle our new brand page for Analytics Demystified:

Circle Analytics Demystified in Google+

We have been sharing lots of information about our recent ACCELERATE conference in San Francisco. Moving forward we hope to share more “quick takes” and multimedia content in Google+ as well as hosting Hangouts with greater and greater regularity to discuss the key topics of the day.

Anyway, I hope if you’re in the U.S. you had a relaxing Thanksgiving and if you’re elsewhere in the world you enjoyed the quiet that happens when the U.S. goes offline.

Analytics Strategy, Social Media

Reflections on the Inaugural #ACCELERATE Conference

 

On Friday, November 19, 2011, the good folk over at Analytics Demystified experimented with a new format for a digital analytics conference, dubbed #ACCELERATE. The key features of the event:

  • It was entirely free to attendees (it was sponsored by TealeafOpinionLab, and Ensighten)
  • It lasted a single day
  • It had two distinct presentation formats — a 20-minute format and a 5-minute format

The 20-minute presentations were  in a “10 Tips in 20 Minutes” format on topics that the organizers selected and then recruited speakers to present. The 5-minute presentations were left entirely up to the presenter when it came to topic selection, but they were encouraged to bring a “Big Idea” and make it “FUN.”

I’ve actually found myself doing more reflection on the conference structure, format, and details than I’ve found myself mulling over the content itself. I’d find that troubling if it weren’t for the fact that I picked up a solid set of intriguing and re-usable nuggets from the content. And, I’ve seen a few blog posts already that do a great job of recapping the event:

  • Michele Hinojosa’s Top 10 Takeaways plays with the “list of 10” format of the event by listing three different sets of 10 takeaways (she left off her own session which provided one of the enduring images for me when she plotted the four different “types” of digital analytics jobs — industry, vendor, agency, consultant — on a 2×2 grid that illustrated how the experiences differ; it’s a handy graphical view of the career development guide she spearheaded for the WAA earlier this year)
  • Corry Prohens’s review of the event recaps the content session by session (but, of course, left out his own excellent session on how to go about recruiting and hiring the right digital analyst for the job).
  • Gabriele Endress recapped the event as well, including a “top 5 learnings” that are spot-on when it comes to the key realities of the dynamic world of digital analytics

I really don’t have much to add to those summaries. The content was great, and I’ve walked away with an array of actions/requests/hopes:

  • I’ve secured a copy of June Dershowitz’s presentation and the blog post that inspired it (top geek humor from the event: “?q=<3”)
  • I’ve prodded Michele to elaborate on her 2×2 grid
  • I’ve been mulling over the vendor-user relationship as described by Ben Gaines (while I have been critical of technology platforms, I also think most vendors with whom I’ve worked closely would put me at least marginally above average on the collaboration/partnership front)
  • I’ve re-cemented Justin Kistner in my brain as my go-to resource for all things Facebook
  • I’m looking forward to Chicago and fervently hoping that Ken Pendergast (or someone) takes another run at making the case for one of the enterprise web analytics vendors to offer a freemium option (I’ve heard that that’s been bandied about over the years at Adobiture, but it’s never been something they’ve been able to effectively justify)

That’s all of the stuff I’m not going to cover in this post. Instead, I’m going to cover more of a meta analysis of the event — a range of factors that made the event stand out and positioned it for on-going evolution and excellence.

Social Media Integration

Social media was heavily incorporated into the event:

  • Twitter-friendliness Part 1 — the event’s name itself — #ACCELERATE — was a ready-made Twitter hashtag. That was clever, as it meant that all Twitter references to the event automatically used Twitter conventions that made the content easy to find, follow, and amplify.
  • Twitter-friendliness Part 2 — throughout the day, Eric Peterson encouraged attendees to use both #ACCELERATE and #measure as they tweeted, and there were incentives for participants to tweet (with quality tweets) both before and during the event (with winners selected using Twitalyzer and TweetReach). This had the effect of #ACCELERATE dominating the #measure world for the day (at one point, TweetReach reported that over 70% of all #measure tweets for the day also included #ACCELERATE in the tweets). That meant that no one who is at least nominally following the #measure hashtag could fail to be aware of the event and aware of the fact that it was a very “socially active” conference.
  • Twitter-maybe-not-so-friendliness Qualifier — the slightly unfortunate side effect of the “10 tips” presentation format, combined with the tweet encouragement, was that it was really easy to simply tweet the title of each “tip,” which often really weren’t all that useful without listening and re-articulating the presenter’s explanation of the tip. A tweet I saw from a non-attendee asked a good question on that front:

“…most of the #measure tweets today were about #ACCELERATE… but was it always relevant?”

  • Post-event buzz bounty — Eric tacked on an incentive for conference attendees to write about (either publicly or privately in an email) their experiences at the event, with the Analytics Demystified team being the judges of the “best” write-up. I suspect that will result in a higher number of blog posts than would otherwise have occurred.

Overall, it was a big win on the Twitter front — I haven’t been to a conference that so actively leveraged the platform both for pre-event buzz generation and during-event content sharing (and further buzz generation). See the last section of Michele Hinojosa’s post for more detail on the Twitter activity.

Presentations Functioning on Two Levels

When it came to the presentation structure, the organizers bent over backwards to set the speakers up for success. In his recap of the event, Corry Prohens credited Craig Burgess with the following observation:

“The conference was also a study on presentation styles and techniques. How often do you get to see 26 presentations in a day? It is a rare opportunity to spot trends and take note of what works. In a field where we all have to present what we know (to clients, stakeholders, etc.) this was a big value-add to the digital measurement insights.”

This was an excellent point. Any conference is going to include sessions that stand out as being fantastic, as well as a few sessions that fall flat. One notable exception (qualifying full disclosure: it’s a conference I’ve never attended): TED.  Whether Eric and company consciously drew inspiration from TED or not, I don’t know, but there are two taglines on the TED home page that could easily be applied to the aspirations for #ACCELERATE:

“Ideas worth spreading”

“Riveting talks by remarkable people, free to the world”

By packing so many sessions into a single day and enforcing brevity (out of necessity), #ACCELERATE had a great pace and kept the attendees engaged for the entire event. Presenters were pushed to bring their “A” game to their sessions, both by repeated reminder-admonitions from Eric, as well as by the inclusion of audience-awarded $500 Best Buy gift cards for the top session of each format.

The presentations were set up to effectively convey useful and engaging content. At the same time, the presentations were set up to give the presenters a set of liberating constraints — establishing distinct guardrails for the content that then empowered the presenters to really focus in on the content and the way they communicated it. This benefited the presenters, certainly, by helping them hone the craft of presenting (that was my experience, at least), but it also benefited the audience by exposing them to a large number of presenters in a concentrated period. I hope everyone took away a few useful nuggets that they can incorporate into their own future presentations (internally or at conferences).

I haven’t attended a single conference in the last 18 months where one of the sub-themes of the conference wasn’t, “As analysts, we’ve got to get better at telling stories rather than simply presenting data.” There is real value in a conference that is designed to help analysts develop their storytelling chops.

Audience Participation

Having the audience directly vote for the winning presentation was another innovation from the event. While it is not at all unheard of to have audience-based voting on presentations, the fact that #ACCELERATE put this at the forefront was something new for digital analytics conferences, as far as I’m aware.

OpinionLab’s DialogCentral platform was leveraged to allow real-time voting and feedback on each session as it occurred. I saw a demo of DialogCentral over a year ago, found it intriguing, and then could never remember what it was called or what ever happened to it, so it was good to see it put into action. Any audience member who had a smartphone could quickly navigate to a mobile-optimized site and vote the presentation on a 5-point scale, leave an open-ended comment, and leave contact info if desired.

There were some glitches on that front, in that there were some participants who did not have smartphones (well, 2 or 3), and at least one attendee reported that the system did not work on her Blackberry. Overall, the voting occurred in smaller numbers than I think the organizers hoped, but it was a great idea and it worked perfectly adequately for a first-time attempt.

And…It Was Free

It’s easy to simply rattle off that “free is better” and leave it at that.  As a first-time event, I’m sure the fact that the event was fully sponsor-supported helped make it fill up quickly. The challenge with having a free event is that the registrants have no real skin in the game — it’s easy to sign up first and then figure out if you can actually attend. If you can’t, well, no worries, because it’s no money out of your pocket! Having co-organized Web Analytics Wednesdays in Columbus — also free events — for several years now, I’ve lived with this challenge firsthand. Trying to accurately predict the no-show rate is an art unto itself, which introduces a range of logistical headaches.

At the other extreme from “free,” the major established digital analytics conferences all have hefty price tags, which makes them cost-prohibitive for many potential attendees who are operating in organizations that have extremely limited training and conference budgets (not to mention the personal budgets for analysts who are in between jobs and could really benefit from the networking opportunities at conferences). That, I suspect, leads to misaligned speaker incentives — members of the industry desperately angling for speaking slots so they can reduce the cost of the overall conference attendance rather than because they have something unique and worthwhile to share.

I could totally see #ACCELERATE evolving to have a nominal registration fee — something like $100 would ensure there was a real commitment required by registrants, but it would also make it totally feasible for someone to attend without corporate backing (make it $25 for students, and, heck, provide bartered alternatives where people can blog about the event or get referral credits).

Overall, free is good, and that made the event right-sized — ~300 people was enough to keep a single track, provide plenty of opportunity for worthwhile networking, while also keeping the setting relatively intimate.

I’m looking forward to Chicago!

Analytics Strategy

Gilligan Meets Super #ACCELERATE — Recreated

I had a ball at the inaugural #ACCELERATE event last Friday, created and hosted by Analytics Demystified and sponsored by OpinionLab, Tealeaf, and Ensighten. I was lucky enough to snag one of the Super #ACCELERATE sessions — 12 presenters, 5 minutes each — that closed out the day.

The instructions we got from Eric Peterson for the Super #ACCELERATE sessions were simple and clear (and reinforced multiple times):

  1. NO MORE than 5 minutes
  2. One BIG IDEA
  3. Have FUN

With that, I noodled on a variety of topics and then decided to use the opportunity to try to bring together a couple of thoughts I’ve had over the past six months to see if I could coherently articulate how they could all play together in an envisioned future.

Several people asked for a reproduction of the presentation, so I’ve recorded it as a video with voiceover (you don’t get the added imagery of me standing behind a podium, but I don’t think that overly detracts from the experience). The video version below is 30 seconds longer than 5 minutes because I’ve added an intro slide and a set of credits that were not part of the actual presentation.

The slides themselves are also posted on SlideShare (no audio included).

I’ll have another post (or maybe two) of reflections on the event. I’ll also be eagerly looking forward to the next #ACCELERATE event slated for April in Chicago.

Analysis, Reporting, Social Media

Digital and Social Measurement Based on Causal Models

Working for an agency that does exclusively digital marketing work, with a heavy emphasis on emerging channels such as mobile and social media, I’m constantly trying to figure out the best way to measure the effectiveness of the work we do in a way that is sufficiently meaningful that we can analyze and optimize our efforts.

Fairly regularly, I’m drawn into work where the team has unrealistic expectations of the degree to which I can accurately quantify the impact of their initiatives on their top (or bottom) line. I’ve come at these discussions from a variety of angles:

This post is largely an evolution of the last link above. It’s something I’ve been exploring over the past six months, and which was strongly reinforced when I read John Lovett’s recent book. As I’ve been doing measurement planning (measurement strategy? marketing optimization planning?) with clients, it’s turned out to be quite useful when I have the opportunity to apply it.

Initially, I referred to this approach as developing a “logical model” (that’s even what I called it towards the end of my second post that referenced John’s book), but that was a bit bothersome, since “logical model” has a very specific meaning in the world of database design. Then, a couple of months ago, I stumbled on an old Harvard Business Review paper about using non-financial measures for performance measurement, and that paper introduced the same concept, but referred to it as a “causal model.” I like it!

How It Works

The concept is straightforward, it’s not particularly time-consuming, it’s a great exercise for ensuring everyone involved is aligned on why a particular initiative is being kicked off, it sets up meaningful optimization work as individual tactics and campaigns are implemented, and it positions you to be able to demonstrate a link (correlation) between marketing activities and business results.

This approach acknowledges that there is no existing master model that shows exactly how a brand’s target consumers interact and respond to brand activity. The process starts with more “art” than “science” — knowledge of the brand’s target consumers and their behaviors, knowledge of emerging channels and where they’re most suited (e.g., a QR code on a billboard on a busy highway…not typically a good match), and a hefty dose of strategic thought.

The exact structure of this sort of model varies widely from situation from situation, but I like to have my measurable objectives — what we think we’re going achieve through the initiative or program that we believe has underlying business value — listed on the left side of the page, and then build linkages from that to a more definitive business outcome on the right:

It should fit on a single page, and it requires input from multiple stakeholders. Ultimately, it can be a simple illustration of “why we’re doing this” for anyone to review and critique. If there are some pretty big leaps required, or if there are numerous steps along the way to get to tangible business value, then it begs the question: “Is this really worth doing?” It’s an easy litmus test as to whether an initiative makes sense.

What I’ve found is that this exercise can actually alter the original objectives in the planning stage, which is a much better time and place to alter them than once execution is well under way!

Once the model is agreed to, then you can focus on measuring and optimizing to the outputs from the base objectives — using KPIs that are appropriate for both the objective and the “next step” in the causal model.

And, over time, the performance of those KPIs can be correlated with the downstream components of the causal model to validate (and adjust) the model itself.

This all gets back to the key that measurement and analytics is a combination of art and science. Initially, it’s more art than science — the science is used to refine, validate, and inform the art.

Conferences/Community

First ever ACCELERATE is this happening TODAY!

It seems a long time coming but the moment is nearly here: Analytics Demystified’s first conference of our own happens TODAY, Friday, November 18th, in San Francisco. The prep work is largely done, the conference has been full for two months, and we have a 40 person wait list of folks hoping to be able to join us.

Amazing.

Thanks to the generosity of Tealeaf, Ensighten, and OpinionLab, plus our great Web Analytics Wednesday sponsors (iJento, ObservePoint, Causata, and Coremetrics/IBM) the party starts Thursday at 6:00 PM and the education and networking starts Friday at 9:00 AM. ACCELERATE is full, but there is still room to join us at Web Analytics Wednesday if you’re in town.

If you’re not able to join us here are a few ways you can participate virtually:

  1. We will be encouraging people to share insights via Twitter on both the #ACCELERATE and #measure hashtags. Set your favorite Twitter client to monitor these tags and watch the stream.
  2. We created a Twitter list of many of the participants Twitter handles. Follow this list and see what folks at the event are sharing.
  3. We will be trying to post content to our new Google+ page for Analytics Demystified. Admittedly, we don’t really use Google+ that much but since it allows for longer-form sharing and photos we’re going to try.

Those of you who couldn’t make it to San Francisco should pay attention to these streams later in the day if nothing else: we will be announcing the next ACCELERATE location for 2012 and opening up registration!

If you have any questions about the event now would be a really good time to ask them. Email us directly and we will do our best.

Analytics Strategy, General

Do You Trust Your Data?

A recurring theme in our strategy practice at Analytics Demystified is one of data quality and the businesses ability and willingness to trust web analytics data. Adam wrote about this back in 2009, I covered it again in 2010, and all three of us continue to support our client’s efforts to validate and improve on the foundation of their digital measurement efforts.

Not that I am surprised — far from it — given that the rate at which senior leadership and traditional business stakeholders have been calling us to help get their analytical house in order. It turns out management doesn’t want to “get over” gaps in data quality; they want reliable numbers they can trust to the best of the company’s ability to inform the broader, business-wide decision making process.

To this end, and thanks to the generosity of our friends at ObservePoint, I am happy to announce the availability of a free white paper Data Quality and  the Digital World. Following up on our 2010 report on page tagging and tag proliferation, this paper drills into the tactical changes that companies can make to work to ensure the best possible data for use across the Enterprise. In addition to providing ten “tips” to help you create trust in your online data, we provide examples from ObservePoint customers including Turner, TrendMicro, and DaveRamsey.com, each of whom have a great story to tell about data auditing and validation.

One surprise when doing the research for this document was that multiple companies cited examples of something we have coined “data leakage.” Data leakage happens when business users, agencies, and other digital stakeholders start deploying technology without approval and, more importantly, without a clear plan to manage access to that technology. Examples are myriad and almost always seem harmless — that is until something goes wrong and the wrong people have access to your web traffic, keyword, or transactional data.

The idea of data leakage is one of the reasons that we have teamed up with BPA Worldwide to create the Analytics Demystified GUARDS audit service, and unsurprisingly GUARDS audits include an ObservePoint analysis to help identify possible risks when it comes to consumer data privacy. You can learn more about the GUARDS consumer data privacy audit on our web site.

If you’re being asked about the accuracy and integrity of your web-collected data, if you know you cannot trust the data but aren’t sure what to do about it, or if you suspect your company may potentially be leaking data through tag-based technologies, I would strongly encourage you to download Data Quality and  the Digital World from the ObservePoint site. What’s more, if you need help reseting expectations about data and it’s usage across your business, don’t hesitate to give one of us a call.

Download Data Quality and the Digital World now!

 

Analysis, Analytics Strategy, Reporting

The Analyst Skills Gap: It's NOT Lack of Stats and Econometrics

I wrote the draft of this post back in August, but I never published it. With the upcoming #ACCELERATE event in San Francisco, and with what I hope is a Super Accelerate presentation by Michael Healy that will cover this topic (see his most recent blog post), it seemed like a good time to dust off the content and publish this. If it gives Michael fodder for a stronger takedown in his presentation, all the better! I’m looking forward to having my perspective challenged (and changed)!

A recent Wall Street Journal article titled Business Schools Plan Leap Into Data covered the recognition by business schools that they are sending their students out into the world ill-equipped to handle the data side of their roles:

Data analytics was once considered the purview of math, science and information-technology specialists. Now barraged with data from the Web and other sources, companies want employees who can both sift through the information and help solve business problems or strategize.

That article spawned a somewhat cranky line of thought. It’s been a standard part of presentations and training I’ve given for years that there is a gap in our business schools when it comes to teaching students how to actually use data. And, the article includes a quote from an administrator at the Fordham business school: “Historically, students go into marketing because they ‘don’t do numbers.'” That’s an accurate observation. But, what is “doing numbers?” In the world of digital analytics, it’s a broad swath of activities:

  • Consulting on the establishment of clear objectives and success measures (…and then developing appropriate dashboards and reports)
  • Providing regular performance measurement (okay, this should be fully automated through integrated dashboards…but that’s easier said than done)
  • Testing hypotheses that drive decisions and action using a range of analysis techniques
  • Building predictive models to enable testing of different potential courses of action to maximize business results
  • Managing on-going testing and optimization of campaigns and channels to maximize business results
  • Selecting/implementing/maintaining/governing data collection platforms and processes (web analytics, social analytics, customer data, etc.)
  • Assisting with the interpretation/explanation of “the data” — supporting well-intended marketers who have found “something interesting” that needs to be vetted

This list is neither comprehensive nor a set of discrete, non-overlapping activities. But, hopefully, it illustrates the point:

The “practice of data analytics” is an almost impossibly broad topic to be covered in a single college course.

What bothered me about the WSJ article are two things:

  • The total conflation of “statistics” with “understanding the numbers”
  • The lack of any recognition of how important it is to actually be planning the collection of the data — it doesn’t just automatically show up in a data warehouse

On the first issue, there is something of an on-going discussion as to what extent statistics and predictive modeling should be a core capability and a constantly applied tool in the analyst’s toolset. Michael Healy made a pretty compelling case on this front in a blog post earlier this year — making a case for statistics, econometrics, and linear algebra as must-have skills for the web analyst. As he put it:

If the most advanced procedure you are regularly using is the CORREL function in Excel, that isn’t enough.

I’ve…never used the CORREL function in Excel. It’s certainly possible that I’m a total, non-value-add reporting squirrel. Obviously, I’m not going to recognize myself as such if that’s the case. I’ve worked with (and had work for me) various analysts who have heavy statistics and modeling skills. And, I relied on those analysts when conditions warranted. Generally, this was when we were sifting through a slew of customer data — profile and behavioral — and looking for patterns that would inform the business. But this work accounted for a very small percentage of all of the work that analysts did.

I’m a performance measurement guy because, time and again, I come across companies and brands that are falling down on that front. They wait until after a new campaign has launched to start thinking about measurement. They expect someone to deliver an ROI formula after the fact that will demonstrate the value they delivered. They don’t have processes in place to monitor the right measures to trigger alarms if their efforts aren’t delivering the intended results.

Without the basics of performance measurement — clear objectives, KPIs, and regular reporting — there cannot be effective testing and optimization. In my experience, companies that have a well-functioning and on-going testing and optimization program in place are the exception rather than the rule. And, companies that lack the fundamentals of performance management that try to jump directly to testing and optimization find themselves bogged down when they realize they’re not entirely clear what it is they’re optimizing to.

Diving into statistics, econometrics, and predictive modeling in the absence of the fundamentals is a dangerous place to be. I get it — part of performance measurement and basic analysis is understanding that just because a number went “up” doesn’t mean that this wasn’t the result of noise in the system. Understanding that correlation is not causation is important — that’s an easy concept to overlook, but it doesn’t require a deep knowledge of statistics to sound an appropriately cautionary note on that front. 9 times out of 10, it simply requires critical thinking.

None of this is to say that these advanced skills aren’t important. They absolutely have their place. And the demand for people with these skills will continue to grow. But, implying that this is the sort of skill that business schools need to be imparting to their students is misguided. Marketers are failing to add value at a much more basic level, and that’s where business schools need to start.

Reporting, Social Media

The New Facebook Insights — One More Analyst's Take

Facebook released its latest version of Facebook Insights last week, and that’s kicked off a slew of chatter and posts about the newly available metrics. Count this as another one of those. It’s partly an effort to visually represent the new metrics (which highlights some of the subtleties that are a little unpleasant, although, in the end, not a big deal), and it’s partly an effort to push back against the holy-shit-Facebook-has-new-metrics-so-I’m-going-to-combine-the-new-ones-and-say-we’ve-now-achieved-measurement-nirvana-without-putting-some-rigorous-thought-into-it posts (not linked to here, because I don’t really want to pick a fight).

Basically…We’re Moving in a Good Direction!

At the core of the release is a shift away from “Likes” and “Impressions” and more to “exposed and engaged people.” There are now a slew of metrics available at both the page level and the individual post level that are “unique people” counts. That…is very fine indeed! It’s progress!

Visually Explaining the New Metrics

As I sifted through the new Facebook Page Insights product guide (kudos to Facebook for upping the quality of their documentation over the past year!) with some co-workers, it occurred to me that a visual representation of some of the new terms might be useful. I settled on a Venn diagram format, with one diagram for the main page-level metrics and one for the main post-level metrics.

Starting with page-level metrics:

Defining the different metrics — heavily cribbed from the Facebook documentation:

  • Page Likes — The number of unique people who have liked the page; this metric is publicly available (and always has been) on any brand’s Facebook page.
  • Total Reach — The number of unique people who have seen any content associated with a brand’s page. They don’t have to like the page for this, as they can see content from the page show up in their ticker or feed because one of their friends “talked about it” (see below).
  • People Talking About This — The number of unique people who have created a story about a page. Creating a story includes any action that generates a News Feed or Ticker post (i.e. shares, comments, Likes, answered questions, tagged the page in a post/photo/video). This number is publicly available (it’s the “unique people who have talked about this page in the last 7 days”) on any brand’s Facebook page.
  • Consumers — The number of unique people who clicked on any of your content without generating a story.

A couple of things to note here that are a little odd (and likely to be largely inconsequential), but which are based on a strict reading of the Facebook documentation:

  • A person can be counted in the Total Reach metric without being counted in the Page Likes metric (this one isn’t actually odd — it’s just important to recognize)
  • A person can be counted as Talking About This without being included in the Reach metric. As I understand it, if I tag a page in a status update or photo, I will be counted as “talking about” the page, and I can do that without being a fan of the page and without having been reached by any of the page’s content. In practice, this is probably pretty rare (or rare enough that it’s noise).
  • Consumers can also be counted as People Talking About This (the documentation is a little murky on this, but I’ve read it a dozen times: “The number of people who clicked on any of your content without generating a story.” Someone could certainly click on content — view a photo, say — and then move on about their business, which would absolutely make them a Consumer who did not Talk About the page. But, a person could also click on a photo and view it…and then like it (or share it, or comment on the page, etc.), in which case it appears they would be both a Consumer and a Person Talking About This.
  • A person cannot be Consumer without also being Reached…but they can be a Consumer without being a Page Like.

Okay, so that’s page-level metrics. Let’s look at a similar diagram for post-level metrics:

It’s a little simpler, because there isn’t the “overall Likes” concept (well…there is…but that’s just a subset of Talking About, so it’s conceptually a very, very different animal than the Page Likes metric).

Let’s run through the definitions:

  • Reach — The number of unique people who have seen the post
  • Talking About — The number of unique people who have created a story about the post by sharing, commenting, or liking it; this is publicly available for any post, as Facebook now shows total comments, total likes, and total shares for each post, and Talking About is simply the sum of those three numbers
  • Engaged Users — The number of unique people who clicked on anything in the post, regardless of whether it was a story-generating click

And, there is a separate metric called Virality which is a simple combination of two of the metrics above:

That’s not a bad metric at all, as it’s a measure of, for all the people who were exposed to the post, what percent of them actively engaged with it to the point that their interaction “generated a story.”

The Reach and Talking About metrics are direct parallels of each other between the page-level metrics and the post-level metrics. However (again, based on a close reading of the limited documentation), Consumers (page-level) and Engaged Users (post-level) are not analogous. At the post-level, Talking About is a subset of Engaged Users. It would have made sense, in my mind, if, at the page-level Talking About was a pure subset of Consumers…but that does not appear to be the case.

KPIs That I Think Will Likely “Matter” for a Brand

There have been several posts that have jumped on the new metrics and proposed that we can now measure “engagement” by dividing People Talking About by Page Likes. The nice thing about that is you can go to all of your competitors’ pages and get a snapshot of that metric, so it’s handy to benchmark against. I don’t think that’s a sufficiently good reason to recommend as an approach (but I’ll get back to it — stick with me to the end of this post!).

Below are what I think are some metrics that should be seriously considered (this is coming out of some internal discussion at my day job, but it isn’t by any means a full, company-approved recommendation at this point).

We’ll start with the easy one:

This is a metric that is directly available from Facebook Insights. It’s a drastic improvement over the old Active Users metric, but, essentially, that’s what it’s replacing. If you want to know how many unique people are receiving any sort of message spawned from your Facebook page, Total Reach is a pretty good crack at it. Oh, and, if you look on page 176 of John Lovett’s Social Media Metrics Secrets book…you’ll see Reach is one of his recommended KPIs for an objective of “gaining exposure” (I don’t quite follow his pseudo-formula for Reach, but maybe he’ll explain it to me one of these days and tell me if I’m putting erroneous words in his mouth by seeing the new Facebook measure as being a good match for his recommended Reach KPI).

Another possible social media objective that John proposes is “fostering dialogue,” and one of his recommended KPIs for that is “Audience Engagement.” Adhering pretty closely to his formula there, we can now get at that measure for a Facebook page:

Now, I’m calling it Page Virality because, if you look up earlier in this post, you’ll see that Facebook has already defined a post-level metric called Virality that is this exact formula using the post-level metrics. The two are tightly, tightly related. If you increase your post Virality starting tomorrow by publishing more “engage-able” posts (posts that people who see it are more like to like, comment, or share), then your Page Virality will increase.

There’s a subtle (but important this time) reason for using Total Reach in the denominator rather than Page Likes. If you have a huge fan base, but you’ve done a poor job of engaging with those fans in the past, your EdgeRank is likely going to be pretty low on new posts in the near term, which means your Reach-to-Likes ratio is going to be low (keep reading…we’ll get to that). To measure the engage-ability of a post, you should only count against the number of people who saw the post (which is why Facebook got the Virality measure right), and the same holds true for the page.

Key Point: Page Virality can be impacted in the short-term; it’s a “speedboat measure” in that it is highly responsive to actions a brand takes with the content they publish

This is all a setup for another measure that I think is likely important (but which doesn’t have a reference in John’s book — it’s a pretty Facebook-centric measure, though, so I’m going to tell myself that’s okay):

I’m not in love with the name for this (feel free to recommend alternatives!). This metric is a measure (or a very, very close approximation — see the messy Venn diagram at the start of this post) of what percent of your “Facebook house list” (the people who like your page) are actually receiving messages from you when you post a status update. If this number is low, you’ve probably been doing a lousy job of providing engaging content in the past, and your EdgeRank is low for new posts.

Key Point: Reach Penetration will change more sluggishly than Page Virality; it’s an “aircraft carrier measure” in that it requires a series of more engaging posts to meaningfully impact it

(I should probably admit here that this is all in theory. It’s going to take some time to really see if things play out this way).

Those are the core metrics I like when it comes to gaining exposure and fostering dialogue. But, there’s one other slick little nuance…

Talking About / Page Likes

Remember Talking About / Page Likes? That’s the metric that is, effectively, publicly available (as a point in time) for any Facebook page. That makes it appealing. Well, two of the metrics I proposed above are, really, just deconstructing that metric:

This is tangentially reminiscent of doing a DuPont Analysis when breaking down a company’s ROE. In theory, two pages could have identical “Talking About / Page Likes” values…with two very fundamentally different drivers going on behind the scenes. One page could be reaching only a small percentage of its total fans (due to poor historical engagement), but has recently started publishing much more engaging content. The other page could have historically engaged pretty well (leading to higher reach penetration), but, of late, has slacked off (low page virality). Cool, huh?

What do you think? Off my rocker, or well-reasoned (if verbose)?

Analytics Strategy, Social Media

QR Codes — How They Work (at least…What Matters for Analytics)

I’ve had a couple of situations in the past few weeks where I’ve found myself explaining how QR codes work and what can/cannot be tracked under what situations. To whit, this post focuses on tracking considerations — not the what and why of QR codes themselves. This is an “…on data” blog, after all!

Nevertheless, the Most Basic of the Basics

A QR code contains data in a black-and-white pixelated pattern. That’s all there is to it. It can store lots of different types of data (only a finite amount, of course), but the most common data for that pattern to store is a URL. For instance, the QR code below stores the URL for this blog post:

Please, DON’T Do What I Just Did!

Here’s the key point to this whole post: the example above is a perfect example of how NOT to generate a QR code.

Two reasons:

  • It will not be possible to track the number of scans of the QR code
  • The QR code is needlessly complex, which requires a larger, more involved QR code

With the QR code above, the QR code reader on a person’s phone reads the underlying URL and routes the user to the target address:

The problem here is that, if you’re using QR codes in multiple places — printed circulars, product packaging, in-store displays, etc. — and they’re sending the user to the same destination URL, you won’t be able to distinguish which of the different physical placements is generating which traffic to that destination URL.

That’s a problem, because, inevitably, you’ll want to know whether your target users are even scanning the codes and, if so, which codes they’re scanning. It would be one thing if QR codes were inherently attractive and added to the aesthetics of analog collateral. But, like their barcode ancestors, they tend to lack visual appeal. If they’re not adding value and not being used, it’s best that they be removed!

Why, Yes, There IS a Better Way. I’m Glad You Asked.

The QR code below sends the user to the exact same destination (this post):

Notice anything different? For starters, the code itself is much, much smaller than the first example above. That’s nice — it takes up less room wherever it’s printed! Designers will hug you (well, they won’t exactly hug you — they’ll still blanch at your requirement to drop this pixelated box into an otherwise attractively designed piece of printed material…but they’ll gnash their teeth moderately less than if they were required to use the much larger QR code from above).

The trick? Well, this new QR code doesn’t include the full URL for this page. Rather it has a much simpler, much shorter, URL encoded in its pixels:

http://goo.gl/H104m.qr

It makes sense, doesn’t it, that a shorter URL like this one will require fewer black and white pixels to be represented in a QR code format? This URL, you see, was generated using http://goo.gl — a URL shortener. You can also generate QR codes using http://bit.ly. Both are free services and both have a reputation of high availability.

Using some flavor of URL shortener is one of those things consultants and tradesfolk refer to as a “best practice” for QR code generation. What’s going on is that the process relies on an intermediate server-side redirect (of which goo.gl and bit.ly are both examples) to route the user to the final destination URL. This alters the actual user flow slightly so that it looks something like the diagram below:

That adds a little bit of complexity to the process, and, depending on the user’s QR code reader and settings therein, he/she may actually see the intermediate URL before getting routed to the final destination. That’s really not the end of the world, as it’s a fairly innocuous step with a dramatic upside. (Technically, this approach introduces an additional potential failure point into the overall process, but that plays out as more of a theoretical concern than a practical one.)

Why Is This Marginally Convoluted Approach Better?

By introducing the shortened URL, you get two direct benefits:

  • A smaller, cleaner QR code (we covered that already)
  • The ability to count the number of scans of each unique QR code

This second one is the biggie. To be clear, this isn’t going to distinguish between each individual printout of the same underlying QR code, but it will enable you to, for instance, identify scans of a code that is printed on a particular batch of direct mail from scans that are printed in a newspaper circular.

How is it doing that, you ask? Well, exactly the same way that URL shorteners like goo.gl and bit.ly provide data on how many times URLs created using them were scanned: when the “URL Shortener Server” gets a request for the shortened URL, it not only redirects the user to the full destination URL, but it increments a count of how many times the URL was “clicked” (and, in the case of a QR code, “click” = “scanned”) in an internal database. You can then access that data using the URL shortener / QR code generator’s reporting system.

But Wait! There’s MORE!

Take another look at the full URL that the shortened URL (embedded in the QR code) is redirecting to:

http://www.gilliganondata.com/index.php/2011/10/12/qr-codes-how-they-work-at-least-for-analytics?utm_source=gilliganondata&utm_medium=qr_code&utm_campaign=oct_2011_blog

Notice how it has Google Analytics campaign tracking parameters tacked onto the end of it? That’s a second recommended best practice for QR codes that send the user to web sites that have campaign tracking capabilities. This is just like setting up a banner ad or some other form of off-site promotion or advertising: you control the URL, so you should include campaign tracking parameters on it! This will enable you to look at post-scan activity — did users who scanned the QR code from the product packaging convert at a higher rate on-site than users who scanned the in-store display QR code? You get the idea.

A Final Note on This — Where bit.ly and goo.gl Come Up Short

The upsides to goo.gl and bit.ly QR code generation is that they’re free and have decent click/scan analytics. The downside is that, once a short URL is generated, the target URL can’t be edited (they have their reasons).

Paid services such as the service offered by 3GVision i-nigma both offer solid analytics and allow QR codes to be edited after the short URLs (which the QR codes then represent) are created. This makes a lot of sense, because a printed QR code may stay in-market for a sustained period of time, while the digital content that supports the placement of that code may need to be updated. Or, say that someone creates a QR code and uses a target URL that is devoid of campaign tracking parameters — with a service like 3GVision’s, you can add the tracking parameters after the QR code has been generated and even after it has gone to print (any resemblance to actual situations where this has occurred is purely coincidental! …or so the blogger innocently claimed…). You can’t go backwards in time and add campaign tracking for scans that have already occurred, but you can at least “fix” the tracking going forward.

As is my modus operandi, this has been a pretty straightforward concept with a couple of tips and best practices…and I’ve turned it into a rather verbose and hyper-descriptive post. <sigh> I hope you found it informative.

Analytics Strategy, Conferences/Community, General

Finally! Standards come to Web Analytics

Last week I had the pleasure of traveling to Columbus, Ohio to participate in Web Analytics Wednesday, hosted by Resource Interactive’s Tim Wilson and generously sponsored by the fine folks at Foresee. We opted for an “open Q&A” format that turned out pretty well. Turns out the web analysts in Ohio are a pretty sharp bunch so all of the questions I fielded were of the “hardball” type.

One question in particular surprised me, and the answer I gave forced me to elucidate a point I have been pondering for some time but have never voiced in public. The question came from Elizabeth Smalls (@smallsmeasures, go follow her now) who asked, and I paraphrase, “How can we best explain the differences in the numbers we see between systems?” and “Is there any chance the web analytics industry will ever have ‘standards’?”

Long-time readers know I have followed the Web Analytics Associations’s efforts to establish standards closely over the years, helping to create awareness about the work and also pushing the Association to “put teeth” behind their definitions and encourage vendors to either move towards the “standard” definitions or, at worst, elucidate where they are compliant and where they differ from the WAA’s work.

Sadly the WAA’s “standards” never really caught on as a set of baseline definitions against which all systems could be compared to help explain some of the differences in the data. As a result practitioners around the globe still struggle when it comes time to explain these differences, especially when moving from one paid vendor to another.  But none of this matters anymore for one simple reason …

Google Analytics has become the de facto standard for web analytics.

Google has become the standard for web analytics by sheer force of might, persistence, and dedication. By every measure, Google Analytics is the world’s most popular and widely deployed web analytics solution. Hell, in our Analysis Exchange efforts we focus exclusively on the use of Google Analytics because A) we know that 99 times out of 100 we will find it already deployed and B) nearly all of our mentors have had enough exposure to Google Analytics to effectively teach it to our students.

What’s more, as Forrester’s Joe Stanhope opined the recently published Forrester Wave for Web Analytics, web analytics as we knew it doesn’t really exist anymore:

“Few web analytics vendors restrict their remit to pure on-site analytics. Most vendor road maps incorporate emerging media such as social and mobile channels, data agnostic integration and analysis features, usability for a broad array of analytics stakeholders, and scalability to handle the rising influx of data and activity.”

Joe says “few” vendors remained focused on on-site analytics, but it would be more precise to say “one” vendor — Google — has maintained interest in how site operators measure their efforts with any level of exclusivity and sincerity. In fact, I don’t think we need to call the industry “web analytics” anymore … it is probably more accurate to say we have “Google Analytics” and “Everything Else.”

Everything else is enterprise marketing platforms. Everything else is integrated online marketing suites. Everything else … is all of the stuff that has been layered on top of solutions we have historically considered “web analytics” as a response to an event that can only be accurately described as the single most important acquisition in our sector, period.

Google Analytics is the de facto standard for web analytics, and this is great news.

Assuming you take care with your Google Analytics implementation, whenever there is a question about the data you will have a fairly consistent[1] view for comparison. Switching from one vendor to another? Use Google Analytics to help explain the differences between the two systems! Worried that your paid vendor implementation is missing data? Compare it to Google Analytics to ensure that you have complete page coverage! Not sure if a vendor’s recent change in their use of cookies impacted their data accuracy? Yes, you guessed it, compare it to Google Analytics!

With Google Analytics you have a totally free standard against which all other data can be reconciled.

Now keep in mind, I am absolutely not saying that all you need is Google Analytics — nothing could be further from the truth. Despite a nice series of updates and the emergence of a paid solution that may be appropriate for some companies, I agree with Stanhope when he says that “Google Analytics Premium still lags enterprise competitors in several areas such as data integration, administration, and data processing …”

But that’s a debate for the lobby bar, not this blog post.

If you’re looking for a set of rules that can be universally applied when it comes to the most basic and fundamental definitions for the measures, metrics, and dimensions that our industry is built upon, you don’t have to look anymore. Google has solved that problem for the rest of us, and we should thank them. Now, thanks to Google, we can focus on some of the real problems facing our industry … which again, is a debate best left to the lobby bar.

What do you think? Are you running Google Analytics on your site? Do you use it when you see anomalies in data collected through other systems? Have you used it to validate a move from one paid vendor to another? Or do you believe that the WAA standards already provide the solution I am ascribing to Google?

As always I welcome your opinions and feedback.


[1] Yes, when Google changed the definition of a “session” that impacted their consistency, but once they corrected the bug they introduced it seems the number of complaints has gone down significantly. What’s more, the change made sense and in general we should be in favor of “improving on standards whenever possible” don’t you think?

Adobe Analytics

Purchases to Date – Part II [SiteCatalyst]

Last week I described a new way to track how much money visitors had spent on your site prior to their current visit. This week, I am going to expand on this topic and provide some other cool uses of this concept. If you haven’t read my last post, I suggest you do that before reading this one.

Revenue by Product Category

In the last post, you may recall that we were able to quantify how much money the visitor had purchased in the past and break down current reports by those amounts. In the scenario I described previously, we could only see the total revenue amount across all product categories (in the previous scenario the product categories we discussed were Electronics, Clothing and Furniture). However, there is no reason that you cannot create a separate Counter eVar for each product category (or your major product categories if you have too many!). Doing this will allow you to see how much visitors had spent on just Electronics, for example, prior to future Success Events like Cart Adds or Orders. This might be good for companies that have distinct teams focused on each product category. To do this, the code might look like this:

s.events=”purchase”
s.products=”;SKU111;1;300.00;; evar1=Electronics,;SKU222;1;400.00;; evar1=Clothing,;SKU333;1;200.00;;evar1=Furniture”
s.eVar40=+900
s.eVar41=”+300″
s.eVar42=”+400″
s.eVar43=”+200″

By doing this, there would be one Counter eVar which shows that the visitor in our example above had spent $300 (row five) in Electronics prior to his/her second visit which might result in a report like this:

You would then see a report like this for each product category, though I would still recommend one Counter eVar like the one first described, which combines revenue for all product categories combined. Keep in mind that you could also use Product Merchandising to see total previous revenue (eVar40 in our example) by product category, but since you only get two levels of breakdown in SiteCatalyst reports, splitting out each product category into its own Counter eVar provides one more level of breakdown…

Orders to Date

As long so you are going to go through the effort to see how much money the current visitor had spent on your site, why not also track how many Orders they had completed? Doing this is very similar, though it will use up more eVars. Here is how you would do it. First, set a new eVar in the Admin Console and set it to be a Counter eVar with an expiration of “Never” or possibly “1 Year” depending upon how long you want to keep the data. Once this is done, on the purchase thank you page, simply set the Order Counter eVar to “+1,” as you normally would set a Counter eVar like this:

s.events=”purchase”
s.products=”;SKU111;1;300.00,;SKU222;1;400.00,;SKU333;1;200.00″
s.eVar41=”+1″

Kind of anticlimactic huh? By doing this on every purchase thank you page, you can track how many orders each website visitor completed and can then use this in analysis efforts. Next time you want to see how many times people who have added items to the shopping cart today have ordered in the past, simply open this new “Previous Orders” Counter eVar and add the appropriate metric(s):

Here we can see that 21.13% of the Cart Additions that took place today were from visitors who had not ordered on our site in the past (ignoring those pesky cookie deleters!). If we wanted, we could also break this report down by Product to see which Products they had purchased. Also, keep in mind that this example shows Cart Additions, but that we could have just as easily added Orders, Revenue, Internal Searches or any other website metric we wanted to this report to see how many orders had taken place prior to that Success Event. If desired, we could also use SAINT Classifications to group this “Previous Orders” Counter eVar into logical buckets of say “1-2 Orders,” 3-5 Orders,” “5-10 Orders,” etc…

Final Thoughts

So there you have it! Between this post and the last one, hopefully you have some new ideas to try out on your website so you can leverage past purchase behavior when doing your web analyses. If you have any questions/comments, feel free to leave them here. Thanks!

Analytics Strategy

Monish Datta Gives #cbuswaw w/ Eric Peterson & ForeSee Thumbs-Up

We blew past our previous attendance record at the latest Columbus Web Analytics Wednesday, and the speaker did not disappoint! We were fortunate to have Eric Peterson in town and extremely lucky to have Foresee as our sponsor — covering the food and drink for a larger-than-initially-predicted turnout, as well as providing a copy of Larry Freed’s new book to each person who asked a question. All in all, the event got a figurative thumbs-up from many of the attendees, and I caught a literal thumbs up from Monish Datta as well:

WAW Columbus - October 2011

Eric played to a packed house, which he handled with ease:

WAW Columbus - October 2011

The evening’s format was simply a “Q&A with Eric Peterson.” Knowing our audience, I was confident that the questions would be good ones, and they were!

I’ve used Twitter as a crowdsourced note-taking tool in the past at events like 2011 eMetrics San Francisco, and it has worked out well. So, for this event, I made sure that our standard event hashtag — #cbuswaw — was included on notecards scattered around the room (along with the username for our speaker — @erictpeterson — and our sponsor — @foresee). I set up a TweetReach tracker ahead of the event based on the hashtag and then just sat back and let the “note taking” begin!

In the end, we had 179 tweets from 49 different people:

For a small networking event in central Ohio, that seemed like plenty of taking of notes! Several attendees were following the stream of tweets and retweeting as various thoughts caught their eyes (counting myself amongst that group), so it’s a reasonable leap, I think, that looking at the “most retweeted” tweets is a quick-and-dirty way to get a  read on what content was most resonant with the in-person audience.

The most retweeted tweets:

Social media was definitely one hot topic, for which Eric had some thoughts about overall maturity and challenges, but he also referred attendees to his partner, John Lovett’s, book on the subject.

There was also a discussion about “standards” for web analytics. Eric had some new and interesting thoughts on that front…but I found out later that he’d been tossing those around in his head for a while and has a draft blog post written on that subject. So, keep an eye on his blog to see if that gets fleshed out.

I honestly don’t remember if it was the social media question or the standards question that led to a discussion of “measuring engagement,” but John Hondroulis managed to dig up Eric’s post from 2007 on the subject and get that shared out to the crowd.

And, the inevitable privacy topic came up, which garnered a few tweets about the WAA’s Code of Ethics.

All in all, it was a fantastic event!

Analytics Strategy

How Google Analytics In-Page Analytics / Overlay Works

I’m starting to think that page overlays are the new page-level clickstream — they’re what well-meaning-but-inexperienced business users see in their minds’ eyes as a quick and clear path to deep insights when, generally, they are not. I’ve had a couple of clients over the last year ask for overlays (in one case, “provided weekly for all major pages of the microsite”), and the overlays were never an effective mechanism for helping them drive their businesses forward. (One request was for overlays from Sitecatalyst; the other was for overlays from Google Analytics.)

I seldom use overlays for reporting or analysis. The reason isn’t that they don’t have very real usefulness in certain situations, but, rather, because those certain situations are extremely rare in my day-to-day work. As the “page” paradigm — in its basic-HTML simplistic glory — goes the way of daytime soap operas, and as brands’ digital presences increasingly are intertwined combinations of their sites and social media platforms, the number of scenarios where an overlay provides a view of the page that is both reasonably complete and actually useful are few and far between.

That’s a bit broader of a topic than I was aiming to cover with this post, though.

I recently needed to explain to a client why it wasn’t simply a matter of “fixing” the Google Analytics implementation on his site to get the overlays to work properly. I did some digging for documentation that explained the underlying mechanics of GA’s in-page overlays (similar to what Ben Gaines wrote about Sitecatalyst ClickMap a couple of years ago when he was still at Omniture), and…I couldn’t find what I was looking for. This post is trying to be that documentation for the next person who is in the same situation. If you have deeper knowledge of the underlying mechanics of Google Analytics than I have, and I’ve misrepresented something here, please leave a comment to let me know!

Google Analytics <> Sitecatalyst <> ClickTale

There are different ways to capture/present clickmap and heatmap overlays. In order of increasing robustness/usefulness (I’m leaving out a number of vendors because I simply don’t have current knowledge of their specifics):

  • Google Analytics, at its core, uses some basic reverse-engineering of page view data to generate its in-page analytics (overlays). It looks nice in their video…but the video uses a very basic site, which doesn’t reflect the reality of most sites for medium-sized and large companies
  • Adobe Sitecatalyst gets a bit more sophisticated with its approach, which automatically closes some of the gaps in the GA approach while also allowing for working around a chunk of the challenges that are inherent with overlays; see Ben’s post that I referenced earlier if you want to really get into the details there!
  • ClickTale is a solution that was developed from the ground up to provide workable overlays and heatmaps. As such, it takes an even more robust approach — capturing both mouse movements and clicks. The “downside” (in quotes because this is a limitation in theory — not in practice) is that ClickTale does not track all sessions. It samples sessions — still collecting plenty of data to provide you with highly usable data, but business users inevitably get heartburn when they find out that they’re not capturing everything.

Make sense? The point is that there are different ways to skin the overlays cat. This post just covers Google Analytics.

How Google Analytics Figures Out Overlays

For each user session, Google Analytics gets a “hit” for each page viewed during the session, and it records a timestamp for each page view, so it knows the sequence in which pages were viewed in the session. Consider a simple, 3-page site, where the main page (page_A) has links to the other two pages.

 Now, let’s have three visitors come to the site (Visitor 1111, Visitor 2222, and Visitor 3333). All three enter the site on Page_A, but then:

  • Visitor 1111 clicks on the link to Page_B and then exits the site
  • Visitor 2222 clicks on the link to Page_C and then exits the site
  • Visitor 3333 clicks on the link to Page_B and then exits the site

Google Analytics would have captured a series of page views that looked something like this:

Visitor ID Timestamp Page Viewed
Visitor 1111 09:03:16 Page_A
Visitor 1111 09:03:24 Page_B
Visitor 2222 09:04:12 Page_A
Visitor 2222 09:04:53 Page_C
Visitor 3333 09:10:22 Page_A
Visitor 3333 09:10:54 Page_B

With a little sorting and counting and cross-referencing, Google Analytics can figure out that:

  • There were 3 visits to Page_A
  • The “next page” that two of those visitors went to from Page_A was Page_B
  • The “next page” that one of those visitors went to from Page_A was Page_C

That’s how Google Analytics generates the Next Page Path  area of the Navigation Summary report for a page (and, with the same basic technique, this is how the Previous Page Path is generated):

Make sense? Good. So, how does this become in-page analytics? In-page analytics, really, is just a visualization of the Next Page Path data. To do that:

  1. Google Analytics pulls up the current version of the page at the URL being analyzed with in-page analytics
  2. It compiles a list of all of the “next pages” that were visited (with the number of “next page” page views for each one)
  3. It scans the page for the URLs of those “next pages” and then labels each link that references one of those pages with the number of pageviews (and the % of total “next page” page views that the value represents)

Pretty simple, and pretty solid…except when various common situations occur, which we’ll get to next.

Oh, the Many Ways that In-Page Analytics Breaks Down

In-page analytics is problematic when any of the following situations occur on a page:

  • A link has a target URL that is not part of the current site (e.g., a link to the brand’s Facebook page or YouTube channel): Google Analytics doesn’t capture the “next page” viewed, so it can’t deduce how many times the link was clicked (Note: a best practice, obviously, is to have event tracking or social tracking implemented in these situations, so Google Analytics can report on how many times the link was clicked…but this doesn’t work it’s way back into in-page analytics overlays)
  • A link points to a PDF or file download: this is similar to the previous scenario, in that the “next page” doesn’t execute the Google Analytics page tag; again, even if a virtual page view is captured on the click, that is, technically, different from the actual target URL in the <a href=”…e> that points to the file, so Google Analytics doesn’t make the connection needed to render this on the overlay. In other words, the virtual page view will show up on the Navigation Summary in the Next Page Path list, but it won’t show up on the overlay.
  • Multiple links on the page point to the identical next page: because GA uses the URL of the “next pages,” it doesn’t inherently capture which link pointing to the specific next page is the one that was clicked. The standard workaround for this is to force the URLs to be unique by tacking on a junk parameter to the end of the second URL (e.g., have one link point to “Page_B.htm” and the second link point to “Page_B?link=2”). This will make the target URLs unique in GA’s view…but will also make base reporting for Page_B a bit trickier, as there will be two different rows in the Pages report for the same page (if your <title> tags are well-formed, you can work around this by using the Page Titles dimension in the Pages report)
  • Links are embedded in “hidden” content, such as Javascript menu dropdowns: this is simply a limitation of the overlay paradigm, in that it is often impossible to make all of the links on a page visible at once. With in-page analytics, as you mouse over areas that make the links appear, the in-page analytics data will appear as well, but it still requires moving all around the page to reveal all of the links to view all of the “next page” data
  • Links are embedded in Flash: in-page analytics simply can’t effectively add clicks to links that are embedded in Flash objects
  • Links appear to reference the same page: some implementation of DHTML that trigger overlays or other interactive in-page content wind up including something like “<a href=”#”…”, which looks to Google like a link back to the current page. This confuses GA mightily!
  • The link is removed from the page: say you run a promo for a week and then take the hyperlinked image off of the page. When you pull up in-page analytics for that week, GA will know that there were a lot of “next page” views to the target for that promo…but it only has the current page for use in generating an overlay, so it won’t know where to overlay the page views for that promo
  • The links on the page aren’t spaced far enough apart: this is a practical reality, in that I have never seen an overlay where there aren’t some overlay details that obscure the details for other links that are located in close proximity. Obviously, you’re not going to design your site to be overlay-friendly…so you just have to accept this limitation.

The kicker is that these are not obscure, corner-case scenarios. They’re common occurrences, and they lead to most overlays presenting an incomplete picture of activity that occurs on the page.

A Handful of Additional Thoughts

In-Page Analytics are seldom useful. To the best of my knowledge, this is neither an area in which Google is investing to make improvements, nor is it an area that seasoned web analysts are really clamoring for updates.

However, overlays have their place, I think. But, they need to be done right, which is something on which ClickTale is focused (Michele Hinojosa wrote a good overview of the platform last year if you want to read another analyst’s perspective).

Related to overlays, although not strictly overlay-ish, is a feature of Satellite by Search Discovery, whereby you can very easily enable tracking of all clicks on unlinked content (how many times have you been on a site where you think clicking on a product image will take you to the product’s page…and it doesn’t take you anywhere at all!). I think this is some ClickTale-ish like functionality, but that may be something of a stretch. It was a nifty concept, though.

So, that’s it on GA’s In-Page Analytics. Understand what it does and how it does it, and you will be able to identify the (extremely rare) situations when it will be useful.

Adobe Analytics

Purchases to Date – Part I [SiteCatalyst]

Website visits don’t occur in a vacuum. People who are on your site today may or may not have been there in the past and if they have been there, some have purchased items and some have not. But how do you know if the current reports you are looking at in SiteCatalyst reflect those who have purchased in the past or not? How do you look at SiteCatalyst reports by how much they have purchased in the past? Having this context can greatly improve the analysis you are doing so in this post, I will share some techniques which allow you to easily segment your visitors by how much they have spent in the past…

Why Do This?

Before diving into how to do this, let’s explore the rationale. Imagine that you are a retailer selling Electronics, Clothing and Furniture. One question you might ask is “I wonder how much money all of the people who are on my site today have spent in the past?” Wouldn’t it be cool to see that 25% of the people who bought something today had purchased $500 or more in prior visits? Do people who have purchased more than $700 in the past convert at higher rates than those who have only purchased $300? Do people who have bought $400 or more in Electronics tend to only buy and look at Electronics products? As you can see, there are an endless number of analytics questions that can be studied once you know how much money current visitors have previously spent.

Surprisingly, however, there is no easy way to see this in SiteCatalyst. One way to do this is to create Segments. However, since there are so many segments that could be built, this is not always an easy option. To answer the questions above, you’d have to create different segments for each dollar amount and product category (i.e. people who have spent $100, $200, $500, etc…). Plus, you’d have to pull the data using DataWarehouse or ASI. Of course, this becomes much easier in SiteCatalyst v15 (if you are lucky enough to have access to it!), but it still requires a lot of segments to be built. Therefore, I will share a different approach that you can consider to accomplish this using a Counter eVar. As a quick refresher, a Counter eVar is a type of eVar that you increment as needed and retains a numeric value for each website visitor. This counter can be incremented by “1” each time it is set, or it can be incremented by any other number as needed. In past posts, I have described using Counter eVars to track # of Pages Viewed and Ben Gaines described how to use Counter eVars to score visitors. If you want to learn more about Counter eVars, please review this old blog post.

The Solution

With the set-up and refresher out of the way, let’s dig in. As mentioned above, in this scenario, we are a retailer selling three main product categories and want to see how much money each visitor has spent prior to the current visit. To do this, in addition to setting the Products string during the purchase event, we would set a Counter eVar equal to the amount that is being purchased like this:

s.events=”purchase”
s.products=”;SKU111;1;300.00,;SKU222;1;400.00,;SKU333;1;200.00
s.eVar40=”+900″

Notice that we have added up the purchase amount and passed it to a new Counter eVar40. In the above example, if the current visitor hadn’t previously visited the site, the value in his/her Counter eVar after this purchase would be $900. Since Counter eVars don’t have a notion of currency, the value that will be stored in the Counter eVar report in this case would be “900.00” (I would suggest that you round numbers to the nearest dollar since having decimals will make applying SAINT Classifications difficult). Keep in mind that you should set the Counter eVar to be Most Recent (Last) Allocation and set expiration to “Never” (or something like 90 days) in the Admin Console. That is all of that we have to do from an implementation standpoint.

So now let’s see how we use this. If the above visitor comes back to the website next week and adds a few products to the shopping cart and we pause time for a second and were to look at the resulting SiteCatalyst report, we would see something like this:

As shown here, we can now answer the question of how much money visitors had spent in the past at the time they added items to the shopping cart today. In this case, it looks like about half (49%) of people adding items to the cart today had not purchased previously. The visitor mentioned above would fall into row five in this report as part of the 1.38% of people who had purchased $900 in a previous visit. The same principle would apply to Orders and Revenue, so you could see a report like this:

When you extrapolate this principle by thousands of website visitors, you can see some interesting trends about what percent of website visitors transacting today had purchased in the past and how much they had spent. Next we can make this report more readable by applying SAINT Classifications to the Counter eVar to bucket the dollar amounts spent into logical groupings:

Now we have a new report that was previously unavailable! Pretty cool, huh?

In addition, if we wanted to take things to the next level, we could break this report down by Products to see which Products made up the Revenue in past visits:

 

Final Thoughts
So that is one way to see how much visitors on your site have purchased previously so you can add that to your existing web analyses. Next week, I will continue with “Part II” of this topic and go into some additional ways you can apply this concept so stay tuned…Thanks!

Adobe Analytics

Merchandising eVars [SiteCatalyst]

After blogging about Omniture SiteCatalyst for a few years now, one of the topics I have always avoided discussing is Merchandising eVars (not to be confused with the separate Omniture Merchandising product). The reason for this, is that I find them to be very confusing and was sure that no matter how hard I tried to explain them, I would probably mess it up. For years, I have waited for someone to write about them, but seeing as no one has written extensively about them (at least according to a quick Google search!) and having been inspired by some other great blog posts I have read lately in which people have said that it is ok to not have all of the answers, I have decided to face my fears and go ahead and do my best to describe Merchandising eVars. My hope is that this post will serve as a first step in getting the SiteCatalyst community to understand these nuanced eVar and that it might spawn some good discussion and other blog posts by others who have spent a lot more time with them (like Kevin W.) so that one way or another, the topic will be adequately covered.

Why Merchandising eVars?

So why did Omniture make a special type of Merchandising eVar and why are they so complicated? If we go back in time to when I started using SiteCatalyst (version 9.x) and there were no Merchandising eVars, there were a few problems that existed. First was the Category parameter in the Products string. If you have been using SiteCatalyst for a while, someone has probably told you to NEVER use the first parameter (Category) in the Products string. They often don’t tell you why, but the reason is that if you do, the Product you pass will be forever tied to the Category in that string. That means that if you later decide to put the same product in a different product category, SiteCatalyst will ignore it and always use the first one it saw. If each of your products has only one product category and it will be that way forever, you can go ahead and use the Category parameter (or simply classify products using SAINT Classifications). But since most clients like to have products in more than one category, they asked for a way to assign the same product to different merchandising categories, hence, Merchandising eVars!

Let’s look at an example. Say that you have a retail site and that you sell ceiling fans, but those fans can be found by people going through “Lighting” or “Bedroom” product categories. Now let’s say that you would like to know how many Cart Adds or Purchases take place when people found ceiling fans through one of these product categories, but not the other. Sounds simple enough right? But it wasn’t in the past. If you had used the Products string to assign a specific ceiling fan to “Lighting,” it would always be bound to that product category. Instead, you would need a way to dynamically assign the specific product category for each product in each specific instance to get the data you were looking for. By doing this, you could see how often the ceiling fan was purchased via “Lighting” and how often it was purchased via “Bedroom.” Since then, there have been many different uses for Merchandising eVars, but I think it is important to understand the underlying problem that they were created to solve, as I find this helps to understand how they work and why they are different from traditional eVars. So when you think of Merchandising eVars just remember that their purpose is to assign a different eVar value to each product at the time Success Events take place.

Using Merchandising eVars

So now that we know a bit about how Merchandising eVars originated, let’s discuss how they are used. As you can imagine, connecting a different eVar value to each product is not a simple task. That is a lot of information for SiteCatalyst to keep straight! There would have to be some specific ways for you to implement this such that SiteCatalyst knows when you want each product to be tied to each Merchandising eVar value. Fortunately (or unfortunately!), SiteCatalyst has not one, but two methods of binding eVar values to products. One method is called Product Syntax and the other is called Conversion Variable Syntax.

Product Syntax
I find the Product Syntax method to be the most straightforward, and what I recommend most often, so I will start with that one. In this method, you use a special parameter slot within the Products string to declare which Merchandising Category you want to assign to each product. To do this, let’s re-visit the syntax for the Products string:

s.products=”category;product;quantity;price;event_incrementer;
merch_category1|merch_category2

As you can see, towards the end of the Products string, there is a slot reserved for setting Merchandising eVars. In fact, you can set more than one by using a “|” separator. Using this syntax, if a Cart Addition occurs, you can set your Cart Add Success Event and Merchandising eVars as shown in this example:

s.events=”scAdd”
s.products=”;Fan-11980;;;;evar1=Lighting”

Here we can see that we are manually assigning the product category of “Lighting” to the product “Fan-11980” at the time of Cart Addition. However, there are some back-end settings that also need to be made to allow for this to function properly. First, we need to call Omniture Client Care and ask that Merchandising be enabled for the appropriate eVar (eVar1 in this case). Once Merchandising has been enabled, we need to go to the Admin Console and select the Product Syntax option under the new Merchandising setting that will now be visible. When using Product Syntax, the second Merchandising setting (called Merchandising Binding Event) is disabled (but for some reason looks like you can use it!) so my advice is to just ignore that setting altogether. Here is what the settings should look like when you are done:

As with other eVars, you still have to decide what Allocation you’d like (First or Last) and how long the eVar should retain its value before it expires. But beyond that, you are good to go and the hardest part is making sure your developers are keeping track of which product categories should be associated with each product. If you know the value that you want to pass to the eVar for each product on the page (product category in the preceding example), I recommend you use the Product Syntax approach.

Conversion Syntax
The second approach to setting Merchandising eVars is the Conversion Variable Syntax. This approach is a bit more confusing and is normally used when you want to associate a different eVar value to each product, but the value you want to set in that eVar is only known prior to the Success Event taking place, instead of on the same page. The only way I can think of to explain this is through an example. Let’s imagine that your boss wants to know which internal search phrases were used prior to each product being purchased. Now, let’s pretend that a visitor comes to the website and searches on “ceiling fans,” finds Product 123 in the list and adds it to the cart. Next, the visitor searches for “bathroom vanities,” again scans the list, finds Product 789 and adds it to the cart. Then the visitor purchases both items a few pages later. In this example, if we were to use a traditional eVar (with Most Recent allocation), each Cart Addition would be correctly associated with the correct search phrase – “ceiling fans” = product 123 and “bathroom vanities” = product 789. So far so good. But when the visitor purchases both products, guess which internal search phrase would get the credit? If you said “bathroom vanities” you are correct! Since that was the last search phrase SiteCatalyst saw, it would get credit for both products. This is because a traditional eVar cannot associate a different value for each product.

However, by using the Conversion Syntax and Merchandising, in this scenario, each product would be associated with the specific search phrase that was used to find it for both the Cart Add and Purchase Success Events. So how do we configure this? First, we would work with Client Care to declare eVar1 to be a Merchandising eVar. Next, we would decide when we would like to have Omniture bind the internal search phrase to the eVar value. For most clients, the default is to bind at the Product View (prodView) event and the Cart Add (scAdd) event (though you can choose from any Success Events you’d like). By binding to the Product View and Cart Add, you are telling Omniture that if one of those two events happens, you want Omniture to bind the last value passed to the Merchandising eVar (internal search phrase in our example) with the product being viewed or added to cart. This is how these settings would look in the Admin Console:

Well…there you have it. My first attempt at facing my fears and explaining about Merchandising eVars. I have also written a more advanced post on Merchandising you can check out. Please comment here and I will do my best to get any question answered. Thanks!

Analytics Strategy

Moneyball Will Put Web Analytics on the Map

So, my prediction is that the movie Moneyball, set to release this Friday September 23rd, will add a level of awareness to Analytics that skyrockets our little cottage industry straight to household status.

For many of us in the analytics and optimization business, Michael Lewis’ book Moneyball is something of a bible. I know that when I first read it back in 2003, it made me want to become a web analyst. The book chronicles the unorthodox methods of one maverick baseball manager who was forced to break the traditional paradigm of scouting and recruiting big market baseball players to build a winning team that didn’t match his shoestring budget. The manager was Billy Beane, responsible for the 2002 Oakland A’s baseball club, who irrevocably changed the business of baseball using analytics.

Back in 2009, when Steven Soderberg was directing the film, the critics were calling this a niche movie with a purported $60M budget. But since then, with Bennett Miller taking the Director’s chair, this film is set to leap off movie screens across the country. This isn’t merely because they wrangled A-listers like Brad Pitt and Jonah Hill to star in the film, but because this movie has universal appeal. Baseball, business, and Brad Pitt. What brand doesn’t want to imagine themselves as the underdog who bucked the system and came out ahead of the game? Even the biggest brands will see the potential for doing more with less as depicted in the movie. And my guess is that many c-level executives will walk into their offices on Monday and ask who’s running their analytics. Brad Pitt is about to put the sexy into analytics. While, this parallels are somewhat different, I think that just like Pitt’s 1992 movie A River Runs Through It catapulted flyfishing to mainstream status, Moneyball will do the same thing for web analytics. While there may not be a flashmob at the next eMetrics event with newbies clamoring to become Certified Web Analysts, there will certainly be a widespread awakening to what we do.

The thing about Moneyball is that despite the fact that analytics enabled the team to recognize talent and even predict what/who was likely to be successful, it also reveals that running a business purely by the numbers doesn’t guarantee your win. This is akin to the debate ignited by my partner Eric T. Peterson about whether or not your business should be data-driven. While I agree with Eric’s argument on many levels, commentary from the other side of the argument penned by Brent Dykes makes a lot of sense too. I’ll go on record as saying that I do believe that both of these guys are trying to slice it too thin by getting into the semantics of analysis because they’re both right. What we do as analytics professionals requires a balance of data and experience. So the way I see it, both these guys are arguing for similar results. The Oakland A’s got the jump on most major league teams back in their day by using data for competitive advantage. But just like many of the stalwart directors and scouting veterans likely thought, it didn’t get them all the way to the world championship. In analytics too, we need to balance data with business acumen. Tipping the scales all the way toward managing by business experience and intuition won’t net big wins any more than managing purely by the numbers.

What we can take away from analytics and now thanks to the movie Moneyball is that data can gets us a whole lot closer to the answers. While Billy Beane’s character depicts a relentless pursuit of his goal using data, his visibly abrasive personality and callous nature of treating players reveals that balance is required. The fact is that analytics are everywhere in business today. In baseball, Billy Beane still works for the Oakland A’s and my beloved Redsox hired Bill James (another Sabermetrics guru), but many statistical sports pros” have built successful businesses using data and real-time analytics – not just in baseball, but other sports too. A quick look at NBA basketball teams reveals that numerous big leaguers are employing interns, analysts and consultants to study the numbers. And of course, businesses too. For every digital proprietor, business-to-business operation, or consumer facing brand selling today; using data to understand customers and to improve digital marketing has undeniable allure. So, have we finally made it to the mainstream? Well, I think we’re close and that this movie will certainly help.

So the next time you’re explaining to your neighbor – or grandmother – what it is that you do for work … Don’t be surprised when they say “Oh, it’s like that movie Moneyball!” Just smile and say, “Yep, it’s something like that.”

Analytics Strategy

Reflections from the Google Analytics Partner Summit

Having recently become a Google Analytics Certified Partner, we got to participate in our first Partner Summit out in Mountainview, California, last week. It was unfortunate that the conference conflicted with Semphonic’s XChange conference (There really aren’t that many digital analytics conferences, are there? Maybe I should publish a proposed schedule for 2013 for a non-conflicting master schedule?), but I’m looking forward to reading through the reflections from huddlers who were down in San Diego on the blogosphere in the coming weeks!

Onto my shareable takeaways from the Google Analytics summit…

CRAZY Coolness Is on the Way

<sigh> This is the stuff where I can’t provide any real detail. But, essentially, the first two hours of the summit were one live demo after another of very nifty enhancements to the platform, some of which are coming in the next few weeks, and some of which won’t be out until 2012. Some of the enhancements fall in the “well…the Sitecatalyst sales folk won’t be able to use that as a Google Analytics shortcoming when they’re a-bashing it” category, and some fall in the “where on earth did they come up with that — no one else is even talking about doing that” category.

Very cool stuff, and with a continuing emphasis on ease of implementation, ease of management, and a clean and usable UI. Clearly, when v5 rolled out and Google emphasized that the release was more about positioning the under-the-hood mechanics for more, better, and faster improvements in the future, they meant it. Agility and a constant stream of worthwhile enhancements are the order of the day.

I Don’t Know My Googlers

Two presenters — both spoke a couple of times, either formally or when called upon from the stage — really stood out. Maybe I’ve just been living in an oblivious world, but I wasn’t familiar with either one:

  • Phil Mui, Group Product Manager — Phil is apparently a regular favorite at the summit, and he got to run through a lot of the upcoming features; he’s a very engaging speaker, and he’s both excited about the platform while also in tune (for the most part) with how and where the upcoming enhancements will be able to be put to good use by users
  • Sagnik Nandy, Engineering Lead, Google Analytics Backend and Infrastructure — it was a pleasure to listen to Sagnik walk through all manners of how the platform works and what’s coming in the future; the backend is in good hands!

Both of these guys (all of the Googlers, actually) are genuine and excited about the platform. Avinash Kaushik’s passion and thoughtfulness (and healthy impatience with the industry) is alive and well…and entertaining as all get out!

Google Analytics Competitive Advantage

I owe Justin Cutroni for this one, but it was one of the more memorable epiphanies for me. As we chatted about GA relative to the other major web analytics players, he pointed out a fundamental difference (which I’m expanding/elaborating on here):

  • Adobe/Omniture, Webtrends, and IBM (Coremetrics and Unica) are all largely fighting on the same playing field — striving to develop products that have a better feature set at a better price than their competition. This is pretty basic stuff, but it requires pretty careful P&L management — R&D investment that, ultimately, pays a sufficient return through product revenue
  • Google is playing a different game — their products are geared towards driving revenue from their other products (Google Adwords, the Google Display Network, etc.). That actually makes for a very different model for them — much less of a need to manage their R&D investment against direct Google Analytics income (obviously), as well as a totally different marketing and selling model.

There is a certain inherent degree of commoditization of the web analytics space. With a relatively small number of players, R&D teams are focused as much on closing feature gaps that their competitors offer as they are on developing new and differentiating features. In a sense, Google is more focused on “making the web better” — raising the water level in the ocean — while the paid players are geared solely towards making their boats bigger and faster.

I fervently hope that Adobe, Webtrends, and IBM are able to remain relevant over the long term. Competition is good. But, it may very well be a very steep uphill battle for structural reasons.

Silly Me — I thought Tag Management Was a 2-Player Field

Several of the exhibitors at the conference offer some flavor of tag management. The conference was geared towards Google Analytics, so their focus was on GA, but all of them clearly had the “any tag, any Javascript” capability that Ensighten touts (TagMan is the other player I was aware of, but, due to crossed signals, I haven’t yet seen a demo of their product).

The most impressive of these tools that I saw was Satellite from Search Discovery, which Evan LaPointe presented during Wednesday night’s blitz “app integration” session, and which he showed me in more depth on Thursday morning. In his Wednesday night presentation, Evan made a pretty forceful point that, if we’re talking about “tag management,” we’re already admitting defeat. Rather, we should be thinking about data management — the data we need to support analyses — rather than about “the tag.”

Subtle semantic framing? Perhaps. But, it falls along the same lines of the “web analytics tools are fundamentally broken” post I wrote last month that set off a vigorous discussion, and which wound up being timed such that Evan’s post about web analytics douchiness had a nice tie-in.

In short, Analytics Engine is impressive for its rich feature set and polished UI. Equally, if not more, exciting is the mindset behind what the platform is trying to do — get analysts and marketers thinking about the data and information they need rather than the tags that will get it for them.

In Short, Not a Bad Couple of Days!

The nature of any conference is that there will be sessions and conversations that are either not informative or not relevant to the attendee. That’s just the way things go. If I walk away with a small handful of new ideas, a couple of newly established or deepened personal relationships with peers, and validation of some of my own recent thinking, I count the conference a success. The Partner Summit delivered against those criteria — there were a few sessions I could have lived without, at least one session that wildly under-delivered on its potential, and some looseness with the Day 2 schedule that made it difficult to bounce between tracks effectively. But, overall, it was a #winning event.

 

 

Adobe Analytics, Analytics Strategy, Conferences/Community

More seats opening for ACCELERATE 2011!

As I have mentioned a few times before, the initial response to our ACCELERATE event announcement caught us off guard — we honestly didn’t plan to be full after a single day of registrations. Because we hate to disappoint folks we set about figuring out how to increase our room capacity, and thanks to the generosity of our sponsors Tealeaf, Ensighten, and OpinionLab, I’m happy to announce we have succeeded!

Between today and October 1st we will be accepting more registrations for the event on Friday, November 18th in San Francisco. These registrations will still be provisional (e.g., on the “wait list”) but we are committed to having a final list by the first week in October so that folks can make travel plans, etc. If you are interested in joining us, I strongly recommend you go to the ACCELERATE site and register today.

Speaking of the ACCELERATE site, we have added information about many of the fine folks who will be presenting “Ten Tips in Twenty Minutes.” We are extremely honored to have great speakers including Bill Macaitis, VP of Online Marketing at Salesforce.com, Michael Gulmann, VP of Global Site Conversion at Expedia, and a half-dozen other brilliant analysts, practitioners, and vendors representing great companies like Sony Entertainment, AutoDesk, Symantec, and many more.

What’s more, we are honored to have ESPN’s Ben Gaines, formerly of Omniture/Adobe fame and the creator of the @OmnitureCare twitter account. Ben will be sharing tips on managing expectations in vendor relationships and I have to say we’re pretty excited to be hosting Ben’s first “non-vendor” appearance in the web analytics world.

We have also put up a registration for the big Web Analytics Wednesday event we will be holding on Thursday, November 17th, generously sponsored by Causata, Coremetrics/IBM, iJento, and ObservePoint. The location is still TBD but is looking like Roe in downtown San Francisco.

So, if you’re interested in joining us at ACCELERATE, your action items today are:

  1. Register on the expanded wait list at the ACCELERATE web site
  2. Register for the Web Analytics Wednesday event
  3. Tweet something like “I want to attend #ACCELERATE 2011! http://j.mp/accelerate2011 #measure”

(Okay, the last action item is more of a wish-list thing for us … 😉

Conferences/Community

Big San Francisco Web Analytics Wednesday event!

We just posted a Web Analytics Wednesday event in San Francisco in November that promises to be the event of the year in the Bay Area. Thanks to the generous sponsorship of Causata, Coremetrics/IBM, iJento, and ObservePoint we will be able to host several hundred folks — which is great news because our ACCELERATE 2011 event is the next day.

We will post location details soon but go to  the Web Analytics Wednesday site and sign up today if you want to be sure to be able to join us.

Sign up for Web Analytics Wednesday on Thursday, November 17th in San Francisco now!

Also, if you have a minute, go have a look at our sponsor’s web sites. With the exception of Coremetrics we believe each of our sponsors are companies you may not know much about but we think are exciting and have something unique to offer the web analytics community.

On behalf of Adam and John, we all hope to see you in San Francisco this November!

Analytics Strategy, Social Media

The Social Technology Spectrum

Social media technologies are massively confusing today. Not because they aren’t powerful or capable of substantially benefitting your organization, but because there are so many to choose from…

During my research while writing my book, Social Media Metrics Secrets (Wiley, 2011) and through countless interviews with social media practitioners and leading vendors in the industry, I developed a categorization schema for understanding social media technologies. I call this the Social Media Technology Spectrum. Across this spectrum, there are five primary functions that businesses can accomplish with social media technologies:

Discover > Analyze > Engage > Facilitate > Manage

While, I go into great detail about each category in the book, I’ll offer an overview here:

The Discovery Tools (Social Search) Discovery tools are social media solutions that effectively act as search engines for social media channels and platforms. Typically, Social Search technologies are freely available, but they don’t allow you to save search queries, download data or export results. Example Discover vendors include: SocialMentionIceRocketBacktweets, Topsy, and hundreds more.

The Analysis Technologies (Social Analytics) These tools are most commonly associated with listening platforms, but in my view, Social Analytics vendor requirements include: filters, segments, visualizations and ultimately analysis. Example Analyze vendors include: Alterian SM2, Omniture SocialAnalytics, Radian6, Sysomos, and many more.

The Engagement Platforms (Engagement/Workflow) Vendors in this category extend their Social Analytics capabilities to include workflow delegation and engagement capabilities from directly within the interface, it places more controls at the fingertips of your internal business users. Example Engage vendors include: Crimson Hexagon, Hootsuite, Objective Marketer, Collective Intellect, and many more.

The Hosting and Facilitation Tools (Social Platforms) If you need to offer your community a social media destination like a user group, a forum, or a designated social media website. That’s where the Social Facilitation technologies provide a platform that can facilitate the conversation, the dialogue and the learning experience. Example Facilitate vendors include: Mzinga, Pluck, Ning, Lithium, Jive, Telligent and many more.

The Management Solutions (Social Management) This group of technology offerings includes social customer relationship management tools, internal collaboration solutions, and social media aggregation services that enable businesses to manage their social media efforts in an orchestrated way. Example Manage vendors include: BatchBook, Flowtown, Salesforce Chatter, Yammer and many more.

As you can see, each category has associated vendors. While there is certainly some cross-over here, there is also a lot more depth to each of the categories. For each category, you can delve deeper by specific social media channel (i.e., there’s a whole cast of Social Analytics tools specifically for Twitter). Yet, in a technology environment that is so cluttered with options and new entrants, I feel that some categorization is merited.

But what do you think? … Am I on the right track here? Do you use technologies from multiple categories? …What did I miss?

Analysis, Reporting, Social Media

"Demystifying" the Formula for Social Media ROI (there isn't one)

I raved about John Lovett’s new book, Social Media Metrics Secrets in an earlier post, and, while I make my way through Marshall Sponder’s Social Media Analytics book that arrived on bookshelves at almost exactly the same time, I’ve also been working on putting some of Lovett’s ideas into action.

One of the more directly usable sections of the book is in Chapter 5, where Lovett lays out pseudo formulas for KPIs for various possible (probable) social media business objectives. This post started out to be about my experiences drilling down into some of those formulas…but then the content took a turn, and one of Lovett’s partners at Analytics Demystified wrote a provocative blog post…so I’ll save the formula exploration for a subsequent post.

Instead…Social Media ROI

Lovett explicitly notes in his book that there is no secret formula for social media ROI. In my mind, there never will be — just as there will never be unicorns, world peace, or delicious chocolate ice cream that is as healthy as a sprig of raw broccoli, no matter how much little girls and boys, rationale adults, or my waistline wish for them.

Yes, the breadth of social media data available is getting better by the day, but, at best, it’s barely keeping pace with the constant changes in consumer behavior and social media platforms. It’s not really gaining ground.

What Lovett proposes, instead of a universally standard social media ROI calculation, is that marketers be very clear as to what their business objectives are – a level down from “increase revenue,” “lower costs,”and “increase customer satisfaction” – and then work to measure against those business objectives.

The way I’ve described this basic approach over the past few years is using the phrase “logical model,” – as in, “You need to build a logical link from the activity you’re doing all the way to ultimate business benefit, even if you’re not able to track those links all the way along that chain. Then…measure progress on the activity.”

Unfortunately, “logical model” is a tricky term, as it already has a very specific meaning in the world of database design. But, if you squint and tilt you’re head just a bit, that’s okay. Just as a database logical model is a representation of how the data is linked and interrelated from a business perspective (as opposed to the “physical model,” which is how the data actually gets structured under the hood), building a logical model of how you expect your brand’s digital/social activities to ladder up to meaningful business outcomes is a perfectly valid  way to set up effective performance measurement in a messy, messy digital marketing world.

No Wonder These Guys Work Together

Right along the lines of Lovett’s approach comes one of the other partners at Analytics Demystified with, in my mind, highly complementary thinking. Eric Peterson’s post about The Myth of the “Data-Driven Business” postulates that there are pitfalls a-looming if the digital analytics industry continues to espouse “being totally data-driven” as the penultimate goal. He notes:

…I simply have not seen nearly enough evidence that eschewing the type of business acumen, experience, and awareness that is the very heart-and-soul of every successful business in favor of a “by the numbers” approach creates the type of result that the “data-driven” school seems to be evangelizing for.

What I do see in our best clients and those rare, transcendent organizations that truly understand the relationship between people, process, and technology — and are able to leverage that knowledge to inform their overarching business strategy — is a very healthy blend of data and business knowledge, each applied judiciously based on the challenge at hand. Smart business leaders leveraging insights and recommendations made by a trusted analytics organization — not automatons pulling levers based on a hit count, p-value, or conversion rate.

I agree 100% with his post, and he effectively counters the dissenting commenters (partial dissent, generally – no one has chimed in yet fully disagreeing with him). Peterson himself questions whether he is simply making a mountain out of a semantic molehill. He’s not. We’ve painted ourselves into corners semantically before (“web analyst” is too confining a label, anyone…?). The sooner we try to get out of this one, the better — it’s over-promising / over-selling / over-simplifying the realities of what data can do and what it can’t.

Which Gets Back to “Is It Easy?”

Both Lovett’s and Peterson’s ideas ultimately go back to the need for effective analysts to have a healthy blend of data-crunching skills and business acumen. And…storytelling! Let’s not forget that! It means we will have to be communicators and educators — figuring out the sound bites that get at the larger truths about the most effective ways to approach digital and social media measurement and analysis. Here’s my quick list of regularly (in the past…or going forward!) phrases:

  • There is no silver bullet for calculating social media ROI — the increasing fragmentation of the consumer experience and the increasing proliferation of communication channels makes it so
  • We’re talking about measuring people and their behavior and attitudes — not a manufacturing process; people are much, much messier than widgets on a production line in a controlled environment
  • While it’s certainly advisable to use data in business, it’s more about using that data to be “data-informed” rather than aiming to be “data-driven” — experience and smart thinking count!
  • Rather than looking to link each marketing activity all the way to the bottom line, focus on working through a logical model that fits each activity into the larger business context, and then find the measurement and analysis points that balance “nearness to the activity” with “nearness to the ultimate business outcome.”
  • Measurement and analytics really is a mix of art and science, and whether more “art” is required or more “science” is required varies based on the specific analytics problem you’re trying to solve

There’s my list — cobbled from my own experience and from the words of others!

Analytics Strategy, General

The Myth of the "Data-Driven" Business

You may have noticed I have been pretty quiet in my blog lately aside from sharing news about our ACCELERATE event in San Francisco in November. It’s partially because honestly I’ve been swamped with new clients, existing work, and the never-ending effort to be a good husband, dad, and friend in the midst of Demystifying web analytics …

But being busy is no excuse to stop sharing ideas and encouraging conversation so let’s dive into something that has increasingly become a pet-peeve of mine: the notion leveraging web analytics to create a “data-driven” business.

I’m sure I have used this phrase in the past in an effort to describe the transformation that companies need to go through in the digital world, relying less on “gut feel” and more on cold, hard data to guide business decision making. Hell, a lot of smart of people have, including Omniture’s Brent Dykes and Google Analytics Evangelist Avinash Kaushik who has gone so far as to describe creating a data-driven culture as the “holiest of holy grails.”

Becoming “data driven” is the way to silence the HIPPO and to more firmly establish the value of our collective investments in digital measurement, analysis, and optimization technology. It sounds great, except for one thing:

A “data-driven business” would be doomed to fail.

I think that perhaps what people mean when they talk about being “data-driven” is the need for a heightened awareness of the numerous source of data and information we have available in the digital world, enough so that we are able to take advantage of these sources to create insights and make recommendations. On this point I agree — better use of available data in the decision making process is an awesome thing indeed.

My concern arises from the idea that any business of even moderate size and complexity can be truly “driven” by data. I think the right word is “informed” and what we are collectively trying to create is “increasingly data-informed and data-aware businesses and business people” who integrate the wide array of knowledge we can generate about digital consumers into the traditional decisioning process. The end-goal of this integration is more agile, responsive, and intelligent businesses that are better able to compete in a rapidly changing business environment.

Perhaps this is mere semantics — you say “potato” I say “tuberous rhizome”  — but given the sheer number of consultants, vendors, and practitioners talking about creating, powering, and working in the mythical “data-driven business” I have started to worry that we’re about to shoot ourselves in the collective foot. We (meaning the web analytics industry as a whole) have done this before, first by claiming that web analytics was easy, then by insisting that cookies were harmless … and personally I’d prefer we avoid yet another self-imposed crisis of credibility if possible.

And while this may be semantics, I do disagree with Brent Dykes assertion that in the absence of carrot-and-stick accountability that web analytics breaks down and fails to create any benefit within the business, although I do understand fully where Mr. Dykes is coming from. I simply have not seen nearly enough evidence that eschewing the type of business acumen, experience, and awareness that is the very heart-and-soul of every successful business in favor of a “by the numbers” approach creates the type of result that the “data-driven” school seems to be evangelizing for.

What I do see in our best clients and those rare, transcendent organizations that truly understand the relationship between people, process, and technology — and are able to leverage that knowledge to inform their overarching business strategy — is a very healthy blend of data and business knowledge, each applied judiciously based on the challenge at hand. Smart business leaders leveraging insights and recommendations made by a trusted analytics organization — not automatons pulling levers based on a hit count, p-value, or conversion rate.

Kishore Swaminathan, Accenture’s chief scientist, in his discussion on “What the C-suite should know about analytics” outlines how an over-dependence on data can lead to “analysis-paralysis”, stating:

“Data is a double-edged sword. When properly used, it can lead to sound and well-informed decisions. When improperly used, the same data can lead not only to poor decisions but to poor decisions made with high confidence that, in turn, could lead to actions that could be erroneous and expensive.”

Success with web analytics and optimization requires a balance, and business leaders who will be successful analytical competitors in the future will need to develop a top-down strategy to govern how their businesses will leverage both digitally-generated insights and the collective know-how of their organizations. Conversely, being “driven” implies imbalance and over-correction — going out of your way to devalue experience, ignore process, and eschew established governance in favor of a new, entirely metrics-powered approach towards decision making.

You can do this, but to Swaminathan’s point, what if the numbers you’re using are wrong?

I think that creating a “data informed” business is a huge victory and for most companies a major step in the right direction. What’s more, working to create a “data informed” business shows respect for the hard work, commitment, and passion your employees have for their jobs and your company and products.

Rather than walk in and “embarrass the boss” with your profound and amazing knowledge of customer interactions, you can actively work with your management team by providing insights and recommendations that reflect your knowledge of how the entire business works, not just your amazing talent as web analytics implementer (or analyst, whatever …)

But I digress.

I’m interested in your collective thoughts here people. Am I over-reaching after a blogging hiatus and unnecessarily sniping in hopes of an early Fall dust-up in Google+? Or have you had the same thoughts and/or concerns, that by insisting that everyone needs to do exactly what the data tells them that we risk alienating (again) the very consumers of our efforts? Do you work at a truly “data driven” business and do what the numbers tell you each and every time? Or are you working to create a practice where otherwise smart, hard-working, and passionate marketers, merchandisers, and business leaders can benefit from the type of information and insights you are uniquely able to provide as a digital measurement, analysis, and optimization specialist?

While you consider your response I’ll leave you with a story that has shaped some of my thinking about web analytics over my career. Years ago my good friend Shari Cleary brought me into CBS News in New York to train her editorial team on Hitbox (yeah, Hitbox, I told you it was years ago!) Most of my clients at the time were “new school” but not these guys — they were hardcore news editors from the TV side of the business who had been tasked with making digital news work.

I talked and talked and talked about how powerful Hitbox was and how real-time analytics was going to power the content they put out there in the world. The editors were polite and showed real interest in the training until at one point the oldest and most grizzled of the group stopped me.

“Son, we’re not going to let the data make the decisions for us regarding editorial content,” he said with all sincerity. I was, of course, shocked to hear this — I mean, hell, that is what Hitbox was for! Figuring out which stories generated page views and which needed to be rolled off the page into obscurity.

“Umm, why is that?” I asked, figuring he’d lay into me about the inaccuracy of the system or how painful it was to use Hitbox …

“Because if we let the data drive editorial, all you will read about at CBS News is Paris Hilton’s breasts and Lindsay Lohan’s drinking problem.”

Needless to say, I stopped talking about real-time, data-driven changes to editorial content.

As always I welcome your comments, criticism, and feedback.

Reporting, Social Media

Have You Picked Up a Copy of "Social Media Metrics Secrets" Yet?

John Lovett’s Social Media Metrics Secrets hit the bookshelves (Kindle-shelves) earlier this month, and it’s a must-read for anyone who is grappling with the world of social media measurement. It’s a hefty tome as business books go, in that Lovett comes at each of the different topics he covers from multiple angles, including excerpting blog posts written by others and recapping conversations and interviews he conducted with a range of experts.

As such, it’s simply not practical to provide an effective recap of the entire book. Rather, I’ll give my take on the general topics the book tackles, and then likely have some subsequent posts diving in deeper as I try to put specific sections into action.

Part I

The first three chapters of the book are foundational material, in that they lay out a lot of the “why you should care about social media,” as well as set expectations for what isn’t possible with social media data (calculating a hard ROI for every activity) as well as what is possible (moving beyond “counting metrics” to “outcome metrics” to enable meaningful and actionable data usage). Early on, Lovett notes:

Analytics solutions and social media monitoring tools are often sold with the promise that “actionable information is just a click away,” a promise that an increasing number of companies have now realized is not usually the case.

That encapsulates, by extension, much of the theme of the first part of the book — that it requires that a range of emerging tools, skills, processes, and organizational structures to come together to make social media investments truly data-driven activities. In addition to the social analytics platforms that Lovett discusses in greater detail later in the book, he makes a case for data visualization as a key way to make reams of social media data comprehensible, and he paints a picture of a “social media virtual network operations center” — a social media command center that harnesses the right streams of near real-time social media data, presents that data in a way that is meaningful, and has the right people in place with effective processes for putting that information to use.

Part II

In Part II of the book, Lovett starts with some basics that will be very familiar to anyone who operates in the world of performance measurement — aligning key metrics to business objectives, using the SMART (Specific, Measurable, Attainable, Relevant, Times) methodology (although Lovett extends this to be “SMARTER” by adding “Evaluate” and “Reevaluate) for establishing meaningful goals and objectives, understanding the difference between accuracy and precision, and so on. This material is presented with a very specific eye towards social media, and then extended to provide a list common/likely business objectives for social media, which each objective drilled into to identify meaningful measures.

These objectives build directly on the work that Lovett did with Jeremiah Owyang of Altimeter Group in the spring of 2010 when they published their Social Marketing Analytics: A New Framework for Measuring Results in Social Media paper. In the book, Lovett substantially extends his thinking on that framework — broadening from four common social media objectives to six, laying out the “outcome measures” that apply for each objective, and then providing pseudo-formulas for getting to those measures (pseudo-formulas only because Lovett emphasizes the need for social media strategies to not be premised on a single channel such as Facebook or Twitter, and he also didn’t want the book to be wholly outdated by the time it was published — the formulas are explicitly not channel-specific, but anyone who is familiar with a given channel will be well-armed with the tools to develop specific formulas that ladder up to appropriate outcome measures). In short, Chapter 5 is one area that warrants a highlighter, a notepad, and multiple reads.

Part III

Part III of the book really covers three very different topics:

  • Actually demonstrating meaningful results — looking at how to get from the ask of “what’s the hard ROI?” to an answer that is satisfactory and useful, if not a “simple formula” that the requestor wishes for; Lovett devotes some time to explaining the now-generally-accepted realization that the classic marketing funnel no longer applies, and then extends that thinking to demonstrate what will/will not work when it comes to calculating social media ROI
  • Social analytics tools — while Lovett makes the point repeatedly that there are hundreds of tools out there, which can be overwhelming, he nonetheless managed to narrow down a list of seven leading platforms (Alterian SM2, Converseon, Cymfony, Lithium, Radian6, Sysomos, and Trendrr) and conducted an extensive evaluation of them. He includes how that evaluation was organized and the results of the analysis in Chapter 8. While the information is sufficiently detailed that a company could simply take his list and choose a platform, the evaluation is set up as an illustration of what should go into a selection process, so it’s a boon to anyone who has been handed the task of “picking the best tool (for our unique situation).”
  • Consumer privacy — this is a very hot topic, and it’s a messy area, so Lovett tries to lay out the different aspects of the situation and what needs to happen to get to some reasonably workable resolution over the next few years. It’s a portion of the book that I’ve already referenced and quoted internally, as it is very easy for marketers and vendors to get caught up in the cool ways they can make content more relevant…without thinking through whether consumers would be okay with those uses of the data

After reading the book once, I’ve already found myself flipping back to certain sections to the point that I’ve got Post Its coming out of it to mark specific pages. Overall, the book is sufficiently modular that individual chapters (and even portions of chapters) stand alone.

Buy it. Buy it now!

 

Adobe Analytics, Conferences/Community, Social Media

Are you a Super Accelerator?

When John, Adam, and I announced the ACCELERATE conference last week we really didn’t expect the response we got, much less that the seats we had planned for would fill in just over a day. Once we got over the initial shock we set about trying to figure out how to accommodate more of the over 300 people who have already registered for the event … and we’re getting closer every day to solving that problem.

We are continuing to take provisional registrations and being on this list is the most sure way to be able to join us in November. If you’re interested, please sign up for the ACCELERATE 2011 wait list.

In the interim we wanted to call your collective attention to our “Super Accelerator” session at the end of the day. Unlike our main speaking slots where brilliant practitioners from companies including Sony, Nike, Expedia, Autodesk, Symantec, Salesforce.com and more will be sharing “Ten Tips in Twenty Minutes”, the Super Accelerator is designed to allow up-and-comers in our community to share a single idea in five minutes or less.

Five minutes! How easy is that?

Just think about the amazing things you could share with ACCELERATE attendees in five minutes? Off the top of my head:

  • The Number One Reason You Should Join the Web Analytics Association
  • The Best Way to Get Your Manager to Think About Web Analytics Data
  • How to Make I.T. Your Friend (and How That Will Help You as an Analyst)
  • How to Take Advantage of Web Analytics Wednesday for Social Networking
  • The Most Important Hashtags Analysts Should Follow in Twitter
  • Why Strategy is Important to your Company’s Investment in Web Analytics

That list goes on and on and on, and I’m sure the best ideas are those that I’m not even thinking of!

We already have five people signed up for the dozen slots we have but we are looking for seven more folks who meet the following criteria:

  • Really want to attend ACCELERATE 2011 (since if you’re presenting, you have to be there)
  • Are willing to commit to creating and presenting a three-slide, five minute talk
  • Have a true passion for digital measurement, analysis, and optimization
  • Love to present, or want to learn to love presenting
  • Love awesome technology …

If the last criteria seems out-of-place, you need to know that the audience will be providing real-time feedback on each Super Accelerator session (thanks to our friends at OpinionLab) and the presenter who earns the best overall score will get a $500 gift card from Best Buy!

How cool is that? I know!

If you’re interested in joining us at ACCELERATE 2011 and being part of the Super Accelerator session I would encourage you to do the following RIGHT AWAY since we expect this session to fill up fast:

  1. Go to the ACCELERATE 2011 web site and REGISTER (you’ll be put on the wait list)
  2. Go to Twitter and tweet “I want to present at #ACCELERATE 2011 as a Super Accelerator! http://j.mp/accelerate2011 #measure”

We are watching the #ACCELERATE tag and will get back to you ASAP. These slots are filled on a first-come basis so DON’T DELAY and sign up today!

 

 

Analytics Strategy

Privacy Whitewashing, History Sniffing, and Zombie Cookies, Oh My…

This content originally posted on the ClickZ Marketing News & Expert Advice website with thoughtful comments and numerous reactions on August 11, 2011.

There’s a great deal of fear, uncertainty, and doubt (FUD) in the hearts and minds of consumers regarding their privacy online. While not totally unmerited, this FUD is fueled by mainstream media sources like The Wall Street Journal and USA Today, that typically paint the issues with a stark black and white perspective. Unfortunately, this perspective corrals all advertisers, website operators, and would-be digital trackers into a single category of shameful voyeurs.

While some tracking practices may indeed be dubious, other allegations are accused of slander. Both scenarios are reason enough to give conscientious consumers pause, thereby placing your online business and the way you track customers in jeopardy. The root of the problem is a fundamental communication breakdown.

What’s Really Going on Behind the Privacy Curtain?
The majority of first-party digital measurement (“first-party” data is obtained by the entity that owns and controls the domain) is designed to improve the user experience online by making processes easier, enabling faster access to relevant goods and services, as well as offering time-saving conveniences for everyday users. These practices have been going on since the dawn of consumerism, and for the most part are tolerated and even appreciated by consumers as long as they adhere to some semblance of consumers’ rights. However, consumers must retain the right to shop, browse, and otherwise interact online in an anonymous manner if they choose to do so. Thus, the opt-out policy. But technologies today have inadvertently enabled ways to circumvent the opt-out by regenerating cookies (dubbed “zombie cookies”) or embedding locally stored objects into users’ machines. These practices are wrong and deftly explained and criticized in Eric T. Peterson’s whitepaper, “Flash LSO’s: Is Your Privacy at Risk?” (registration required).

The flip-side to first-party tracking is third-party tracking, (“third-party” data is obtained from the first party and typically not reasonably known to the end user). This data is often employed by ad-serving technologies as a method for targeting consumers. The primary objection to third-party data is that it can be used to track visitors across multiple domains (“history sniffing” or “daisy-chaining”), thereby creating a history of multi-site browsing behavior that reveals aggregate details on consumer actions unbeknownst to the user.

Most third-party data sources still don’t know names, nor do they profit from selling any personally identifiable information. Instead, anonymous user data is brokered to a slew of third-party advertisers, ad exchanges, ad networks, ad platforms, data aggregators/exchanges, and market research companies who work to serve up relevant content based on the websites users visited. I hate to break it to folks, but that’s how most content websites work. Visitors get free content, hosts deliver ads. It’s a trade-off that most of us are willing to accept. It’s also this trade-off that’s sucking any remnants of serendipity out of the Internet, because things just don’t happen by coincidence today; they happen by marketing.

If They Want Out, Show Them the Door!
The fact is that if consumers don’t want to be tracked, then you must offer them a simple and permanent way out for the wary. Of course, browsers can do this today and consumers can take proactive steps to delete cookies, but it’s still the responsibility of the business to offer choice. Your primary responsibility as a vendor or business is to educate your users through effective communication. This is where most of the confusion festers because vendors don’t provide easy-to-understand guidelines about how their technologies are designed to be used; and businesses often don’t educate their customers about how they treat personal data. As a result, technologies are used inappropriately and consumers feel violated by targeted content and there’s typically a whole lot of fingerpointing going on to pass the blame.

If you’re a business, it’s your responsibility to understand how the technologies you use for digital tracking work, but also to give consumers a choice regarding their ability to remain anonymous and to opt out of all types of tracking. For first-party data collectors, this should be a relatively straightforward exercise; don’t retain customer information if they don’t want you to. If you need more guidance on the right thing to do as a practitioner or data collector, visit the Web Analytics Association’s (WAA) Code of Ethics that outlines the core tenets of ethical first-party, data-handing practices.

For third-party data collection, organizations like the Network Advertising Initiative (NAI) or the Digital Advertising Alliance (DAA) offer third-party opt-out choices for consumers. Consider joining one of these coalitions to join the ranks of the self-regulated. Alternatively, you can brush up on third-party data collection guidelines issued by organizations like TRUSTe, who act in the best interests of consumers by offering guidance on what to do and what not to do regarding digital data collection.

Create an Action Plan for Maintaining White-Hat Digital Tracking Practices
Finally, the best thing that you can do as a vendor, a marketer, or a business is to operate above the fray of privacy pundits by following a few key principles. Take these steps to use digital tracking in the way in which it was designed and to deliver value for your customers and your business:

1. Understand the technologies. While this sounds relatively basic, you must know what the technologies you build or deploy are capable of doing. While getting inside the minds of the devious shouldn’t consume all your time, vendors should issue guidance for utilization as well as educate constituents about how technologies function.

2. Keep PII safe, secure, and private. It should go without saying that keeping customer data safe and private is a top priority, but go beyond offering lip service and spell it out for consumers. Demonstrate how you protect and secure data by communicating to your audience about the measures you take to do so and instill confidence by provisioning multiple safeguards.

3. Divulge data usage practices. If your business is collecting and utilizing first- or third-party data, make it known by divulging your practices in clear and readable language. This requires keeping the legalese to a minimum and offering consumer-friendly policies and explanations for what you’re trying to accomplish. Transparency is the best practice here, so explain what you’re doing and how visitors benefit.

4. Empower consumers to opt out. This one bears repeating…give consumers a way out. And for crying out loud, don’t opt them back in if they don’t request it. This is potentially the biggest threat to online privacy today and as more and more organizations abide by consumer preferences, the ones who don’t will be outed and ultimately tarnish their reputations.

5. Spread the word. The Internet offers many incredible opportunities for networking, commerce, education, and entertainment, but collectively we must act as stewards of consumer data. Perhaps I’m naïve, but I believe that most data collectors are ethical and simply need to do a better job of describing what they’re up to and where the value exchange exists for consumers.

I personally applaud researchers like Ashkan Solanti and Jonathan Mayer for the work they do and for keeping vendors honest about the realities of their digital tracking applications. We need more education and we desperately need to voice the digital measurement side of the argument to crystallize the validity of what we do as analytics professionals.

The online privacy discussion won’t dissipate anytime soon, so the best we can do is communicate effectively, demonstrate value, and offer choice. Do you agree?

Adobe Analytics, Analytics Strategy, Conferences/Community

ACCELERATE 2011 is SOLD OUT

Yesterday we announced that Analytics Demystified was bringing an entirely new type of event to San Francisco in November: ACCELERATE!

Today I am chagrined to announce that ACCELERATE 2011 in San Francisco is SOLD OUT!

Suffice to say, we didn’t expect to sell out overnight, nor did we expect to have so many people traveling to the event from around the globe. We have registrations from as far away as London, Spain, Shangahi, and India; we have registrations from New York, Boston, Seattle, Portland, Phoenix, Boulder, and more!

We are still accepting provisional (“wait listed”) registrations but will likely stop doing that by the end of the week. If you want to join us I strongly recommend registering for the ACCELERATE 2011 wait list IMMEDIATELY.

Also, if you’re already on the list, you will help ensure your seat at the table by joining our “Super Accelerator” session at the end of the day. More details are available at the ACCELERATE mini-site under the “LEARN MORE” link.

As our clients, prospects, and friends complete their registrations we will develop a better sense of exactly how many we can accommodate. At that point we will email registrants directly and provide confirmation.

On behalf of John, Adam, our sponsors at Tealeaf, OpinionLab, and Ensighten, and especially myself we are grateful for the community’s response to ACCELERATE and will do everything possible to get as many folks to the table as we can.

 

Adobe Analytics

Thoughts On Our 1st G+ Foray

This week the Demystified Partners forfeited our weekly meeting to hang out with our fellow #measure peeps on Google+. We felt that the thread about web analytics technologies and where innovation would surface in our industry needed a deeper discussion. If you haven’t seen the original thread yet, make sure you go check it out. Also, if you need a G+ invite, let us know we’ve got plenty to share. But our exercise was as much about continuing the conversation as it was testing out a new social medium.

Before going live on G+, we practiced for about a half hour, where all of our browsers crashed and we experienced various video connection in’s and out’s as we tinkered with and tuned our machines. By showtime, we had a few stalwart veterans join, including Tim Wilson who was dialed in on a 4G connection while driving home with his wife from camping. As our discussion grew, We added up to nine people, which didn’t quite push the limits of G+ as we had hoped, but it was an all-star #measure cast including: @erictpeterson @adamgreco @mymo @Exxx @tgwilson @joestanhope @OMlee @keithburris and yours truly.

Our conversation began with quips from each of the participants about how we’re still grappling with digital measurement technologies. Despite most of us being in this web analytics industry for years, and in some cases decades. Time passed quickly as we debated from all sides of the vendor/consultant/practitioner perspective. After a brief privacy sidebar, we asked each other where innovation would emerge from in analytics and really why we were pursuing these digital data anyway? I think we edged the needle just a little bit by agreeing that what we do matters because we’re educating our employers and clients on the power of data; and just possibly making the Internet a slightly better place. Ok, when I chatted this in the G+ hangout mid discussion, I warned everyone not to throw up on their keyboards, so I’ll do the same for you. But as cheesy as that sounds, our quorum agreed that wasn’t such a bad goal. What do you think?

So, the technology of G+ did prove that it was up to the task of handling this type of group discussion. People could talk and share their ideas on video or contribute to the conversation using the chat functionality. But we’re curious to know if you’re interested in joining us for a future G+ hangout?

If you are and willing to hang out with us to discuss the hottest topics is digital analytics, let us know because we’ll plan another one soon. Heck, we’ll even make this a regular event if you’re interested. What do you all say?

Analysis, Analytics Strategy, Social Media

What I Learned About #measure & Google+ from a Single Blog Post

Quite unintentionally, I stirred up a lengthy discussion last week with a blog post where I claimed that web analytics platforms were fundamentally broken. In hindsight, the title of the post was a bit flame-y (not by design — I dashed off a new title at the last minute after splitting up what was one really long post into two posts; I’m stashing the second post away for a rainy day at this point).

To give credit where credit is due, the discussion really took off when Eric Peterson posted an excerpt and a link in Google+ and solicited thoughts from the Google+/#measure community. That turned into the longest thread I’ve participated in to date on Google+, and subsequently led to a Google+ hangout that Eric set up and then moderated yesterday.

This post is an attempt to summarize the highlights of what I saw/heard/learned over the past week.

What I Learned about the #measure Community

Overall, the discussion brought back memories of some of the threads that would occasionally get started on the webanalytics Yahoo! group back in the day. That’s something we’ve lost a bit with Twitter…but more on that later.

What I took away about the group of people who make up the community was pretty gratifying:

  • A pretty united “we” — everyone who participated in the discussions was contributing with the goal of trying to move the discussion forward; as a community, everyone agrees that we’re at some sort of juncture where “web analytics” is an overly limiting label, where the evolution of consumer behavior (read: social media and mobile) and consumer attitudes (read: privacy) are impacting the way we will do our job in the future, and where the world of business is desperately trying to be more data-driven…and floundering more often than succeeding. There are a lot of sharp minds who are perfectly happy to share every smart thought they’ve got on the subject if it helps our industry out — the ol’ “a rising tide lifts all boats” scenario. That’s a fun community with whom to engage.
  • Strong opinions but small egos — throughout the discussion that occurred both on Google+ and on Twitter (as well as in several blog posts that the discussion spawned, like this one by Evan LaPointe and Nancy Koon’s inaugural one and Eric’s post), there were certainly differing points of view, but things never got ugly; I actually had a few people reach out to me directly to make sure that their thoughts hadn’t been taken the wrong way (they hadn’t been)
  • 100s of years of experience — we have a lot of experience from a range of backgrounds when it comes to trying to figure out the stickiest of the wickets that we’re facing. That is going to serve us well.
  • (Maybe) Agencies and vendors leading the way? — I don’t know that I learned this for sure, but an informal tally of the participants in the discussion showed a heavy skewing towards vendor and agency (both analytics agencies and marketing/creative/advertising agencies) representation with pretty limited “industry” participation. On the one hand, that is a bit concerning. On the other hand, having been in “industry” for more of my analytics career than I’ve been on the agency side, it makes sense that vendors and agencies are exposed to a broader set of companies facing the same challenges, are more equipped to see the patterns in the challenges the analytics industry is facing, and are being challenged from more directions to come up with answers to these challenges sooner rather than later.

These were all good things to learn — the people in the community are one of the reasons I love my job, and this thread demonstrated some of the reasons why that is.

Highlights of the Discussion

Boiling down the discussion is bound to leave some gaps, and, if I started crediting individuals with any of the thoughts, I’d run the serious risk of misrepresenting them, so feel free to read the Google+ thread yourself in its entirety (and the follow-up thread that Eric started a few days later). I’ve called out any highlights that came specifically from the hangout as being from there (participants there were Adam GrecoJohn LovettJoseph StanhopeTim WilsonMichael HelblingJohn RobbinsEmer KirraneLee IsenseeKeith Burtis, and me), since there isn’t a reviewable transcript for that.

Here goes:

  • Everyone recognizes that a “just plug it in and let the technology spit out insights” solution will likely never exist — the question is how much of the technical knowledge (data collection minutia, tool implementation nuances, reporting/analysis interface navigation) can be automated/hidden. A couple of people (severalpublicly, one privately) observed that we want (digital) analytics platforms to be a like a high-performance car — all the complexity as needed under the hood, but high reliability and straightforward to operate. Pushing that analogy — how far and fast it runs will still be highly dependent on the person behind the wheel (the analyst).
  • Adobe/Omniture and Google Analytics had near-simultaneous releases of their latest versions; both companies touted the new features being rolled out…but both companies have stressed that there was a lot more about the releases that were under-the-hood changes that were positioning the products for greater advances in subsequent releases; time will tell, no? And, several people who have actually been working  with SC15 (I’ve only seen a couple of demos, watched some videos, and read some blog posts — the main Omniture clients I support are over a year out from seeing SC15 in production), have pointed out that some of the new features (Processing Rules and Context Data, specifically) will really make our lives better
  • There was general consensus that Omniture has gotten much, much better over the years about listening to customer feedback and incorporating changes based on that feedback; there is still a Big Question as to whether customer-driven incremental improvements (even improvements that require significant updates on the back end) will get to true innovation — the “last big innovations” in web analytics were pointed out as being a decade ago (I would claim that the shift from server logs and image beacons to Javascript-based page tags was innovative and wasn’t much older) — or whether “something else” will have to happen was a question that did not get resolved
  • Getting beyond “the web site” is one major direction the industry is heading — integrating cross-channel data and then getting value from it — introduces a whole other level of complexity…but the train is barrelling along on a track that has clearly been laid in that direction
  • We all get sucked into “solving the technical problem” over “focusing on the business results” — the tools have enough complexity that we count it a “win” when we solve the technical issues…but we’re not really serving anyone well when we stop there; this is one of those things, I suspect, that we all know and we constantly try to remind ourselves…and yet still get sucked into the weeds of the technology and forget to periodically lift our heads up and make sure we’re actually adding value; John Lovett has been preaching about this conundrum for years (and he hits on it again in his new book)
  • Marketing/business are getting increasingly complex, which means the underlying data is getting more complex (and much more plentiful — another topic John touches on in his book), which means getting the data into a format that supports meaningful analysis is getting tougher; trying to keep up with that trend is hard enough without trying to get ahead!
  • Tag management — is it an innovation, or is it simply a very robust band-aid? Or is it both? No real consensus there.
  • Possible areas where innovation may occur: cross-channel integration, optimization, improved conversion tracking (which could encompass both of the prior two areas), integration of behaviora/attitudinal/demographic data
  • [From the hangout] “Innovation” is a pretty loaded term. Are we even clear on what outcome we’re hoping to drive from innovation?
  • [From the hangout] Privacy, privacy, privacy! Is it possible to educate the consumer and/or shift the consumer’s mindset such that they are informed about why that “tracking” them isn’t evil? Can we kill the words “tracking” and “targeting,” which both freak people out? Why are consumers fine with allowing the mobile or Facebook application access to their private data…but freak out about no-PII behavioral tracking (we know why, but it still sucks)?
  • [From the hangout] How did a conversation about where and how innovation will occur devolve into the nuts and bolts of privacy? Why does that happen so often with us? Is that a problem, or is it a symptom of something else?

Yikes! That’s my attempt to summarize the discussion! And it’s still pretty lengthy!

What I Learned about Google+

I certainly didn’t expect to learn anything about Google+ when I wrote the post — it was focusing on plain ol’ web (site) analytics, for Pete’s sake! But, I learned a few things nonetheless:

The good:

  • Longer-form (than 140 characters) discussions, triggered by circles, with the ability to quickly tag people, are pretty cool; Twitter sort of forced us over to blog posts (and then comments on the posts) to have discussions…and Google+ has the potential to bring back richer, more linear dialogue
  • Google+ hangouts…are pretty cool and fairly robust; we had a few hiccups here and there, but I was able to participate reasonably well from inside a minivan traveling down the highway that had the other four members of my family in it (Verizon 4G aircard, in case you’re wondering); and, as the system detects who is speaking, that person’s video jumps to the “main screen” pretty smoothly. It’s not perfect (see below), but we had a pretty meaty conversation in a one-hour slot (and credit, again, to Eric Peterson for his mad moderation skills — that helped!)

The not-so-good:

  • Discussions aren’t threaded, and the “+1” doesn’t really drive the organization of the discussion — multiple logical threads were spawned as the discussion continued, but the platform didn’t really reflect that, which many discussion forums have supported for years
  • Linking the blog post to the discussion was a bit clunky. Who knows what long tail search down the road would benefit from seeing the original post and the ensuing conversation? I added a link to the Google+ discussion to the post after the fact…but it’s not the same as having a string of comments immediately following a post (and if Google+ fizzles…that discussion will be lost; I’ve made a PDF of the thread, but that feels awfully 2007)
  • Google+ hangouts could use some sort of “hand-raising” or “me next” feature; everyone who participated in the hangout worked hard to not speak over anyone else, but we still had a number of awkward transitions

So, that’s what I took away. It was a busy week, especially considering I was knocking out the first half of John Lovett’s new book book (great stuff there) at the same time!

Adobe Analytics, Analytics Strategy, Conferences/Community

Announcing ACCELERATE 2011!

We are incredibly excited to announce that registrations are open for our newest community initiative designed for digital measurement, analysis, and optimization professionals, ACCELERATE!

The first event will be held this year in San Francisco on Friday, November 18th at the Mission Bay Conference Center at UCSF, and thanks to generous support from Tealeaf, OpinionLab, and Ensighten, ACCELERATE 2011 is completely free.

Our agenda is still being finalized, but we will have thought- and practice-leaders from amazing companies including Nike, Symantec, AutoDesk, Salesforce.com and, of course, Analytics Demystified. Also, since we recognize that some of the brightest talent in our field works for solution providers, we’ve invited a few practice leaders from the vendor community to present as well, thusly ensuring great content across the board.

The format at ACCELERATE is completely new, and we believe our “Ten Tips in Twenty Minutes” style will create the maximum number of insights possible for attendees of all backgrounds. What’s more, we have a dozen ten open slots for new speakers in our “Super Accelerator” session to showcase up-and-coming talent — and we’re having those folks compete for a $500 gift card from Best Buy based on audience votes.

Did I mention that ACCELERATE 2011 is completely free?

If you’re interested in joining us we encourage you to visit the ACCELERATE 2011 mini-site sign-up register today. Space is limited to the first hundred or so folks who sign up and we’ve already had registrations from New York, Boston, Seattle, Portland, Columbus, and San Francisco.

Go to the ACCELERATE 2011 site and register to attend right now!

EVENT DETAILS:

Location: Mission Bay Conference Center, San Francisco
Date: Friday, November 18, 2011 from 9:00 AM to 4:30 PM
Registration: Open now, limited to the first hundred or so folks who sign up

If you have any questions about ACCELERATE 2011 please leave comments below or email us directly.

Analytics Strategy, General, Social Media

Massive Web Analytics Throw-down in Google+

Much to my chagrin, having been outed by the local newspaper for my original dismissal of Google+, it appears that the web analytics community is prepared to go “all in” in the social network. What’s more, because we’re no longer bound by 100-odd characters (after we @respond and #measure tag), suddenly some incredibly bright minds are able to rapidly contribute to an emerging meme.

Interested? I knew you would be.

Head on over to my stream at Google+ and catch up on the conversation stemming from Tim Wilson’s recent critique of Adobe SiteCatalyst 15. Certainly the thread has diverged somewhat but if you’re in web analytics and on Google+ we would all welcome your contribution.

>>> Web Analytics Platforms are Fundamentally Broken

If you’re not on Google+ click on this link as I have bunches of invites I can share.

Analytics Strategy

Web Analytics Platforms Are Fundamentally Broken

Farris Khan, Analytics Lead at ProQuest and Chevy Volt ponderer extraordinaire, tweeted the question that we bandy about over cocktails in hotel bars the world over during any analytics gathering:

His tweet came on the heels of the latest Beyond Web Analytics podcast (Episode 48), in which hosts Rudi Shumpert and Adam Greco chatted with Jenn Kunz about “implementation tips.” Although not intended as such, the podcast was skewed heavily (95%) towards Adobe/Omniture Sitecatalyst implementations. As the dominant enterprise web analytics package these days, that meant it was chock full of useful information, but I found myself getting irritated with Omniture just from listening to the discussion.

My immediate reply to Farris’s tweet, having recently listened to the podcast, reflected that irritation:

Sitecatalyst throws its “making it much harder than it should be” talent on the implementation side of things, and I say that as someone who genuinely likes the platform (I’m not  a homer for any web analytics platform — I’ve been equally tickled pink and wildly frustrated with Google Analytics, Sitecatalyst, and Webtrends in different situations). I’m also not criticizing Sitecatalyst because I “just don’t understand the tool. ” I no longer get confused by the distinction between eVars, sProps, and events. I’ve (appropriately) used the Products variable for something totally separate from product information. I’ve used scView for an event that has nothing to do with a shopping cart. I’ve set up SAINT classifications. I’ve developed specs for dynamically triggering effectively named custom links. I’ve never done a stint as an Adobiture employee as an implementation engineer, but I get around the tool pretty well.

Given that I’ve got some experience there, I’ve also worked with a range of clients who have Sitecatalyst employed on their sites. As such, I’ve rolled my eyes and gnashed my teeth at the utter botched-ness of multiple clients’ implementations, and, yes, I’ve caught myself making the same type of critical statements that were rattled off during the podcast about companies’ implementations:

  • Failure to put adequate up front planning into their Sitecatalyst implementation
  • Failure to sufficiently document the implementation
  • Failure to maintain the implementation going forward on an on-going basis
  • Failure to invest in the people to actually maintain the implementation and use the data (Avinash has been fretting about this issue publicly for over 5 years)

In the case of the podcast, though, I wasn’t participating in the conversations — I was simply listening to others’ talk. The problem, though, was that I heard myself chiming in. I jumped right  on the “it’s the client’s fault” train, nodding my head as the panel described eroded and underutilized implementations. But, then a funny thing happened. As  I stepped back and listened to what “I” would have been saying, I got a bit unsettled. I realized I’d been seduced by the vendor. Through my own geeky pride at having cracked the nut of their inner machinations, I’d crossed over to vendor-land and started unfairly blaming the customer for technology shortcomings:

If the overwhelming majority of companies that use a given platform use it poorly…shouldn’t we shine a critical light on the platform rather than blaming the users?

I love digital analytics. I enjoy figuring out new platforms, and it’s fun to develop implement something elegantly and then let the usable data come pouring in that I can feed into reports and use for analysis. But:

  • I’ve been doing this for a decade — hands-on experience with a half-dozen different tools
  • It’s what I’m most interested in doing with my career — it beats out strategy development, creative concepting, campaign ideation, and any and every other possible marketing role
  • I’m a sharp and motivated guy

In short…I’m uniquely suited to the space. I’m neither the only person who is really wired to do this stuff nor even in the 90th percentile of people who fit that bill. But the number of people who are truly equipped to drive a stellar Sitecatalyst implementation are, best case, in the low thousands, and, worst case, in the low hundreds. At the same time, demand for these skills is exploding. Training and evangelization is not going to close the gap! The Analysis Exchange is a fantastic concept, but that’s not going to close the gap, either.

There is simply too much breadth of knowledge and thought required to effectively work in the world of digital analytics for a tool to have a steep learning curve with undue complexity for implementation and maintenance. The Physics of the Internet means there are a relatively finite number of types of user actions that can be captured. Sitecatalyst has set up a paradigm that requires so much client-side configuration/planning/customization/maintenance/incantations/prayer that the majority of implementations are doomed to take longer than expected (much longer than promised by the sales team) and then further doomed to be inadequately maintained.

The signals that Adobe is slowly taking steps to merge the distinction between eVars and sProps is an indication that they realize that there are cases where the backend architecture needlessly drives implementation complexity. But, just as the iPhone shattered the expectations we had for smartphones, and the iPad ushered in an era of tablet computing that will garner mass adoption, Adobe has a very real risk of Sitecatalyst becoming the Blackberry of web analytics. Sitecatalyst 15, for all of the excitement Adobe has tried to gin up, is a laundry list of incremental fixes to functional shortcomings that the industry has simply complained about for years (or, in the case of the the introduction of segmentation, a diluted attempt to provide “me, too” functionality based on what a competitor provides).

The vendors have to take some responsibility for simplifying things. The fact that I can pull Visits for an eVar and Visits for an sProp and get two completely different numbers (or do the same thing for instances and page views) is a shortcoming of the tool. We’ve got to get out of the mode of simply accepting that this will happen, that a deep and nuanced understanding of the platform is required to understand the difference, and then gnashing our teeth when more marketers don’t have the interest and/or time to develop that deep understanding of the minutia of the tool.

<pause>

Although I’ve focused on Sitecatalyst here, that doesn’t mean other platforms are beyond reproach:

  • Webtrends — Why do I have to employ black magic to get my analysis and report limits set such that I don’t miss data? Why do I have to employ Gestapo-like processes to prevent profile explosion (and confusion)? Why do I have to fall back on weeks-long reprocessing of the logs when someone comes up with a clever hypothesis that needs to be tested?
  • Google Analytics — Why can’t I do any sort of real pathing? Why do I start bumping up against sampled data that makes me leery…just when I’m about to get to something really cool I want to hang my hat on? Why is cross-domain and cross-subdomain tracking such a nightmare to really get to perform as I want it to?

My point here is that the first platform that gets a Jobs-like visionary in place who is prepared to totally destroy the current paradigm is going to have a real shot at dominating over the long haul. There are scads of upstarts in the space, but most of them are focused on excelling at one functional niche or another. Is there the possibility of a tool (or one of the current big players) really dramatically lowering the implementation/maintenance complexity bar (while also, of course, handling the proliferation of digital channels well beyond the traditional web site) so that the skills we need to develop can be the ones required to use the data rather than capture it?

Such a paradigm shift is sorely needed.

Update: Eric Peterson started a thread on Google+ spawned by this post, and the lengthy discussion that ensued is worth checking out.

Reporting, Social Media

Gamification — One Angle to Consider w/ Social Media Campaigns

At least once a month, something comes up that reminds me of the power of applying the lens of gamification to campaign planning. While slightly off topic for this blog (I’ll touch on measurement towards the end), it’s something that continues to rattle around in my skull, so I might as well work those thoughts out in a post.

The Basics

A fairly succinct explanation of what game mechanics is can be found in a paper published last October by Bunchball, a gamification platform provider:

At its root, gamification applies the mechanics of gaming to nongame activities to change people’s behavior. When used in a business context, gamification is the process of integrating game dynamics (and game mechanics) into a website, business service, online community, content portal, or marketing campaign in order to drive participation and engagement.
:
The overall goal of gamification is to engage with consumers and get them to participate, share and interact in some activity or community. A particularly compelling, dynamic, and sustained gamification experience can be used to accomplish a variety of business goals.

The key here — and this is actually the biggest detriment of the term itself — is that “gamification” is not simply “playing games.” All too often, I have conversations with people who immediately think XBox, Playstation, Kinect, Farmville, or any number of other “traditional games” when the topic of gamification comes up. That’s an entirely appropriate starting point, but it’s by no means the whole story.

Gamification is about using human nature’s inherent interest in being engaged with others, being rewarded, achieving goals, and, yes, having some fun in the process.

A Recent (and Simple) Example

During a #measureX trip to New Orleans, one of the other people on the trip mentioned that she had been doing a lot of travelling lately, and she tries to fly American Airlines, because they have good flights to most of the places she goes, and she is close to reaching the Gold Level of their Frequent Flier Program. Frequent flier programs are an example of gamification applied for the direct benefit of the brand, allowing travelers to earn points towards different levels, at which they are awarded with different perks. These programs don’t directly drive engagement with other consumers, but that’s another key to gamification — it’s not a one-size-fits-all deal.

And an Even More Recent #measure Example

Even the elusive @AnalyticsFTW has indulged in some gamification of late, with an infographic-creating contest to win a pass toeMetrics in NYC, <shamelessplug>where I will be speaking on Twitter analytics </shamelessplug>. It’s simply a matter of offering a prize (a valuable one, in this case), and then letting the #measure community spread the word, with entrants being challenged to come up with something original and amusing. On the one hand, it’s a “simple contest,” but it’s a simple contest that:

  • Forces all potential entrants to actually stop and think about the value of eMetrics
  • Requires an investment of time and energy to illustrate that value in a clever way (which causes them to thinkmore about the value of eMetrics)
  • Generates marketing collateral for the event that others will come and look at (user-generate content, baby! Not a single designer finger on the paid eMetrics team was lifted to generate the material)

It’s brilliant, really.

An Entirely Different (and More Involved) Example

I was tapped/volunteered to teach a “Microsoft Excel Tips & Tricks” brown bag lunch at work a couple of months ago. It was content that I knew attendees would get value out of…but with a title that didn’t exactly have a “Cowboys & Aliens”-type mystique that would be a natural attendance draw (and, while personable enough, I’m not exactly the office equivalent of Daniel Craig or Harrison Ford).

I applied some game mechanics to promote the event by distributing a series of cards around our various offices (physical cards as well as digital versions to our remote locations):

The cards led to a video (PowerPoint with low-fi voiceover) with details as to the “game,” which required participants to do a little searching and a little collaboration with another office before posting “the answer” on the wall of a Facebook group.

Here’s what I hoped to achieve:

  • Engage as many employees as possible just enough with the type of content that I would be presenting that they would have an opportunity to pause and think, “Hmmm… I might actually get something useful out of this”
  • Extend that engagement beyond our main office in Columbus to our satellite offices and remote workers
  • Find out if I could apply game mechanics without consulting a gamification expert and achieve good results

My KPI for the effort was pretty simple: a “healthy turnout” at the brown bag. I had a handful of additional measures in place:

  • Whether or not anyone actually managed to complete the challenge and, if so, how long it took for that to happen
  • The number of clicks on the goo.gl link/QR code link driving to the YouTube video
  • The number of views of the video
  • The number of people who walked by my desk and either chuckled or shook their head

In the end, we had a full room for the brown bag. KPI achieved!

We’re over 300 employees now, and my other measures played out as follows: 159 clicks on the link, 131 video views, and a half-dozen people who chuckled and shook their heads as they walked by my desk. Not bad.

Most surprisingly, though, was how quickly and to what extent people got into the activity. I launched on a Wednesday evening after almost everyone was gone for the day. At 8:29 AM on Thursday morning…9 seconds apart…two people (from two different offices, and they’d both colluded with the same person in a third office) posted the winning answer on the Facebook group’s wall. Considering that I was a little concerned that the whole thing would be a total dud, I certainly didn’t expect to have winners before 8:30 AM on the first day!

Different from “Games”

So, gamification is not simply “playing games.” It’s using the aspects of human nature that make playing games fun and engaging…and then leveraging those to drive interest and engagement around a brand, a product, an event, or something else. It’s an utterly intriguing concept, and it’s not hard to spot examples of marketers putting these ideas to good use.

Another paper/presentation on the subject published late last year by Resource Interactive has some additional good nuggets on the topic:

Game On: Gaming Mechanics

View more presentations from Resource Interactive

 

Measuring the Results

Any marketing initiative should be measured. Campaigns that rely heavily on game mechanics are easier to measure than a lot of always-on social media activities (a Twitter feed, a Facebook page, etc.). That is, they’re easier to measure if there is a clear objective for the effort, and if that objective is something that gamification is good at supporting: driving engagement and/or driving awareness (and education) through word-of-mouth. Meaningful KPIs may include:

  • The number of people exposed to the campaign
  • The number of people who participated in the game mechanics aspects of the game
  • The number of people who reached a certain level of engagement with the campaign

Now, this sets me up for the criticism, “Well, yeah, but did it drive business results.” In some cases, CTAs can be embedded in the game that can lead to conversions that can be measured as results. But, there is, admittedly, some requirement that the entire campaign has been designed with a logical link to business value. For instance, for a low-awareness brand targeted at a niche audience, then a campaign that grows awareness across a community that represents that niche, and that does so at a relatively low cost will often be a no-brainer when compared with low-engagement paid media.

Benchmarks will seldom be available for these types of campaigns. Get over it! If you’re developing a compelling campaign, it’s going to need some degree of originality, which means there won’t be a sea of comparable campaigns at your fingertips for benchmarking. That makes establishing targets a bit scary. Set a target anyway. Think through what would be acceptable and what would be clearly awesome based on other, more traditional ways you could have chosen to invest those same dollars. More often than not, if it’s a well-designed game-mechanics-applied campaign, you will know whether you are on to a good idea early in the planning, and you will be very pleasantly surprised by the results.

Social Media

You’re Using the Wrong Social Media Metrics!

This content originally posted on the ClickZ Marketing News & Expert Advice website on July 14, 2011.

In my experience, I’ve found that the vast majority of practitioners measuring social media currently rely on the wrong metrics. Metrics such as fans, followers, +1’s, shares, likes, and dislikes are easily captured and readily delivered by social networks, but they represent merely the low-hanging fruit of social analytics. These are the “counting metrics” of social media because using them typically equates to counting up digital trivia. Effective measurers of social media go beyond counting metrics to create outcome-based metrics and ultimately report on business value metrics to senior stakeholders across the enterprise. In this column, I’ll elaborate on the minutia of counting metrics and where they can add value to your social media operations, as well as how to take the next step of creating outcome and business value metrics to ratchet up your social analytics game to the next level.

Testing the Social Media Waters

The temptation for businesses to experiment with social media is practically irresistible. And in fact, you’d be foolish not to venture into new and emerging channels if your target audience leads you there. But experimentation and ongoing participation in social media must continually prove out the potential for business value. Often times, this potential is demonstrated in metrics that are indicative of volume and activity. Counting metrics do just that because they are measures that tell you how deep the social media pool really is. These counting metrics are typically the freebies offered by social media networks that quantify the basic observational statistics of participation. The stats include: number of users, number of fans, number of followers, number of posts, number of comments per post, number of check-ins, number of ratings, number of reviews…and so on. You quickly see that there’s numbers on top of numbers.

Yet, stopping at this point and using only counting metrics to measure and manage social media is not only just plain lazy, but also detrimental to your business. These metrics are important for gauging the health and activity of your social media operations, but they fail to tell you if you’re achieving your business goals. Counting metrics can offer insights into how many people are swimming and if the water is too cold, or just right. They can also tell you how many people you are reaching with your social media messages and if your content is worthy of passing on to their friends and followers. But, what counting metrics cannot tell you is who the lifeguards should be watching, and where management needs to focus their efforts. Thus, it’s imperative that you go beyond the counting metrics offered by social media platforms to formulate outcome metrics that constitute real measures of success.

Identifying Outcome Metrics for Social Media Measurement

Stepping away from the pool for a moment, I ask you to consider why you’re participating in social media in the first place. Are you working to build awareness for your new products or services? Do you want to initiate a dialogue with your customers to solicit their input on what you could be doing more effectively? Are you building goodwill with consumers by giving back through social media and encouraging philanthropy? Or, can you increase your profits by selling directly through social media platforms? The answers to these questions reveal the business outcomes that you should be working towards when participating in social media. It’s only when you have a clear understanding of what you’re trying to accomplish with your social media efforts that you can develop truly effective measures of success. If you can’t pinpoint why you’re participating in social media today, or if your answers are flimsy and won’t stand up to the scrutiny of executive leadership, I strongly advise that you stop everything and rethink your efforts.

However, if you have a strategic vision of what you’re trying to accomplish with social media, then developing your outcome metrics will become a much easier task. For example, if gaining exposure is the outcome that you are after, then metrics like reach, velocity, and share of voice will be extremely helpful in determining your progress toward this outcome. Similarly, if you’re working to foster a dialogue with customers, focus on metrics like audience engagement, key influencers, and trending topics. Or if cold hard cash is what you’re after, then metrics like social referral source, cost per acquisition, conversion rates, and average order value will illuminate progress toward your stated social media outcomes. Each of these metrics tells you how well you’re doing according to plan and reveals valuable business information.

Demonstrating Social Media Business Value

Now that you’re straight on using counting metrics for sizing up opportunities and outcome metrics for quantifying purpose, the next step is tying all this together to communicate your fabulous progress. To do this, you need to detach yourself from the metrics that you use everyday to manage your social operations and translate these granular metrics into more generalized business language. Think carefully about the things that matter to your organization and the stakeholders that oversee the business and communicate in ways that resonate with them. In most cases, this means aligning your business objectives with corporate goals. Demonstrate which social media channels are contributing to new customer acquisition, which are adding dollars to the corporate coffers, or which are elevating customer satisfaction. This takes some skill and corporate savvy to indoctrinate non-believers into the world of social media metrics, but it’s an entirely worthwhile endeavor that will pay dividends for your organization in the long run.

I’ve found that the most effective way to present a strategic plan and communicate your successes using metrics is to leverage a framework for social media measurement. The one I use includes an inside-out strategy that begins with corporate goals, then aligns business objectives, maps these to measures of success, and then extends out to operational tactics. Using this framework allows me to solicit feedback from stakeholders by actually including them in the planning process of developing social media programs. This encourages participation and gives everyone involved a vested interest in the success of social media endeavors. Ultimately, your social media metrics should build from the basic counting metrics to outcome-based objectives that wholly support your corporate goals. Once you have a solid plan and a strategic roadmap for how you’ll stitch this all together, then you’re ready to dive into the deep end of the social media pool.

Analytics Strategy, Social Media

Monish Datta: "Justin Kistner KNOWS Facebook Measurement!"

We had a fantastic Web Analytics Wednesday last night, sponsored by Webtrends with social media measurement guru Justin Kistner providing a wealth of information about Facebook measurement (and Facebook marketing in general).

With almost 50 attendees, we were, as best as I can tell, tied with the largest turnout we’ve ever had. Is “number of attendees” an appropriate success measure? Well, yeah, it is. Even better that the group was super-engaged, and I’ve never had so many people track me down to laud the content (including multiple follow-up requests as to whether I had the deck yet!

Justin was gracious enough to share his presentation, and it’s posted below (click through to Slideshare to download the source PowerPoint):

A handful of pictures from the event:

Mingling/Eating/Drinking

Food, Drink, and Chatting

Justin Launches His  Presentation

Justin Gets Things Rolling

The Late Night Lingerers
(that’s Monish Datta in the middle — a wholly gratuitous reference in pursuit of SEO silliness)

The Late Lingerers

Adobe Analytics, Technical/Implementation

SiteCatalyst Implementation Pet Peeves – Follow-up [SiteCatalyst]

I recently blogged a list of my top Omniture SiteCatalyst implementation “Pet Peeves.” While the response to my post was very positive, one reader agreed with most of what I said, but disagreed with a few of my assertions or felt I had made some omissions. First, let me state that I always encourage feedback and comments to my blog posts since that helps everyone in the community learn. In general, the reader was making the point that my post only took into account an implementer’s perspective vs. the perspective of the web analyst. Personally, I don’t like to divide the world into implementers and analysts, since some of the best implementers I know also have a deep understanding of web analysis and vice-versa. Having been a web analytics practitioner using SiteCatalyst at two different organizations, I feel that I am in a good position to know if items I suggest (or discourage) will lead to fruitful analysis. I always try to write my blog from the perspective of the in-house web analyst who has to deal with things that I dealt with in the past, such as adoption, enterprise scalability, training, variable documentation, etc… In fact, I attribute much of my consulting success to the fact that I have been in the shoes of my clients and that they appreciate that my recommendations are based upon actual pains that I have experienced.

Since my original post was a very quick “Top-10” list and didn’t provide an enormous amount detail, and given the interest that it generated, I thought it would be worthwhile to write this follow-up post to address the concerns raised related to my post and to elaborate on the rationale behind some of my original assertions. In the process, it will become clear that I don’t necessarily agree with the concerns raised to my original post, but I am always cognizant of the fact that every client situation is different and every SiteCatalyst implementer has experiences that color their own implementation preferences. I don’t see it as my place to say which techniques are right and which are wrong, but rather to do my best to state what I think is/is not “best practice” and why based upon what I have seen and experienced over the past ten years and let my readers decide how to proceed from there…

Tracking Every eVar as an sProp

The first pet peeve I mentioned is when I find clients that have duplicated every eVar with a similar sProp. I stated that there are only specific cases in which an sProp should be used including a need for unique visitor counts, Pathing, Correlations and to store data that exceeds unique limits for accessing in DataWarehouse. The reader seemed to think I was being hard on the poor sProp and listed a few other cases where they felt duplicating an eVar with an identical sProp or adding additional sProps was justified including:

  1. Using List sProps – The reader suggested that I had made an omission, by not mentioning List sProps as another reason to consider using an sProp in an implementation. I maintain that the use of List sProps was justifiably covered in my statement of other sProp uses that are “few and far between.” I don’t use List sProps very often because I feel that there are better ways to achieve the same goals. As the reader stated, List sProps have severe limitations and there is a reason that they are rarely used (maybe 2% of the implementations I have seen use them). I have found that you can achieve almost any goal you want to use List sProps for by re-using the Products variable and its multi-value capabilities instead. By using the Products variable, you can associate list items to KPI’s (Success Events) rather than just Traffic metrics. Using the reader’s own example of tracking impressions, illustrates the differences perfectly. You can store impressions and clicks of internal ads and calculate a CTR using the Products variable and two success events. This also gives you charts for impressions, clicks and the ratio of the two which can be easily added to SiteCatalyst dashboards. I have found that doing this with a List sProp is difficult, if not impossible and reporting on it is tedious. For more information on my approach, please check out my blog post on the subject.
  2. Page-Based Containers & Segmentation – Here the reader suggested that the need to isolate specific pages using a Page View-based container is important to the life of the web analyst. Ben Gaines from Omniture also commented about this on my original post and I do agree that this can be useful for some advanced segmentation cases. I did not include this in my original list because I find it to be a much more advanced topic than I intended to cover for this quick “Top 10” post. While there may be cases in which you want to set an sProp to filter out specific items using a Page View-based segment container, I find that I often do this using the Page Name sProp which is already present. I do not see too many cases where a client is storing an eVar (let’s say Zip Code) and will say, “I am going to duplicate it as an sProp for the sole purpose of building a Page-Based container segment to include or exclude page views where a page is seen where a Zip Code equaled 123456.” Maybe that happens sometimes, but I still think it falls out of the scope of the primary things you should be considering when deciding whether to duplicate an eVar and I think it is a stretch to say that this functionality establishes the line between those who care about implementation and those who care about web analysis.
  3. Correlations – With respect to Correlations, the reader suggested that users correlate as often as they can since cross-tabulation is so essential to the web analyst. This is exactly why I included Correlations in my list! I also mentioned that this justification for using an sProp may go away in SiteCatalyst v15 where all eVars have Full Subrelations. Also, one of the reasons I prefer Subrelations to Correlations is that Correlations only show intersections (Page Views) and do not show any cross-tabulation of KPI’s (Success Events). Personally, I would disagree with the reader about over-doing Correlations, since in my experience, implementing too many Correlations (especially 5-item or 20-item Correlations), with too many unique values, can cost a lot of $$$, lead to corruption and latency.
  4. Pathing – In the area of Pathing, I think the reader and I are on the same page about its importance which is why I have published so many posts related to Pathing such as KPI (Success Event) Pathing, Product Pathing, Page Type Pathing, etc… Again, I might differ with the reader in that I don’t think enabling Pathing on too many sProps is a good idea since it can cost $$$ and produce report suite latency, which is why I prefer to use Pathing only when it adds value.

At the end of the sProp duplication section, the reader stated that there was no downside to duplicating every eVar as an sProp since it has no additional cost. To this, I would reiterate that my post was not advocating abandoning the use of sProps, but instead, attempting to help readers determine when they might want to use sProps so as to avoid over-using them when they will not add additional value. Even after years of education, I still find that many clients get confused as to whether they should use an eVar or an sProp in various situation, and most people I speak to welcome advice on how to decide if each is necessary.

However, I disagree with the reader’s assertion that duplicating every eVar as an sProp has no costs. Maybe it is due to the fact that I have “been in the trenches,” but in my experience I have seen the following potential negative ramifications:

  • Over-implementing variables and enabling features unnecessarily can cause report suite latency
  • Over-implementing variables can increase page load time, which can negatively impact conversion
  • Over-implementing variables and features can cost additional $$$ as described above (e.g. Pathing, Correlations)
  • When you implement SiteCatalyst on a global scale, you often need to conserve variables for different departments or countries to track their own unique data points. This means that variables (even 75 of them!) are at a premium. Therefore, duplicating variables has, at times, caused issues in which clients run out of usable variables.
  • Most importantly, however, is the impact on adoption. Again, I may be biased due to my in-house experience, but here is a real-life example: Let’s say that you have duplicated all eVars as sProps. Now you get a phone call from a new SiteCatalyst user (who you have begged/pleaded for six months to get to login!). The end-user says they are trying to see Form Completions broken down by City. They opened the City report, but were only able to see Page Views or Visits as metrics. Why can’t they find the Form Completions metric? Is SiteCatalyst broken? Of course not! The issue is that they have chosen to view the sProp version of the report instead of the eVar version. That makes sense to a SiteCatalyst expert, but I have seen the puzzled look on the faces of people who don’t have any desire to understand the difference between an sProp and an eVar! In fact, if you try to explain it to them, you will win the battle, but lose the war. In their minds, you just implemented something that is way too complicated. You’ve just lost one advocate for your web analytics program – all so that you can track City in an sProp when you may not have needed to in the first place. In my experience, adoption is a huge problem for web analytics and is a valid reason to think twice about whether duplicating an sProp is worthwhile. While I’ll admit that duplicating all variables certainly helps “cover your butt,” I worry about the people who are at the client, left to navigate a bloated, confusing implementation…

Therefore, for the reasons listed above, I remain steadfast in my assertion that there are cases where sProps add value and cases where they just create noise. While there will always be edge cases, I think that the justifications I laid out in my original post are the big ones that the majority of SiteCatalyst clients should think about when deciding if they want to duplicate an eVar as an sProp or use an sProp in general.

As an aside, while we are revisiting my original post, I thought of a few more items I wish I would have included so I will list them here:

  1. One other justification for setting an sProp I should have mentioned is Participation. There are some fun uses of Participation that can improve analysis and I find that sProp Participation is easier to understand for most people than eVar Participation so I would add that to my original list.
  2. If you do find a need to duplicate an eVar as an sProp, but it is only for “power users,” keep in mind that you can hide the sProp variable from your novice end-users through the security settings under Groups.
  3. Finally, I see Omniture ultimately moving to a world where there will only be one variable so if you want to be part of that world, please vote for my suggestion of doing this in the Ideas Exchange here.

VISTA Rules

Another pet peeve I mentioned is that I often find clients who are using VISTA rules too often or as band-aids. The reader stated that VISTA rules are a good alternative to JavaScript tagging since they can speed up page load times. I think this is another situation where my time working at Omniture and in-house managing SiteCatalyst implementations may bias my recommendations. While I agree that page load time is important, most Omniture clients I saw never mentioned using VISTA rules as a way to decrease page load time, but rather as a way to avoid working with IT! Usually, when I find a client that has many VISTA rules, it is because they have a bad relationship with IT, who doesn’t want to do additional tagging, rather than to save page load time. However, if I were to address the reader’s point of page load speed, I would agree that there are cases where using VISTA rules over JavaScript can decrease page load time, but I certainly do not think this should be the primary deciding factor. Great strides have been made in tagging including things like dynamic variable tagging and tag management tools which have greatly reduced page load speeds. I suggest readers check out Ben Robison’s excellent post on VISTA vs. JavaScript which discusses not only page load speed, but also the many other important factors to consider before jumping into VISTA rules.

Another point I’d like to make about VISTA rules is that, in my experience, they have a high likelihood of breaking and leading to periods of bad data. VISTA rules are like Excel macros. They do what you tell them to do, but if something changes, it can easily throw off a VISTA rule and cause incomplete or inaccurate data to be reported in SiteCatalyst. In this point, perhaps I am a bit jaded because I have seen so many different VISTA implementations go awry while I was at Omniture. In fact, it is rare that I find clients that have a VISTA rule that has worked for several years without ever having an issue. And if you do encounter an issue, you will have to pay Omniture around $2,000 to update it – every time. Want to make an update to the VISTA rule? $2,000. Want to turn off the VISTA rule or move it to a different report suite? $2,000! Consultants don’t have to write these checks, but guess who does – the in-house people do! This is why people are so excited about the new V15 processing rules and emerging tag management vendors. It is this tendency to break and the risk of bad data that makes me a bit gun-shy about using VISTA rules simply as a replacement for JavaScript tagging. Moreover, since the reader’s overall premise was that one must keep the web analyst in mind during implementation, I would be cautious about being overly-reliant on a solution like VISTA that is so prone to causing data issues which could thwart the analyst’s ability to do web analysis. I have seen companies that have 20+ VISTA rules and I promise you that they are not huge fans of VISTA right now (though they should really blame themselves not the tool!)! If you do pursue VISTA rules, my advice is that you consider using DB VISTA over VISTA. DB VISTA rules cost a bit more, but do offer more flexibility since you can at least make updates to the data portion of your rules without having to pay Omniture additional $$$.

One additional point to think about when it comes to VISTA rules is the impact they can have on report suite latency. Having too many VISTA rules can slow down your ability to get timely data in SiteCatalyst and I have seen many large organizations have severe (several days) report suite latency due to multiple VISTA rules acting on each server call. This impacts the web analyst’s ability to get the data they need and should be factored into decisions about VISTA rules.

As I stated in my original post, I have nothing against VISTA rules, but do find the overuse of them to be a potential red flag when I look at a new implementation. I often find that excessive use of VISTA Rules can be a symptom of bigger problems which merit investigation. Just like I don’t advocate duplicating sProps or enabling Pathing when not necessary, I don’t advocate the use of too many VISTA rules since it can be great in the short term, but bad in the long term. Now that I am a consultant again, it would be easy for me to recommend VISTA rules left and right, but since I like to have long-term relationships with my clients, I don’t do this since I know what it is like to be around later if/when they have issues!

Final Thoughts
I hope this post provides some good food for thought and more in-depth information about some of the items I listed in my original post. If you would like to discuss any of the above topics in more detail, feel free to leave comments here or e-mail me. Thanks!

Analytics Strategy

An Explanation of Sitecatalyst Events for the Google Analytics Power User

This post is one half of a 2-post series of which, most likely, you are looking for only one of the two posts!

Here’s the guide:

  • If you are well-versed in Google Analytics and are trying to wrap your head around Adobe Omniture (Adobiture) Sitecatalyst “events,” read on! This is the post for you!
  • “If you are well-versed in Sitecatalyst and are trying to wrap your head around Google Analytics “events,” then this sister post is probably a better read.

If you’re looking for information about Coremetrics or Webtrends…well, you’re SOL. If you’re looking for a great flour tortilla recipe, then my limited SEO work on this blog has run so far amok that I’ll just thank my lucky stars that I’m an analytics guy rather than a search guy (but, hey, here’s a great recipe, anyway).

Why “Events” Seem Similar in Google Analytics and Sitecatalyst

At a basic/surface/misleading level, events in Google Analytics and Sitecatalyst are similar. In both cases, they’re something that are triggered by a user action on the site that then sends a special type of call to the web analytics tool:

Alas! The similarity ends there! But, since no one learns multiple tools simultaneously, this surface similarity causes confusion when crossing between tools. Hopefully, these posts will help a person or two overcome that messiness.

Google Analytics Events — Conceptually

Let’s just start by making sure we’re on the same page when it comes to Google Analytics events. In a nutshell, they’re just a handy way to record user actions that don’t get picked up by the base page tag and that don’t get recorded as page views, right? They’re useful for recording outbound link clicks, activities within Flash or DHTML content that don’t warrant the full “page view” treatment (and, hopefully, you have a standard and consistent approach for determining when to use an “event” and when to use a “virtual page view”). Those are Google Analytics events at their most basic level, right? Okay. Cool. Let’s continue.

Sitecatalyst Events — Different in Concept from Google Analytics Events

In Sitecatalyst, events are much more conceptually similar to Google Analytics goals than they are to Google Analytics events (except, of course, when it comes to their name!). In olden times (web analytics olden times, that is), so I hear, Sitecatalyst events were actually called “KPIs” — a nod to the fact that many of the best KPIs are about what visitors do on a site, rather than simply being related to the fact that they arrived on the site in the first place.

So, a key point:

Sitecatalyst “events” really are more like Google Analytics “goals” than they are like Google Analytics “events.”

Sitecatalyst Events — Different in Implementation from Google Analytics Events

If we can agree that Sitecatalyst events are really more like Google Analytics goals than they are like Google Analytics events, then it’s worth pointing out that Sitecatalyst events are primarily set/captured/identified client-side (on a page), while Google Analytics goals are primarily configured/set on the back end by a Google Analytics admin user.

Google Analytics goals are typically established by specifying a specific visitor activity or behavior (or set of behaviors) using already-being-captured data (page title, page URL, time on site, number of pages viewed, or, as of v5, triggering of a specific event category/action/label/value) as a “goal” within a profile. Once the goal is defined, you can track the number of visitors who complete that goal, the conversion rate, and a conversion funnel. And, you can do this (funnels excepted…unless you’re prepared to get fancy pants about it) by visitor segment. In short:

Google Analytics goals are primarily configured on the back end.

Now, you may occasionally do some tweaking on the client (page) side of things in order to enable you to set up the goals, but I’m not going to digress on that point (leave a comment if you’d like me to elaborate and I will).

With Sitecatalyst events, the main work occurs on the client side of things. You establish what activities/occurrences warrant “event” status, and then you make sure your site is configured so that the appropriate event(s) gets recorded any time that activity occurs. Typically,the event occurs at the same time — and in the same Sitecatalyst page tag call — that a view of a page (the pageName) and various other data gets passed to Sitecatalyst. For instance, when a visitor completes a site registration, the Sitecatalyst page tag will likely need to record the traffic to the confirmation page (via pageName and additional sProps) as well as the “event” of a registration being completed. This event (or “completion of a goal”) will be recorded as “event=eventx” in the page tag call. To make use of “eventx” (event1, event2, etc.) requires some backend configuration (including whether the event is serialized or not…but that’s another digression I will avoid). But, for the most part:

Unlike Google Analytics goals, Sitecatalyst events are primarily set up client-side — via values in the “event” variable recorded by the Sitecatalyst page tag.

eVars (aka “conversion variables”) and Sitecatalyst events –> Google Analytics Segments

With Google Analytics goals, it is handy to use segments — standard or advanced — to explore how different subsets of visitors behave. For instance, how do visitors who viewed product details tend to convert to an order as opposed to visitors that do not? Do visitors that entered the site via organic search reach product details pages at a higher rate than visitors who arrived as direct traffic? You get the idea.

Excepting the segmentation capabilities of the lugubriously rolling out v15, as well as the capabilities of Discover, Sitecatalyst relies largely on eVars to do this sort of exploration. With scads of available eVars to work with, and with the appropriate use of subrelations (allowing the cross-tabulation of eVars), it’s largely a matter of solid up-front thinking and planning, combined with a willingness (and ability) to make site-side adjustments over time, to get at “segmented conversions” using Sitecatalyst.

eVars are both a blessing and a curse when viewed through a Google Analytics lens. They’re a blessing because they allow much more detailed and sophisticated capture and analysis of conversion data. They’re a curse because they require much more site-side planning and implementation work!

Does This Help?

Describing  this distinction in a clarifying manner is tricky — it’s quite confusing and frustrating…until it makes sense, at which point it’s hard to identify exactly what made it so confusing in the first place!

If you’ve gone through (or are going through) the process of adding Sitecatalyst to your toolset after being deeply immersed in Google Analytics, please leave a comment as to how you overcame the “events” hurdle. With luck, others will benefit!

Analytics Strategy

An Explanation of Google Analytics Events for the Sitecatalyst Power User

This post is one half of a 2-post series of which, most likely, you are looking for only one of the two posts!

Here’s the guide:

  • If you are well-versed in Adobe Omniture (Adobiture) Sitecatalyst and are trying to wrap your head around Google Analytics “events,” read on! This is the post for you!
  • If you are well-versed in Google Analytics and are trying to wrap your head around Sitecatalyst “events,” then this sister post is probably a better read.

If you’re looking for information about Coremetrics or Webtrends…well, you’re SOL. If you’re looking for a great refried beans recipe, then my limited SEO work on this blog has run so far amok that I’ll just thank my lucky stars that I’m an analytics guy rather than a search guy (but, hey, here’s a great recipe, anyway).

Why “Events” Seem Similar in Google Analytics and Sitecatalyst

At a basic/surface/misleading level, events in Google Analytics and Sitecatalyst are similar. In both cases, they’re something that are triggered by a user action on the site that then sends a special type of call to the web analytics tool:

Alas! The similarity ends there! But, since no one learns multiple tools simultaneously, this surface similarity causes confusion when crossing between tools. Hopefully, these posts will help a person or two overcome that messiness.

Sitecatalyst Events…Conceptually (Just to Make Sure We’re on the Same Page)

With Sitecatalyst, an event is a success event — a conversion to an on-site activity you care about. I was told by a reliable source that, in early versions of Sitecatalyst, events were actually  called KPIs — a nod to the fact that many of the best KPIs are about what visitors do on a site, rather than simply being related to the fact that they arrived on the site in the first place. So, are we cool with that high-level definition of a Sitecatalyst event? Good. Let’s continue…

Google Analytics Events — Conceptually, a Completely Different Animal

A Google Analytics event is very different in both concept and in application from a Sitecatalyst event. It’s much more akin to Sitecatalyst link tracking or a “non-standard” Sitecatalyst call triggered for the sake of counting some activity other than viewing of a basic HTML page (viewing of content in Flash or DHTML…although these also use virtual page views — more on that in a bit) either via pageName or an sProp.

Events are, simply put, a way to record a user action that warrants being recorded, but that does not warrant being counted as a page view.

While events in Sitecatalyst, almost by definition, are “significant” actions — a product added to a car, an order completed, a product details page viewed, a site registration — Google Analytics events are often of much less on-going importance. For instance, they may include:

  • The use of minor navigational buttons or elements (in Flash, in DHTML, or elsewhere)
  • The reaching of a certain point (half viewed, 3/4 viewed, 95% viewed) in a streaming video
  • The exit from the site on an outbound link (virtually identical to Sitecatalyst “exit links,” but requiring explicit coding/customization to track using Google Analytics events)

Up until the most recent release of Google Analytics, and much to the chagrin of analysts the world over, Google Analytics events could not be set as “goals,” and goals in Google Analytics — configured by a Google Analytics admin user rather than anywhere in the page tag — are the closest that Google Analytics comes to the concept of a Sitecatalyst “event.”

Did you catch that? If you’re looking to implement something akin to a “Sitecatalyst event” in Google Analytics, read up on Google Analytics goals.

“eventx” (Sitecatalyst) vs. Category/Action/Label/Value (Google Analytics)

Another, albeit lesser and secondary, difference is that Google Analytics events are named and categorized at the point when the event is fired by a user action. As such, you won’t see a Google Analytics event called event1, event2, etc. that subsequently needs to be described and named on the back end. Google Analytics events have the requisite meta data built into the page-side recording of the action through the inclusion of a “category,” an “action,” and an (optional) “label” and “value” that will then appear in Google Analytics event reporting just as the values were called from the page. Plentiferous detail on the mechanics and syntax are available in the Google Analytics documentation on event tracking.

This is similar to how Google Analytics handles campaign tracking — all of the meta data about a campaign is included in multiple parameters in the target URL for the campaign, whereas, with Sitecatalyst, you have the opportunity to simply use SAINT classifications to map an alphanumeric campaign tracking ID to a range of different classifications.

Google Analytics Events vs. Virtual Page Views

One final area of confusion is the popular “when to use an event versus when to use a virtual page view in Google Analytics” conundrum. Sitecatalyst power users transitioning to Google Analytics can get all sorts of twisted in the head on this, as the basic question just doesn’t make sense…if they’re thinking about “events” in Sitecatalyst terms. If the question doesn’t make sense to you, er…re-read the first half of this post (or leave a scathing comment as to how non-elucidating the first part of the post is!).

When to use one or the other is both situational and a judgment call. The general rule our analysts apply is to consider whether the activity meets either of the following conditions:

  • Activity that is already being recorded as a standard page view elsewhere (e.g., clicks on home page promo areas where the target URL is on the same site and will soon be recorded as a page view…but where you want to be able to easily report or analyze individual promo or promo location clickthroughs)
  • Activity that the visitor would not consider as “viewing a new ‘page’ of content” (e.g., reaching the halfway point in a streaming video)

If either of these criteria is met, our bias is to record the activity as an event rather than as a virtual page view.

(Until recently, the exception to this criteria was if the user action was a “goal” for the site. Since events could not be set as goals, we would be required to a virtual page view. But, Google has, happily, added the ability to use events as goals in v5!)

Does This Help?

Describing  this distinction in a clarifying manner is tricky — it’s quite confusing and frustrating…until it makes sense, at which point it’s hard to identify exactly what made it so confusing in the first place!

If you’ve gone through (or are going through) the process of adding Google Analytics to your toolset after being deeply immersed in Sitecatalyst, please leave a comment as to how you overcame the “events” hurdle. With luck, others will benefit!

Adobe Analytics, Conferences/Community, General

Great jobs and a great gathering in Atlanta next week

Just a quick note from my vacation getwaway to call reader’s attention to two great jobs at The Home Depot and to let Atlanta-area readers know that I will be in town next week for a special “Web Analytics Wednesday on Tuesday” put together by Keystone Solutions Rudi Shumpert and HD’s own Wesley “Big Wes” Hall. The event will be at the Gordon Birsch in Buckhead and I’m hoping that Rudi and Wes will allow an informal Q&A session about some of the great things that are happening in our industry lately.

>>> Register to join us at Web Analytics Wednesday, Atlanta, on Tuesday, July 19th

Regarding the jobs, our client at Home Depot is aggressively putting together a team of digital measurement specialists to help lead the company’s digital efforts forward. We have been helping the company with their digital measurement strategy now for about six months and the effort is really beginning to pay off in terms of their use of technology, the talent they are getting in the door, and the value web analytics brings to the company both online and off.

Have a look at the Senior Analyst and Manager, Web Analytics jobs on our web site and come see me next week at Web Analytics Wednesday if you’d like a personal introduction or have any questions:

>>> Job description, Senior Web Business Analyst at The Home Depot

>>> Job description, Web Analytics Manager at The Home Depot

I hope you are all having a great, relaxing summer and look forward to seeing you at a conference, event, or Web Analytics Wednesday sometime in the near future.

Analytics Strategy

Quick Tip: Track Your Omniture JS File Version [SiteCatalyst]

Have you ever had someone run a report in SiteCatalyst and come running to you saying something like this?

This report doesn’t make sense…There is obviously a tagging issue and you need to fix it ASAP!

If I had a dollar for every time this happened to me, I’d be a rich man! The truth is, that after many wild goose chases, the problem is not usually tagging related (note you can use tools like ObservePoint and DigitalPulse to verify). But if it ever was tagging that caused the issue, it was usually related to the release of a new JavaScript file. That has been the culprit many times for me over the years. Therefore, in this post, I will share a trick you can use to easily find out if data issues you might be experiencing might be related to a new JavaScript file release.

Tracking Your JavaScript File

So how can you use SiteCatalyst to determine if a new JavaScript file you released is wreaking havoc on your data? For example, let’s imagine a scenario where the morning of May 18th, you started seeing some strange data irregularities (possibly by checking Data Quality as described here!). Here is what you need to do:

  1. Each time you create a new version of your JS file, assign it a version number (i.e. 0.5, 0.8, 1.2)
  2. Pass this version number into a tool that can store it and let you know when it sees the version number change
  3. Look at a report that shows you when the version number value has changed (what date it changed and at what time)

Sounds easy right? If only we knew of a tool into which we could pass data, have it be time-stamped and report upon changes in version number values…Hmmm….Where would we find such a tool??

Obviously, we already have that tool and it is SiteCatalyst! We can use the tool we know and love to track each version of the JavaScript file by simply passing in the version number of the file into an sProp on every page (and yes, I get the irony that we are using a JavaScript file which sets a beacon to enable tracking to track itself!). By doing this, we will have a historical record of when each JavaScript file was released. After you pass in the JavaScript File version you will see a report like this:

Here we can see the distribution of page views related to each JavaScript file version. In this case, we have been busy and have had four JavaScript file changes in one month! However, this report isn’t super-useful in answering our initial question: Were the issues we saw on the morning of May 18th related to a new JavaScript file release?

To answer this question, all we have to do is to switch to the “trended” view of this report and we will see a report like this:

Now we can start to see the flow of JavaScript file updates. Looking at this report, we can see that we moved from version 0.5 to version 0.7 (poor version 0.6!) on May 18th… This might support our hypothesis, but to be sure, we can look at this report by hour on May 17th & 18th and see this:

 

Now we can narrow things down to an hour and it looks like the JavaScript file did, in fact, change around 9:00 am on May 18th. As you can see, the simple action of taking the administrative step of keeping your JavaScript file in an sProp can provide an easy way for you to do some sleuthing when you are in a pinch!

Additionally, if you want to further test your hypothesis, you can isolate data that is related to a specific JavaScript file to see if it represents the issue you are seeing. To do this, simply use DataWarehouse to create a segment that only pulls pages that had data collected using a specific JavaScript file version as shown here:

 


Adobe Analytics

Some SiteCatalyst Implementation Pet Peeves [SiteCatalyst]

Over the years, when I have consulted clients who use the SiteCatalyst product, I have encountered some strange implementation items that made me scratch my head. In the beginning, when I saw these odd implementation quirks, I was mildly entertained, but as I saw them more and more, they were soon elevated to “pet peeve” status. Therefore, I thought I’d share some of these items with you to make sure that you are not doing any of them, and also because I am curious to see what other “pet peeves” you may have. Please check out my list (which is by no means exhaustive!) and if you have items that you have seen that bug you, please leave them here as a comment!

Tracking Every eVar as an sProp

I would say that my biggest pet peeve is when clients have an sProp for every eVar they have set (or vice versa). When I see this, it is an early warning sign that the client doesn’t fully understand the fundamentals of SiteCatalyst. While there are definitely cases where you would capture the same data in both an eVar and an sProp, they are usually few and far between. As a rule of thumb, I only set an sProp if:

  • There is a need to see Unique Visitor counts for the values stored in the sProp
  • There is a need for Pathing
  • You have run out of eVar Subrelations and need to break one variable down by another through the use of a Correlation (which will go away in SiteCatalyst v15)
  • There will be many values (exceeding the unique limits and you just want data stored so I can get to it in DataWarehouse or Adobe Insight

For the most part, that is it… Beyond that, I tend to use eVars and Success Events for most of my implementation items.

This is why I shudder when I see 40 eVars set and the same 40 sProps set. I find that this only confuses users since most don’t really understand the difference between the two variable types to begin with! Therefore, my advice is to make sure you understand the difference between eVars and sProps and make sure you use the right variable for the right purpose.

Pathing Enabled Unnecessarily

Another item I have seen a lot is when a customer will have Pathing enabled on an sProp that doesn’t change in a session. For example, let’s say you have people log into your website and you store the Customer ID in an sProp. That Customer ID is designed to be the same for each visitor during the entire visit. However, I often see clients who enable Pathing on this Customer ID sProp. My hunch is that they think this will show them the paths of that Customer ID, but the truth is that it will show no paths for each Customer ID so it is a complete waste of time. Keep in mind that pathing is only useful if values change in the same session. If you pass the same value in on every page of the session, SiteCatalyst will see that as a Bounce of 100% for every Customer ID! Since Adobe (Omniture) will only let you have so many variables with Pathing enabled, you need to make sure you are using them wisely!

No Friendly Page Names

The next pet peeve is when clients don’t pass any values to the Page Name variable and use the default of the URL. This really makes my blood boil! There are so many downsides to doing this when it comes to the Pages report since it impacts Page Views, Unique Visitors and Pathing. For better or worse, the Pages report tends to be a very popular one and I feel that, even if just for the perception of the integrity of your web analytics implementation, you need to take the time to make sure this report is accurate and understandable. For more information on this topic, please refer to my Page Naming Best Practices post by clicking here.

Passing Query Strings to Page Name Variable

On a related note, I have another gripe related to the Page Name variable and it has to do with query string parameters. Many times I find that companies are including query string parameters in the Page Name variable. This is a really bad idea. Here are two common things I see:

  • When a visitor arrives to the website from a campaign, the URL will have a campaign code in the query string parameter and pass this to the Page Name variable (i.e. zyz corp:home:homepage:cid-12345)
  • A company will have a search results page and include the keyword/phrase that the user searched upon to get to that page in the Page Name (i.e. zyz corp:search:searchresults:user manual)

Both of these examples involve the company having one page name essentially split out into hundreds (or thousands) of versions of the Page Name due to the query string parameter. Creating many versions of the same page has the effect of losing Visits, Unique Visitors and Pathing for the true Page Name. Most of the time this situation can be solved by using one Page Name and passing the query string parameter to another variable and using a Correlation. If you really need to have these extra query string parameters associated with pages, I recommend using another sProp instead of the Page Name variable…

Reports with No Data

Another thing I see quite often are implementations that have tons of variables labeled, but that have no data. As a rule of thumb, I recommend you disable any variables that have no data or at a minimum hide them from the menus using the Admin Console. There is nothing more frustrating to an end-user than opening up a report, getting excited to see the data and then realize that there is none! Besides being annoying, it hurts the credibility of your web analytics program. When I am in the midst of a new implementation and things are in flux, one thing I do is to put all reports that are coming, but have no data in ALL CAPS or I add the phrase “(COMING SOON)” after the variable name. This helps me see which variables are left to do and which ones I can begin to QA. However, once the implementation is semi-stable, I urge you to hide variables that are not coming for a while so you don’t annoy people unnecessarily!

No Menu Customization

On a related note, how many SiteCatalyst implementations have you seen where they use the default menu structure? Why would you want to tell users to look in “Customer Conversion 1-10” to find the report they are looking for? Not very helpful is it?

Instead, you should customize your menus so they make sense for your users. This will help in your adoption and make training much easier. For some great tips on how to customize your menus, check out Brent Dykes’ post by clicking here.

No Variable Standardization

The next one is when you have a situation where you have multiple report suites that are really the same website, just for different business units and/or locations and none of them are set-up consistently. I see many clients who are tracking some things in the US, but not in the UK or Japan, even though the websites are identical. When this happens and you select multiple report suites in the Admin Console, here is what you see in the variable screen:

I call the “Multiple Madness” due to what you see in the Admin Console and it is not a good thing! You should make sure that as many of your report suites are as consistent as possible so you can minimize your development time and roll-up data into higher-level report suites.

Wasting of Variables

This next one is a minor one but it is related to wasting variables. Even though there are more variables available now, it doesn’t mean that you should track everything or that every piece of data requires its own variable. For example, I recently ran into a client that was tracking Salutation (Mr., Mrs., Dr., etc…) in an eVar. This makes very little sense. How are you going to do cutting-edge analysis on that? Gender maybe, but I don’t think Salutation is worthwhile. Just because you know it, doesn’t mean you need to track it.

This leads to the other type of waste I see – not using SAINT Classifications to save variables. There are many cases where you can accomplish the same analysis objectives by using SAINT Classifications and save variables along the way. Using the prior example, instead of storing Salutation as an eVar, if you really need it, why not store a Customer ID value and then add Salutation as a classification value of that Customer ID? That saves you one eVar and if you happen to have Full Subrelations on that eVar, you get them on the classification of that eVar as well (which will be less of an advantage when using SiteCatalyst v15 since all eVars will have Full Subrelations).

But here is my favorite example since I see this all of the time! One of Omniture’s common JavaScript Plug-ins is the Time Parting plug-in. This allows you to see data segmented by Day of Week and Hour of Day. However, many clients also store an sProp and/or eVar for Weekday/Weekend through this plug-in. It makes sense that you might want to segment data by Weekday/Weekend, but why use an entirely new variable just to track the binary values of Weekday vs. Weekend? You can easily do a one-time classification of Day of Week and lump Mon-Fri into “Weekday” and Sat-Sun into “Weekend.” That will allow you to achieve the same goal, but saves a variable. Again, this is a minor annoyance, but it is the principle that counts. You can extrapolate this concept by thinking back to the Customer ID example I mentioned above. What if there were ten data points related to a customer that you chose to store in ten separate eVars? You might be able to make these classifications and save ten eVars!

My advice here is to just be thoughtful when assigning variables and if you have cases where there is a direct relationship between two variables that won’t change very often, consider using a SAINT classification and also think about whether you will ever use that data point for an analysis before tracking it in the first place.

VISTA Rule Chaos

The final pet peeve I will mention is related to VISTA Rules. Let me start by saying that VISTA and DB Vista rules are not bad. They can be very powerful, but it is also true that they can be easily misused and wreak havoc on a SiteCatalyst implementation. When using VISTA rules, it is critical that you and your entire team understand WHEN the rules are being used and WHAT they do in terms of setting variables. I have seen many cases where a developer will change a variable not knowing that there are VISTA rules impacting it. You need to make sure VISTA rules are heavily documented and as you change your site or implementation, they need to be factored into the equation. One suggestion I have is to add the phrase (SET VIA VISTA) in the name of any variable that is set via a VISTA rule in your documentation so there is no missing it!

The other pet peeve I have related to VISTA rules is when they are used as a “band-aid” to avoid doing real tagging. In the long-run, this always comes back to haunt you. I see many clients creating band-aids on top of band-aids until things fall apart. I am ok with companies using Vista rules to get things done quickly, but I recommend that, over time, you phase out as many VISTA rules as you can and move their logic to your regular tagging so you have all of your logic in one place.

Final Thoughts
Well, there you have it. Not all of my implementation pet peeves, but a bunch of them that popped into my head. I am sure you have seen some fun ones out there and I’d love to hear about them…Please leave them as comments here!

NOTE: For more details on these points, check out my follow-up post here.

Analytics Strategy

U.S. Privacy and Data Security Legislation Summary/Recap

Andy Kennemer, VP of Social Marketing & Media at Resource Interactive, recently attended the NRF Washington Leadership Conference, which included a meeting of the Shop.org Policy Advisory Group (PAG) meeting, of which he is a member. A major focus of the PAG meeting was the increased legislative focus on privacy and data security. Andy agreed to summarize some of the highlights for me to share here.

Legislation is cyclical, and we’re in a hot period right now.

The focus of our meeting was discussing in detail the various legislative actions in Congress regarding both online privacy and data security. These two issues are separate, but related, and the more they are mixed together in legislation, the more complicated and ambiguous it will make things for retailers and brands.

Last year we saw 3 main efforts:

  1. The FTC’s attempt to establish rule-making authority through a new US Privacy Framework proposal;
  2. Rep Boucher’s attempt to introduce an online privacy bill, which primarily would support the notion of consumer opt-IN to 3rd party tracking; and
  3. Sen. Pryor introduced a bill addressing Commerce Data Security.

The FTC is likely to release a final “staff report” on this matter sometime this year. The Boucher and Pryor bills never made it to the floor for debate.

This year, mainly in the last 3 months, we have seen a flurry of activity like never before.

Key privacy bills introduced this year:

  • Sens. Kerry & McCain introduce broad privacy bill (4/12/11)
  • Reps. Stearns & Matheson introduce broad privacy bill (4/13/11)
  • Sen. Rockefeller introduces “Do Not Track Online” bill (5/9/11)
  • Reps. Markey & Barton introduce “Do Not Track Kids” bill (5/13/11)
  • Sens. Franken & Blumenthal introduce Location Privacy bill (6/15/11)
  • Sen. Wyden & Rep. Chaffetz introduce “GPS” Privacy bill (6/15/11)

Key data security proposals:

  • White House releases Cyber Security proposal (5/25/11)
  • Sen. Leahy re-introduces Judiciary bill from 111th Congress (6/8/11)
  • Dept of Commerce “Green Paper” on Cyber Security framework (6/8/11)
  • Rep Bono Mack revises House-passed data security bill from 111th Congress (6/10/11)
  • Sen. Pryor re-introduces Commerce Bill from 111th Congress (6/15/11)

With so much activity, it’s challenging to even keep track of everything, and which bills and proposals matter the most. There are a few of these that are gaining momentum that, as an industry, we need to watch. The Kerry-McCain bill, White House Cyber Security proposal, and potential final report from the FTC on the privacy framework will have the broadest impact to brands and retailers.

For now, the feedback from a congressional staffer who attended the meeting was:

  • There is a fear of modernity within the government. What needs to be better articulated is how data collection is used to actually help consumers, to have a more relevant and enjoyable online experience.
  • We need the voice of actual consumers. Right now consumer advocacy groups have influence, but it isn’t clear if they really represent the concerns of average consumers.
  • Retailers have not adequately addressed the consequences of legislation, in terms of actual economic harm, or hindering innovation. Some sort of cost / benefit / risks analysis could be helpful (e.g., What does our online experience look like if advertising is not as effective? Are consumers ready to pay for services that are currently free and ad-supported?

This continues to be a complex and rapidly evolving area, and brands cannot afford to simply put their heads in the sand and hope it goes away. Legislation will get passed, but the extent and impact of that legislation is far from clear.

Social Media

TakeFive with TweetReach

TweetReach has started up a new interview series on their blog called TakeFive with TweetReach. The goal of the series is to “provide insight and commentary from notable members of the social media analytics and measurement community, with the goal of facilitating an ongoing conversation around all things measurement.” I’m not going to quibble with their loose interpretation of the term “notable.” I was happy to participate!

Some of my favorite quotes (from me…how egocentric is that) from the interview:

We use our measurement planning process to ensure we have alignment across the stakeholders involved in the campaign. For each tactic or channel, we try to make sure we’re all in agreement on the answers to two questions (we actually call these “the two magic questions”): 1) what is the tactic supposed to do? (these are the objectives for the tactic) and 2) how will we know if it did that? (these are the key performance indicators).

And:

Marketers tend to operate with a hefty level of cognitive dissonance: on the one hand, touting the importance of multi-channel marketing that has congruent and complementary messaging…and then asking, “What’s the value of a fan of my Facebook page?”

And, finally:

When I’m presented with a, “You must prove the ROI of our social media investment!” decree, I tend to redirect slightly and ask the requestor if what they really want to know is, “Did I efficiently and effectively invest in this effort and garner meaningful, quantifiable results from that investment?” If I can get agreement on that…

Brilliant stuff, I say! Check out the full interview on the TweetReach blog!

Adobe Analytics, General

SiteCatalyst Advanced Search Filters [SiteCatalyst]

One of the features that I find deceptively difficult at times in SiteCatalyst is the use of the Search feature. I feel like there are many times I use this and end up messing it up. Therefore, I decided to do my best to share what I have learned about what works and doesn’t work in the hopes that it will save you aggravation and time! I also hope that many you can add a comment to this post with your tips and tricks so we can all learn something…

The Basics

First, let’s start out with the basics. Hopefully if you are a SiteCatalyst user you know that the search function is used to filter results in eVar and sProp reports. You simply enter a value and SiteCatalyst will look for those values in the active report and return those rows. This is handy because you can bookmark reports, make custom reports or add reports to dashboards after you have created the filter so that you never have to apply it again.

For example, let’s start with a Pages report like this:

Obviously we have pages from all sorts of countries, but if we only wanted to look at pages from England, all we would have to do is enter “SFDC:uk:” in the search box (top-right) and we would then see a report like this:

But what if we wanted to see pages from England or France? At this point we have two options. You can either enter “SFDC:uk: OR SFDC:fr” in the search box or use the advanced search editor. Here is what it would look like with the OR statement in the regular search box (look at the top-right portion):

However, believe it or not, if you change the “OR” to be a lower case “or” you will get no results! I kid you not! I call that an “Omniture-ism” and you just have to remember it…

The other way to get to the same report is to use the Advanced Search tool. You get there by clicking on the Advanced link to the right of the search box. Once there, you would enter the appropriate phrase in the first box, click the “+” sign to add another search criteria and then enter the second phrase so it looks like this:

However, it is important that you change the top drop-down box from the default of “if all criteria are met” to “if any criteria are met” or you will get no results.

If you wanted to look for cases where there were pages on the UK website that had the phrase “form” in the pagename, that would be a case where you would use the “if all criteria are met” option and your query should look like this:

This would result in a report like this:

Finally, we can come full-circle and get more advanced and use an “AND” statement in the standard box to get the same result. Here is what the search box would look like:

Again, keep in mind that the “AND” is case-sensitive…

More Difficult Searches

So now that we have covered the basics, let’s get a bit more advanced. First, let’s keep going with our example and say that we need to find all pages in the UK or France that have the word “form” in them. This gets a bit tricky because we are mixing OR and AND statements. Using the Advanced Search query builder, here is how you would enter it:

Conversely, if for some reason we wanted to see any UK Pages that had the phrase “form” in them and all France pages (not sure why, but this is just an example), we would enter this:

Which would result in a report like this:

Note that in this case we had to change the drop-down box back to the “any criteria” option since we did the AND statement within one of the criteria (hey…I told you this was the difficult part!).

The trick here is to combine any OR and AND statements into each row since each of the individual search criteria have to be either an “AND” or “OR” clause.

On a separate note, in the advanced search area, you can change the drop-down which defaults to “Contains” to “Does Not Contain” so if, for example, you wanted to see all UK pages, but exclude those that had “login” in the name you would enter the following criteria:

Note that for this instance, we need the “all criteria are met” option…

Finally, just for fun I entered the following phrase in the “simple” search box…

…and miraculously it produced the same results!! I decided to stop here before I broke anything, but you can feel free to see how far you can push this!!

But wait…There’s more! I have been amazed by how few people I meet know this next one… Imagine that you are looking at an eVar report and you have broken it down by another eVar via Subrelations. Here is an example where I have taken the Site Locale eVar and broken it down by Internal Search Term:

Now, let’s say that you wanted to do a search filter to only see items that mention “Outlook.” The easy way to do this is to just enter the phrase “Outlook” in the search box and SiteCatalyst will show any rows that have that phrase. But what if you wanted to see the phrase “Outlook” in just United States or Japan? No matter what you put in the search box, you will not get the results you are looking for (i.e. outlook AND “united states” OR japan). Would you know how to do this? Most people I meet don’t. Here is how…

When you are using a Subrelation report, you have to keep in mind that SiteCatalyst is running two reports and it doesn’t know which report you want to filter on. Therefore, we need to tell SiteCatalyst which report we want the search term to be associated with. You can do this in the Advanced Search area. When you have a Subrelation report, and you click on the Advanced Search area, you will see a new option that allows you to select one of the two reports being subrelated like this:

Most people haven’t ever noticed this new option so now that we know it is there, all we have to do is select the right report and then enter the search term in the right report and we can get our results. For the example above, we would enter “Outlook” in the search box next to Internal Search Term and “United States OR Japan” in the search box next to Site Locale like this:

Now, since we have been a bit more specific, we can get a nice, clean report like this:

Just keep this handy feature in mind the next time you are trying to search in a Subrelations report and pulling your hair out because you can’t get the results you think you should!

Even More Difficult Stuff

Phew! If you’ve made it this far, you are really devoted to your craft. We’re almost there so hang on…

The next thing that is important to know is that you can use wildcards in your searches. To do this, you use the “*” symbol in the search query. For example, if we wanted to find any pages in the UK that has the phrase “landing” somewhere in the name, we could simply do a search like this:

The next thing to know is that Omniture can be a bit quirky when it comes to the [SPACE] separator in the search box. Let me illustrate. If I enter the phrase “home page” in the search box, here are the results I get:

This seems strange to me since none of these pages have a space in them. That would make you think that a [SPACE] is a valid separator and that this query is the same as “home OR page” right? But if I use that logic and enter this phrase “SFDC:uk: SFDC:fr:” which is really just two phrases separated by a space (just with a colon in the phrase), I get no results. I am sure there is a logical reason for this, but I am not sure what it is. Maybe if SiteCatalyst sees a “:” or a “|” it acts differently (maybe Jorgen can enlighten us on this)?

To be safe, I use the next feature – using quotes – whenever possible. My advice is that if you ever have phrases with spaces in them that you enclose them in quotes and stick to using OR statements. In the preceding example, if I change my “home page” query to be “home page” in quotes, I get the expected result which is no results. Another lesson to be learned here is that you should, whenever possible, avoid putting spaces in values that you think you will search upon. I do my best to remove all spaces from page names since that is the variable I search on the most!

Finally, you can use the “-” sign to remove things from search results. This produces the same effect as using the “Does Not Contain” feature in the advanced search area. As in the previous example, if I want to see all UK pages, but not ones that have the phrase “login”, I can enter the following in the search box:

To see UK pages that do have login in the name, you can also enter this phrase:

But when the results come back, it will mysteriously remove the “+” sign and just uses space as the separator producing the same results.

Final Thoughts…
So there you have it! Pretty much everything I know about using search and advanced search in SiteCatalyst. Do you have any additional tips or tricks? If so, leave a comment here…Thanks!

Adobe Analytics

My Latest SiteCatalyst Wishlist Items [SiteCatalyst]

A few weeks ago I was at the European Adobe (Omniture) Summit in the UK and had the pleasure of being in another one of Brett Error’s “what features are we missing” sessions. I find these sessions to be good and bad at the same time. The good part is that people are expressing what they need and others can validate or invalidate ideas in real-time. The bad part is that I often feel that the features that get voted up are the ones that are easy to understand (like Bounce Rate as a standard metric!), but that there are many features that people SHOULD want, but don’t know it yet. I don’t mean that to come out as sounding pretentious, but the fact is that many people have been using the product for only a few years and it is natural that the needs of those who have been using the product for many more years will have some more advanced feature requests. Unfortunately, many of these advanced features, no matter how important, will be trumped by more basic, globally understood feature requests.

The creation of the Ideas Exchange has been a great help in getting ideas big and small into the product and I am so pleased to see that many of the ideas in there have been added to the product and for that I commend Adobe (Omniture). I think the positive feedback around SiteCatalyst v15 is a direct result of people seeing their ideas manifested in the release.

In this post, I wanted to highlight a few ideas that are in the exchange that might not get as much “play” as they should and why I think they should be undertaken. If you agree and have a Login ID to SiteCatalyst, please feel free to login and vote for them!

SAINT Auto-Classifications
One of the ideas that came up in the UK session I mentioned earlier (and received the most votes!) was the notion of SAINT Auto-Classifications. This idea was submitted by Ben Gaines (probably as an initial test of the Idea Exchange!) the day the exchange came online. As most users know, SAINT Classifications are a way to add meta-data to values you have already captured in SiteCatalyst. It is similar to a pivot table in Microsoft Excel. However, SAINT Classifications have to be uploaded manually and it becomes very tedious over time. The feature request is to provide a way where administrators could set-up rules to auto-classify items or classify them on the fly (as reports open up). For example, if I have a report of campaign tracking codes and a bunch of them start with “seo|,” I could set something up where these would all be automatically classified as “SEO” in the Marketing Channel classification I have set-up. This is just one example, and the possibilities are endless.

The great news is that this idea has recently been changed to “Under Review” and geniuses like Sean Gubler have started playing around with tools to do this so I feel like it is only a matter of time before we see this. Keep your fingers crossed and vote for the idea by clicking here.

Multi-Session Attribution (Allocation)
The next idea is related to eVar attribution. Currently, you can attribute success to eVar values for First Touch, Last Touch. There is an option for Linear allocation, but that only works within one session so it is rarely used. The closest thing available for multi-session attribution is the Cross-Visit Participation plug-in which is really just a “hack” that concatenates eVar values into one string. This plug-in can be useful at times, but has some serious drawbacks.

In today’s world of people bouncing between websites and social media, you cannot count on the visit that people convert being the same one in which they came from a marketing campaign. Therefore, you often have cases where a visitor comes from an SEO keyword, does some product research, leaves the site, comes back the next day from a paid search ad, leaves the site and then comes back a third time just typing in the URL and then converts. This string of traffic sources is difficult to track and analyze using the eVar allocation feature set available today. What I feel is needed is a way to simply have SiteCatalyst extend its Linear Allocation feature to include multiple visits and make that a legitimate setting in the Admin Console. I’d even pay more for it if needed, since not everyone will need that level of sophistication. I personally think that attribution will become a bigger issue in the future as the current browser model fractures so I think this will be an important feature for all web analytics vendors going forward. You can read some of my partner Eric Peterson’s thoughts on appropriate attribution in this white paper. If you’d like to see SiteCatalyst go deeper with attribution, please vote for this idea by clicking here.

Multi-Session Pathing
Along the same lines, the next idea I’d like to suggest is the notion of multi-session Pathing. I suggested this to the Ideas Exchange over a year ago and was surprised to see that it only has 7 votes! Currently, pathing reports are limited to one session. However, it is often the case that visitors come to your website multiple times before they convert. Wouldn’t you want to see paths that span multiple visits for the same person? I realize that this can be data intensive, but even if it is for a subset of data, I think it would be interesting to pick a subset of visitors and see what they do over multiple visits. Currently, you can’t even do this in Discover. While I am not sure of the exact way the feature should be implemented, I feel that having some insight into multi-session pathing is important and should be somewhere on the roadmap. If you agree, you can vote for this idea by clicking here.

Expire eVars Based Upon Event or Time
The last feature request I’ll mention has to do with expiring eVars. Currently, you can expire an eVar based upon a time period (like Visit or 30 Days) or a Success Event but not both. So why is this important? Imagine that you have a situation where you have an eVar set to expire at the Purchase event. A person could come to your website from a specific campaign code and then not return for an entire year and then convert. In that scenario, the campaign code they came from a year ago would get credit for the conversion. However, there are cases in which you would not want that to happen so it would be great if it were possible to have SiteCatalyst expire the eVar at the Purchase event or after 30 days – whichever comes first. That would offer much more flexibility and tighten up eVar attribution across the board. Someone also added a comment to this idea with the idea of allowing an eVar to expire at Success Event X or Success Event Y. That would also be helpful. If you’d like to see this implemented, please click here to vote for it.

Final Thoughts
As I mentioned at the beginning of this post, there are some features that could have a big impact if added to SiteCatalyst, but they are ones that only those who have been through some big battles would know are needed. My hope is that you will think about these features and support them with your votes so we can all benefit. Thanks!

If you have any questions or want to learn more, feel free to contact me for more information.

Analytics Strategy, General

Three Great Jobs at Best Buy

Now that summer is upon us I suspect that some of my personal blogging activity will slow down but I wanted to call my reader’s attention to three great jobs that our good friends at Best Buy just posted:

  • Senior Analyst, Digital Analytics
  • Associate Manager, Digital Analytics
  • Manager, Digital Analytics

Those of you who were at Emetrics in San Francisco this Spring heard some of the story about the work we’ve been fortunate to help with at Best Buy. Those of you coming to Internet Retailer in San Diego on June 16th will get to hear a shortened version of the same story. If you can’t/didn’t make either event I am happy to put interested parties directly in touch with the hiring manager at Best Buy, email me directly for details.

If you are coming to Internet Retailer, come and hear Lynn Lanphier (Best Buy) and I tell their amazing story.

Adobe Analytics

5 Social Media Secrets – M.Tech 2011

The folks over at Thoughtlead have put together what they’re calling a Digital Influence Collaborative. It’s innovative and exciting and a new way to consume content in microbursts. If you haven’t gotten wind of these events yet, you’re missing out.

They typically feature 60 influencers on 60 topics in 60 seconds. Topics vary from Social Media to Enterprise Marketing Management.

Here’s a mashed version of the one I delivered for Mtech 2011:

5 Secrets for LEARNING from Social Media

Analytics Strategy

Web Analytics (How It Works) Explained in 4 Minutes

I was tinkering around a few weeks ago trying to figure out the best way to communicate an idea out to a group of people and hit on using Snagit to record me talking my way through a few PowerPoint slides that had some basic diagrams on them and then uploading the resulting video to YouTube (in that case, as a private video). It worked great — perfectly okay audio quality (I used a USB headset) and perfectly okay graphics. Lo-fi, but using the tools I already had at hand.

Below is an audio slideshow that uses the same approach to provide a very basic overview of how page tag-based web analytics tools work. If you’re a web analyst, I sincerely hope there is nothing new to you here. But, if you’re a web analyst who has repeatedly beaten your head against a brick wall when trying to explain to some marketers you work with that they need to put campaign tracking parameters on the links they use…maybe it’s a video you can send their way! It’s right at 4 minutes long, with a subtle-but-shameless suck-up to my favorite Irish web analyst at the 1:30 mark (it really never hurts to suck up to an Irish(wo)man, now, does it?).

The video is a much simplified overview of what I went into in greater detail in an earlier blog post.

If you’d like to download the slides (.pptx) for your own use (attribution appreciated but not required, and edit at will), you can do so here.

I’d love to hear what you think (of the format and/or of the content)!

Analytics Strategy

Webtrends Table Limits — Simply Explained

A co-worker ran into a classic Webtrends speed bump a couple of weeks ago. A new area of a client’s web site had rolled out a few days earlier…and Webtrends wasn’t showing any data for it on the Pages report. More perplexingly, there was traffic to the new content group that had been created along with the launch showing up in the Content Groups report. What was going on? I happened to walk by, and, although I haven’t done heavy Webtrends work in a few years, the miracle of cranial synapses meant that the issue jumped out pretty quickly (I can’t figure out how to say that without sounding egotistical; oh, well — it is what it is).

Heavy Webtrends users will recognize this as a classic symptom of “table limits reached.” There’s quite a bit written on the subject online…if you know where to look. The best post I found was You need to read this post about Table Limits by Rocky of the Webtrends Outsiders. The last sentence (well, sentence fragment, really) in the post is, “End of rant.” In other words, the post starts AND finishes strong, and the content in between is damn good, too.

What I found, though, was that it took a couple of conversations and a couple of whiteboard rounds to really explain to my colleague what was going on under the hood that was causing the issue in a way that he could really understand. That’s not a knock against him. Rather, it’s one of those things that makes perfect sense…once it makes sense. It’s like reading an analog clock or riding a bicycle (or, presumably, riding a RipStik…I wouldn’t know!).  So, I decided I’d take a crack at laying out a simplistic example in semi-graphical form as a supplement to the post above.

The Webtrends Report-Table Paradigm

First, it’s important to understand that every report in Webtrends has two tables associated with it:

  • Report Table — this is the table of data that gets displayed when you view a report
  • Analysis Table — the analysis table is identical in structure to the report table, but it has more rows, and it’s where the data really gets stored as it comes into the system

Webtrends aggregates data, meaning that it doesn’t store raw visitor-level, click-by-click data and then try to mine through a massive data volume any time someone runs a simple report. Rather, it simply increments counters in the analysis tables. That makes sense from a performance perspective, but can easily lead to a “hit the limits” issue.

Key: neither of these tables simply expands (adds rows) as needed. Both have their maximum row count configured in the admin console. Those limits can be adjusted…but that comes at a storage and a processing load price.

(Now, actually, there are multiple analysis tables for any single report — copies of the underlying table structure populated with data for a specific day, week, or month…but it’s beyond the scope of this post to go into detail there. Just tuck it away as another wrinkle to learn.)

In the rest of this post, I’m going to walk through an overly simplistic scenario of a series of visits to a fictitious site with unrealistically low table limits to illustrate what happens.

The Scenario

Let’s say we have a web site with a series of pages that we’ll call Page A, Page B, Page C,…Page Z. And, let’s say we have our Report Table limit for the Pages report set to “4” (in practice, it’s probably more like 5,000) and our Analysis Table limit set to “8” (in practice, it would be more like 20,000). That gives us a couple of empty tables that look something like this:

Now, we’re going to walk through a series of visits to the site and look at what gets put into the tables.

Visit 1

The first visitor to our site visits three pages in the following order: Page A –> Page B –> Page C  –> <Exit>.

The analysis table gets its first three rows loaded up in the order that the pages were visited, and each page gets a Visits value of 1. If we looked at the Pages report at that point, the Report Table would pull those top 3 values, and everything would look fine:

Visit 2

The next visitor comes to the site and visits 5 pages in the following order: Page B –> Page C –> Page D –> Page E –> Page F –> <Exit>

We’ve now had more unique pages visited than can be displayed in the report (because the report table limit is set to 4). But, that’s okay. After two visits to the site, our Analysis Table would still have a row or two to spare, and the Report Table could pull the top 4 pages from the Analysis Table and do a quick sort to display correctly, using the All Others row to lump in everything that didn’t make the top 4:

If you searched or queried for “Page F” at this point, you wouldn’t see it. It’s there in the Analysis Table, but you’re searching/querying off of the Report Table. That doesn’t mean Page F is lost, though. It just means it has less traffic (or is tied for last) with the last item that fit in the Report Table.

Visit 3

Sequence of pages: Page F –> Page G –> Page H –> Page B –> <Exit>

Following the same steps above and incrementing the values in our Analysis Table, and again looking at a report for the entire period, we see (bolded numbers in the Analysis Table are the ones that got created or incremented with this visit):

Look! Page F is now showing up in the Report Table! Can you see why? Because the Analysis Table has greater row limits, the Report Table can adjust and pick the top-visited pages.

Visit 4

Sequence of pages: Page F –> Page I –> Page J –> Page B –> <Exit>

Here’s where we really start to lose page-level granularity. Our Analysis Table is full, so there are no rows to store Page I and Page J. So, that will add 2 visits to the All Others row in the Analysis Table (while this is a single visit, this is the pages report, and each of those pages received a visit). Our tables now look like this:

Until the Analysis Table gets reset, no pages after Page H will ever appear in a report.

Even if Page I Becomes the Most Popular Page on My Site?

It’s time for a direct quote from the Webtrends Outsider post referenced at the beginning of this post:

Ugly example #1: Your end users contact you wanting to know about traffic to their expensive new microsite.  You know you’ve been collecting the data correctly because you triple-checked the tagging before and after launch.  So you open the Pages report and WebTrends tells you those pages don’t exist.  Those  expensive pages got no traffic at all, apparently.  Knowing how the CEO’s been obsessed with the new microsite, you call in sick indefinitely.

It doesn’t matter if Page I becomes the only page on your site. Until the tables reset, you won’t see the page in your Pages report — it will continue to be lumped into All Others.

And That Is Why…

If you started out on Google Analytics and then switched over to Webtrends you might have noticed something odd about the URLs being captured (I learned it going in the opposite direction): in Google Analytics, the full URLs for each page, including any query string parameters (campaign tracking parameters excluded) are reported by default. In Webtrends, query string parameters are dropped by default. In the case of Google Analytics, you can configure a profile to drop specific parameters, while, in Webtrends, you can configure the tool to include specific parameters.

Why does Webtrends exclude all parameters by default? The table limits is one of the reasons. If, for instance, your site search functionality passes the terms searched for and other details to the search engine using query parameters, the Analysis Table for the Pages report would fill up very quickly…with long tail searches that only received 1 or a small handful of requests.

What to Do?

The most important thing to do is to keep an eye on your table sizes and see which ones are getting close to hitting their limits. If they’re getting close, then consider adjusting your configuration to reduce “fluff” values going in. If that’s not an issue, then you need to bump up your table limits. That may slow down the time it takes for Webtrends to process your profiles, but it will keep you from unpleasant conversations with the business users you support!

Analytics Strategy, Conferences/Community

Amazing news from Analysis Exchange

UPDATED: We got great quotes from the Vice President of Human Resources who hired Jan Alden Cornish that clarify how Analysis Exchange is making a difference when it comes to hiring web analysts.  See below!

If you’ve worked in web analytics and digital measurement for long, or if you’ve ever tried to hire an experienced web analyst, you know that there are not enough qualified, experienced, and well-trained web analysts in the world. What’s more, for the majority of our sector’s development there was literally nowhere someone new could go to get the kind of hands-on education and experience that most hiring managers are looking for. Considered together the web analytics industry has been stuck in a “lose/lose” situation.

The training gap was the central problem we set out to solve in 2009 when we launched the Analysis Exchange. Our goal was to bring “student learners” together with experienced mentors to provide guided education and work to ensure that entry-level analysts were familiar with both the theory and practice of web analytics. Analysis Exchange was designed as a logical “next step” for people who had read books, followed blogs, or taken online training from great groups like the WAA via their University of British Columbia coursework.

What’s more, so that our students would learn to “tell a story with data and analysis” we opted to work with nonprofits from around the globe — a traditionally under-served group when it came to site analysis and insight generation. This turned out to be a great idea, and we are honored every week by a handful of organizations who are willing to help us create valuable training opportunities for our community.

I set a lofty goal for Analysis Exchange when I first announced the effort was open to everyone at the Emetrics Summit in San Jose last May — I wanted to help 1,000 nonprofits and create training opportunities for 500 students. Unfortunately we didn’t meet that goal … but we have made amazing strides, a few of which I’d like to share with you today:

  1. We have grown to over 1,250 members around the world, including 205 nonprofit groups and nearly 650 students. Following the Web Analytics Association we believe Analysis Exchange to be the single largest group of individuals interested in the subject of web analytics in the world — and we’re pretty excited about that!
  2. Our members have completed over 100 projects in the past year. What’s more, our students and mentors have earned awesome scores with an average “likelihood to recommend this mentor/student” score of 9.5 and an average rating for each member’s work of 9.4 (both out of 10.0)
  3. We won a prestigious award from the Web Analytics Association. Analysis Exchange was recognized as the “Most Influential Agency or Vendor” by the WAA at this year’s awards event.
  4. IQ Workforce has just agreed to help us grow and expand our efforts. Given our commitment to incubating new talent within the web analytics community this sponsorship makes great sense (read more about it here) and we’re delighted to have Corry Prohens and his team helping our mentors and students expand their horizons.
  5. We recently had our first student get a full-time job working in web analytics. This more than anything excites me … the fact that Analysis Exchange is working “as designed” for the web analytics community, helping individuals get the experience they need to bridge the gap between “knowledgable” and “employed.”

On this last point I wanted to share a little more detail. We have some pretty motivated mentors and students in the Analysis Exchange. One of our students is Jan Alden Cornish from Carmel, California. Jan has done three projects with us and in one case stepped in and helped out at the very last minute. He’s bright, articulate, and one of the nicest guys you’ll ever meet … so when he called and asked me to provide a reference for him on a job interview I was more than happy to help.

According to Jan:

“Completing three projects with the Analytics Exchange afforded me a rare opportunity to work side by side with seasoned practitioners. Each project had it’s own unique set of challenges. Nothing can replace hands on experience with real data and a need to solve real problems. Digital marketing doesn’t take in an organizational vacuum. These projects also provided me an understanding of organizational context in web analytics takes place.”

We also heard from the Vice President of Human Resources who hired Jan, Cynthia Nelson Holmsky:

“As a major e-commerce website we were recruiting for an E-Commerce Analyst and found an alumni of Analysis Exchange.  While the candidate had many years of business and software analytics, his only web experience was through Analysis Exchange.  However that Exchange experience provided just enough applied web analytics to win him the interview.  During this recruitment I met other candidates with strong business analysis backgrounds who lacked any web experience, and I referred all of them to Analysis Exchange as a great place to learn web analytics and expand their career potential.”

Cynthia clearly understands the challenges facing recruiters and HR specialists looking for web analytics talent (emphasis mine):

“Web analytics is still a young discipline.  Many individuals and businesses want to develop competencies in web analytics, but wonder “Where do you go to develop expertise?”  Many colleges and universities have yet to integrate web analytics into their curricula, or what they cover is not hands-on, so Analysis Exchange is meeting a key need in the marketplace for individuals who want real world experience, while at the same time building supply to meet the demand for web analysis talent in the tech job market.  Plus, the Exchange is meeting the needs of non-profit organizations that normally could not tap into this type of expertise.  Analysis Exchange is a  great idea, and a win-win-win model.

Hopefully Jan will continue to support the Analysis Exchange — as a mentor, now that he is working professionally in the field. I also hope those of you reading this post will consider joining Jan in the Analysis Exchange. Signing up takes less than a minute and there are plenty of projects looking for mentors and students available right now.

Excel Tips

Excel Dropdowns Done Right: Data Validation and Named Ranges

NOTE: There is an updated version of this post posted here. I recommend reading that one rather than this one.

Every once in a very rare while, I find myself not motivated to expound upon deep and meaningful subjects. So, this post is not about the latest turn in world of privacy legislation, it’s not about my deepening fascination with two-tiered segmentation, it’s not about the perplexing and depressing indefinite postponement of Demystified Days, and it’s not even about pondering when Team Evil Forces will have a web site.

Nope. Not today. This is just a good ol’, “Hey, let’s look at a handy capability of Excel…and how to use it to the best of its ability.”

This came up last week when a co-worker asked me: “How do I get dropdowns working in cells in Excel?” She knew she had done it before, but she couldn’t remember how. In the course of showing her, I realized that, therein, was one of those handy little tips worth sharing. I’m going to walk through three different ways to accomplish this:

  • The totally common, mundane way — straightforward, but it has limitations
  • The way I always do it — almost no more effort to implement than the first way…but with fewer limitations
  • The way I may start doing it (sometimes), which would make the approach just that much slicker

Bounce around as you see fit!

The Scenario

You’re using Excel to enter a table of data, where one or more of the columns have a standard set of possible values. For instance, let’s say you’ve made a list of household chores, and you use that list to both assign a priority to each task as well as to note the status of the work:

For both the Priority and the Status column, you’d like to enter the values using a dropdown menu, rather than needing to retype a value in each cell:

The wrinkle is that you expect this list to live for a while, and there’s a good chance that you may want to have other values available for either the Priority or the Status columns (or both). We’ll get to that.

The Standard Excel Way — Data Validation

The quickest way to set this up is with basic data validation:

  1. Highlight all of the cells that will use the same dropdown values
  2. Select Data » Data Tools » Data Validation
  3. Change the Allow dropdown to List
  4. Enter the values in the Source box (separating different values using commas)
  5. Click OK
  6. Repeat for each set of cells that has a unique set of dropdown value options.

That’s all there is to it, and it works.

The Limitation: Suppose that you decided you wanted to add a new value to the list of options, and that, rather than four cells right next to each other, this same data validation rule was used across numerous non-contiguous cells, even cells across multiple worksheets. Going in and updating the available list of values is a real pain. That brings us to…

My Standard Way — Data Validation with a Named Range

I regularly use dropdowns to make Excel-based reports more dynamic — enabling the user to choose whether he wants to see a weekly or a monthly version of the report, as well as to select the specific date range (this isn’t so much for the user’s benefit as it is for mine — it means I don’t make a “new report” each week or month, but, rather, update the data in the same workbook and then update the dropdown to get the current report; read more about my approach for that in this post).

I have a standard way of generating dropdowns that gets around the limitation described earlier: rather than entering the list of values directly in the data validation dialog box, I reference a named range. Using the same household chores scenario, I would accomplish the same end result, sans limitation, as follows:

  1. Add a new worksheet (I usually name it something like “Lookups” and then hide the worksheet once everything is set up so it’s never something that the user sees)
  2. Enter the lists of values at the top of that sheet — one list per column
  3. Select all of the values for one set of dropdown options and enter a name for that range (in this case, “List_Priority”)
  4. Repeat this  for the other list of values (I named it “List_Status” — I like to prepend the names of similar types of named ranges so that they group easily in the Named Ranges dialog box)
  5. Now, it’s the same basic process as described earlier, except, rather than entering the specific values in the data validation Source field, you enter a named range (note the “=” before the named range!):
  6. Click OK, and you’re good to go again!

Now, if you ever want to update values in the list, you can edit the values on the Lookups sheet. This won’t update the cell values that have already been populated — just the available values in the dropdown anywhere that named range is used.

The Limitation: even this approach has a limitation, but it has a couple of workarounds. Let’s say you decide to add a value to one of your lists — say you want to add “Unknown” as an option for Priority. If you simply type it at the bottom of the list, it falls outside of the named range and won’t be reflected in your dropdowns. Two different ways to work around this:

  • After adding the value, edit the named range (Formulas » Defined Names » Name Manager) to include the additional cell
  • Before adding the value, select the bottom value in the current list, right-click, and select Insert » Shift cells down » OK.This will have effectively expanded the named range by a cell. You can then either add the new value in the blank cell or copy and paste the “bottom” value (“Low” in this case) into the blank cell and then enter the new value into the bottom cell

Both of these approaches are a little bit clunky, so let’s add a twist to make the named ranges a bit more elegant…

Data Validation with Named Ranges with a Clever Twist

[Update: See the first comment below — from Julien. As he notes, the formula described here is a little messy, and he proposes a cleaner solution. I’m leaving my original approach here to provide a “multiple ways to skin a cat” demonstration…but I expect I’ll be using the approach described in the comment.]

This is simply a couple of additional steps beyond the steps described in the previous section to make the named ranges a little smarter:

  1. Select Formulas » Defined Names » Name Manager
  2. Select List_Priority and click Edit to see the current definition
  3. Replace the Refers to: formula with the following formula:

=OFFSET(Lookups!$A$2,0,0,COUNTA(Lookups!$A:$A)-1)

And, voila! You can now go nuts with adding and removing values from the Priority list and the dropdowns will have updated values with no additional effort!

To do the same for the List_Status named range, the formula you would use for the named range would be:

=OFFSET(Lookups!$B$2,0,0,COUNTA(Lookups!$B:$B)-1)

To break down the OFFSET formula usage (using List_Priority as the example):

  • Lookups!$A$2: start at cell $A$2, which is the first value in the list
  • 0: stay in that same row (so still at $A$2)
  • 0: stay in that same column (so, again, still at $A$2)
  • COUNTA(Lookups$A:$A)-1: count the number of cells in column A that have values and then subtract 1 (the heading cell: “Priority”); grab an area that is that tall, starting with the cell currently “selected” ($A$2)

By checking Excel’s documentation on the OFFSET function and fiddling around a little bit with the formula, you can see how it’s working pretty easily.

Is It Worth the Effort?

I always use the second option described in this post. You just never know when a hastily hacked together spreadsheet will get “legs” and start growing and expanding its footprint. Better to spend an extra 10 seconds to add flexibility and maintainability.

Will I use the third option? I might. We’ll see. It didn’t occur to me that I should even try until I showed my co-worker the second option…and then watched her immediately get tripped up trying to add a new value to the list. If I’m handing off a document where flexibility in the dropdown values is needed, I might just Google my way back to this post to see how it’s done!

 

Conferences/Community

Demystified Days has been postponed

Unfortunately Analytics Demystified has been threatened with costly litigation over our Demystified Days event series. Out of respect for our current partners, our sponsors, and the entire community we have decided to postpone theses events for the time being.  We are certainly disappointed by this situation, but we remain committed to:

Now that Analysis Exchange has real momentum, our hope was to take this effort to the next level and begin to make real investments in the nonprofits that honor our efforts to train future web analysts through their participation. Our goal for this coming Fall was to donate $10,000 each to six different nonprofit participants in the Analysis Exchange in San Francisco, Atlanta, and Boston. Sadly we have been prevented from making those donations.

Such is life.

At Analytics Demystified we truly do believe in the community — be it the Analysis Exchange, the free web analytics documentation and content we all produce, John’s participation in the Web Analytics Association, my founding of the Web Analytics Forums in 2004, or Adam’s contribution to the award-winning Beyond Web Analytics podcast series. While we regroup and refocus our efforts expect to see us supporting Emetrics, the Web Analytics Association, Analysis Exchange, Web Analytics Wednesday, Beyond Web Analytics, and any other event or organization that is sincere in their investment in the web analytics community.

General, Social Media

The Crowd Has Spoken: Gilligan It Is

(I’ll return to serious posts shortly!)

A couple of weeks ago, I asked for input as to my new profile picture on this blog and elsewhere across the socialmediaverse. The crowd has spoken, a $74 donation has been made to the Appalachian Trail Conservancy, and it looks like I’m now due to have photographic alignment with the blog name:

Part of the inspiration for this exercise was that I’ve had the experience before of knowing what someone looks like as I’ve gotten to know them digitally solely based on a single picture…and then been surprised in some way by their appearance when I actually meet them in person. This came up a couple of times at eMetrics in San Francisco. So, in addition to changing my standard profile picture, I’ve also added a collage of photos to my About page. The challenge there is that I’m an amateur photographer, so am more often behind the camera than in front of it. That made for slim pickin’s on the photo front, but there’s enough there that you can get a better sense o’ me, should you care to have that!

Analytics Strategy

Would you pay $100 per year for Google Analytics?

Back in February our newest partner Adam Greco waxed philosophical about Google offering a paid version of Google Analytics. He got a bunch of feedback and all-in-all the post raised some interesting questions about Google’s place in the web analytics marketplace. Now there is a new rumor — one far less substantiated than the so called “Enterprise” offering Adam discussed — but one with potentially more far reaching implications.

On a call today, we heard that Google may be considering charging everyone for the use of Google Analytics.

Everyone? Yep. Everyone.

The details were sparse and wholly unsubstantiated, but come from a source that we generally trust as reliable. And while we normally don’t deal with rumors here at Analytics Demystified, given Google’s footprint — conservatively estimated to be around 30,000 business sites around the world with perhaps an order (or two) more non-business sites being tracked today — the implications of this rumor are interesting for two reasons:

  1. If Google were to charge a fee similar to other of their offerings, say $100 per year, to use Google Analytics, these fees may produce millions of dollars in annual revenue (and profits) for Google and their investors. Outside projections for Google Analytics installations range into the millions, which, given a reasonable retention rate (say, 10%) would produce substantial revenue. Given the changes Google is going through right now on their management team it’s hard to say how important “revenue” is, especially when the bandwidth and data storage costs for Google Analytics are likely to be significant given estimated volumes and at a time when Google is being criticized over their increasing operating costs. Given the constant criticism over the years of Google’s inability to generate profits outside of their advertising business perhaps this sort of obvious revenue is suddenly appealing.
  2. If Google were to start forcing folks to pay, this might be a huge boon to the emerging “secondary” market of web and digital analytics vendors including Woopra, Chartbeat, Performable, Clicky, Kissmetrics, and a rapidly expanding set of web and mobile analytics vendors who largely charge tens to hundreds of dollars per month. Despite the odds given the footprint “traditional” web analytics vendors have within the Enterprise, combined with the hegemony Google has over entry-level businesses and small companies, in the last three years we have seen a surprising “second coming” of web analytics vendors gaining traction in a variety of niches. Be it real-time analytics for blogs (Chartbeat, Woopra), funnel analysis (Kissmetrics), heat-mapping and session recording (Robot Replay, ClickTale, Reinvigorate), or customer-focused analytics (Performable), these companies are, by-and-large small, agile, and somehow managing to gain adoption despite the presence of Google and “the bigs”.

This second implication is very interesting to me … the fact that against well-established, already deployed, and in Google’s case completely free competition, these start ups are able to grow and, at least in a few cases, thrive (see Clicktale, Robot Replay who was acquired by Foresee Results, Performable, etc.) Imagine the glee that founders and investors in these companies would experience if Google Analytics were to put up a real (albeit potentially small) barrier to entry. It likely wouldn’t be enough to stop companies, but it might be enough to make them think “Hmm, I wonder what else is out there?”

Given that most of these start-ups are focusing on ease-of-use and specific use cases, and in many instances are doing a pretty darn good job (my opinion), this pause might be exactly what these start-ups need. Heck, it might even help some of the bigs, given the trouble they have had selling against Google Analytics juxtaposed against the dramatic interface changes that some are poised to unleash. Don’t get me wrong — I’m not saying that Google charging $100 for analytics magically re-opens the door for traditional vendors with annual contracts in the tens of thousands of dollars. But every substantial change in the marketplace is an opportunity for great management teams, and Google suddenly charging anything would surely be a substantial change.

But I digress.

At the end of the day I personally consider it highly unlikely that Google would start to charge everyone just because they can — it just seems like an unnecessary and “evil” thing to do (despite the fact that we did the same thing earlier this year at Twitalyzer without any negative impact on our business.) Still, Google is held to a pretty high standard, and I suspect that the (relatively) small amount of revenue they would ultimately generate is hardly worth the negative press they would likely receive.

But I’m interested in what you folks think. Would you pay $100 per year for Google Analytics as it exists today? What if they offered more features or functionality? If the latter, what would they need to add to get you to pony up? Or would you immediately pull the code off your site if Google required any kind of payment? If so, why?

I welcome your comments and conversation.

Social Media

Tweeting on Schedule

I’ve been playing around a bit with scheduling my Tweets and thought that I’d share some of my findings with you. But first, I’ll riff a bit on the fragility of this nascent channel and Twitter’s amazing rise to prominence as the 3rd largest social network in this universe. The figure I’m using for scale is 145 million registered users, which came straight from the Twitter CEO, Evan Williams back in November, 2010. But, it wouldn’t surprise me one bit if another 55 million users joined in the past 5 months. That’s the number that’s being bandied about today.

With ad revenues estimated at $45 million and projections escalating at a 3x clip this year, Twitter is rocketing unequivocally skyward. The only problem with attaining massive growth with user populations rivaling the number of people residing in Brazil, is that Tweets are extremely perishable. If you aren’t watching, listening or searching for a Tweet, it’s highly likely that it will slip right past an entire country of users without ever being noticed. That’s a problem. It’s bad because it seriously erodes any value proposition of time or dollars invested in the channel. Thus, the argument for scheduling Tweets.

Researching Tweets

The best research I’m reading about Twitter is coming from Sysmos, where they continue to crank out valuable insights. Back in September, 2010, they found that the average lifespan of a Tweet is about an hour. Sysomos discovered that 92.4% of Retweets happen within one hour after publication and 96.9% of @replies occur within the first hour. This means if your Tweet isn’t circulated after 60 minutes, it’s likely a goner. Of course there are numerous tools that allow you to automate this process. And that’s what I’ve been exploring. Even the most pedestrian Twitter clients now allow you to schedule your 140 character missives for posting at a later time.

What are the drawbacks of scheduling Tweets?

Scheduling Tweets is a tenuous business. For the most part, you should be Tweeting to deliver good content, but also to initiate a dialogue with your followers. If you’re out on the golf course and your Tweets are generating a firestorm of activity, who’s going to respond? Be cognizant of this fact when scheduling Tweets, because if your Tweet gains velocity and lots of people hear it, you better be at the ready to engage. If not, you’ll quickly lose credence as a friendly human and instead come off looking like a bit of a bot yourself. For this reason alone, if you’re planning to schedule Tweets, do so with considered caution and release news or informative Tweets purely to gain exposure. You don’t want to provoke a dialogue when you’re not ready to interact.

Who offers Tweet scheduling?

This isn’t meant to be a full and comprehensive review of Tweet scheduling tools. These are just a few that I’ve used personally, and my observations of each. I look forward to hearing what you think about Tweet scheduling and which tools if any you use. I’ll commit to updating my list as you offer more…

Tweetdeck – Ahh…my first real Twitter client and a darn good one at that. It’s iconic black interface offers de facto functionality and does so with a fine polish. (I’ve tried to use the “light” interface but just can’t make the switch). Tweetdeck is lightning fast with Tweets posted in real-time. But more to the point, they allow users to schedule Tweets in the future by simply selecting the date and time of your desired launch.

Hootsuite – This little freemium gem is quickly becoming my go-to Twitter client. Despite their recent service outage (which wasn’t really their fault), It’s winning me over with the multi-tabbed interface, multi-user efficiency and slick stream views. Hootsuite allows users to pre-schedule Tweets as well, with the option to select the date and time and receive an email when your 140 character missive flies.

Crowdbooster – I gained access to this product only recently and have been intrigued since my first login. This beauty not only allows you to schedule Tweets, but also recommends the best times to give a shout out. I really like that they deliver an explanation of why specific times are best for Tweeting based on when my followers are active and when I’ve gained the greatest reach. Crowdbooster also has the best charting I’ve seen yet from a Tweet scheduling interface that reveals which Tweets attained reach…and RT’s and @replies as well. I’m having fun with this freemium tool and may even upgrade.

Timely.is – Here’s an interesting new app, that I learned about recently. It uses an algorithm to Tweet when your message is likely to reach the largest audience. Currently, they don’t provide any visibility into how they make this determination, but you can override it by forcing the Tweet to send within the next 30minutes. While they do offer a few cheesy “suggested” tweets, this tool is a product of Flowtown and I’ve been waiting to see what these guys bring out of their private beta. This is definitely one to watch.

Buffer – Buffer offers a slick user interface allowing users to schedule Tweets across a number of recommended times. It has links to the Bit.ly API, but requires premium access to utilize this function. Yet, the free version delivers solid capabilities and collaboration functions for adding additional team members. Perhaps the easiest function is the Chome browser extension that enables you to schedule a Tweet directly from a webpage. This makes scheduling convenient and will be helpful in getting to word out on those juicy bits you discover during non-peak times.

LaterBro – Yo, bro…I haven’t actually tried this one yet, but its interface is simple and clean. I trust it works just fine for planning ahead.

Since drafting this blog post has taken beyond my optimal Tweeting window, I’m signing off now. But before I do, here’s a few more Tweet schedulers that I haven’t tried yet. I’m sure there’s a whole lot more too.

What do you use for scheduling Tweets and what do you like about it? Curious minds want to know.

Analytics Strategy, Social Media

Privacy: It's a 2.5-Dimensional Issue

I’m keeping the voting open for another week or so on my “choose a new profile picture” poll, so if you haven’t voted yet, please click over and do so. There’s a charitable donation (by me!) involved!

“Privacy” is a hot topic in the world of marketing analytics, driven primarily by shifting consumer (and, in turn, regulatory) sentiment on the subject. That shifting sentiment, I think, is largely being driven by the increasing integration of social media into our lives and our online behavior.

The WAA stepped up and put together a Code of Ethics a few months ago, and privacy is going to be a recurring topic at eMetrics and other conferences for the foreseeable future. Following the San Francisco eMetrics conference, Stéphane Hamel put together three scenarios and asked the #measure community to vote as to the ethics and allowability of each situation. He then revealed the results and added his own thoughts. Towards the end of that second post, Stéphane noted that he was disappointed by the lack of interest in the exercise, given the generally accepted importance of the topic.

Emer Kirrane responded in the comments:

It’s interesting that there seems to be a correlation between legality and ethics in the minds of your respondents. To me, the Code of Ethics is there as a flag against practices that are deemed unethical by the community, rather than deemed unethical by law.

Stéphane’s concern and Emer’s response have been bouncing around in my brain for several weeks. My conclusion: “ethics vs. legality” is going to continue to give us fits.

I realize this isn’t the first time that “ethics” and “the law” haven’t perfectly aligned (they almost never do, actually, even though that, from a purist point of view, is the goal), but bear with me — it’s worth using that lens to explore the issue and outline the challenges we’re going to have to deal with. These are two very different dimensions of the privacy debate, and one of them is in flux on several fronts.

Why 2.5 Dimensions?

Obviously, there is a legal/regulatory dimension, and there is an ethical dimension. But, really, the legal/regulatory dimension is heavily driven and influenced by consumer perceptions and fears. I actually wrote some thoughts on that a couple of years ago. With high-profile Facebook snafus and high-profile media outlets reporting on cookies and cross-site tracking, politicians have found an issue that their constituents care about (or can be prodded to care about). So, in a sense, the legal/regulatory dimension has some added “oomph” of consumer concerns behind it; I’m calling that “consumer perspective” another half a dimension.

It’s possible that “consumer perception” should be a third dimension in and of itself. But, oh boy, that would make for some hairy sketching in the remainder of this post. I’m pretty sure I’m not just punting, though — the will of the consumer when it comes to something like privacy does generally get manifested through some form of government regulation.

Start with the Basics

Two dimensions: legal and ethical. We can look at them like this:

Various practices raise privacy questions. In theory, we can plot each of them on this (conceptual) grid — there are more than shown here, but I’m just laying out the basic idea of the framework:

In Theory, We’d Have Harmonious Dimensions

If life was simple, we would have perfect clarity for each dimension, and perfect alignment between dimensions:

Notice the shaded quadrants at top left and bottom right — there would be no practices that were ethical but not legal, nor would there be any practices that were legal but unethical.

Alas! Privacy is Rife with Gray Areas!

Reality is more like this — gray areas rather than hard lines along both dimensions:

Ugh. Things get messy. There are more activities that are questionable — they may or may not be legal and/or they may or may not be ethical! Argh!

But Wait! There’s More!

Ever since the web went mainstream, it’s been a more global medium than anything that came before. And, we’ve all run into cases and concerns that our standard web analytics implementation runs afoul of the law in some country somewhere. This grid illustrates that wrinkle, too — the legal/regulatory gray areas live in different places depending on the country (only the U.S. and the E.U. are shown here — it’s an illustrative diagram, people! Not a comprehensive one!):

And the big blue arrow shows where pressure is being applied (back to that half-dimension of consumer fears mentioned at the beginning of this post). It’s a little counterintuitive that the arrow is pointing upward, isn’t it? How could it be that things are trending towards “allowed?” They’re not. Rather, the “interpretation zone” is moving upward — practices that used to be “clearly allowed” aren’t inherently changing what they are, but those practices are moving from “in the clear” towards the gray area.

Helpful?

This was definitely one of those situations where, when I initially had a rough picture in my mind that would represent these two dimensions, it was simple and clear. It was only as I put pen to paper to sketch it out that it turned out to be tricky. Shortly after I finished writing this post (but, obviously, before I published it…as I’m adding this comment at the end), Jason Thompson made a really good case as to what is (misguidedly) driving the legal dimension out of alignment with the ethical perspective. That reminded me that I keep meaning to go back and re-read the last chapter (chapter 9?) of Jim Sterne’s Social Media Metrics book, as I recall that it was an intriguing non-sequitur that considered turning the entire “tracking” model on its head. Food for thought for another post, that.

What do you think? Is this an effective representation of the shifting privacy landscape we’re dealing with? What does it miss?

Social Media

U-Slurping Influence

I’ve noticed something recently that appears to be a burgeoning trend, and I don’t like it. Startups dangling the promise of exclusivity and early admission to their private beta parties in exchange for wielding your influence to “Spread the Word”. Pssst…”The more friends you invite, the sooner you’ll get access!” C’mon! If your product is good, people are going to use it and talk about it. Don’t patronize me with your bad Charlie Sheen references and generic html. This is lazy social media marketing in my opinion. And its a tactic that I won’t pander to.

However, it’s not nearly as bad as hitting submit on a digital form only to realize that the teeny-tiny checkbox in the bottom left hand corner, yeah…the one you didn’t UN-check?? Well, they opted you right into Tweeting to your entire following that you just signed up for the latest whatever on Twitter. These sneaky little broadcast methods are cheap trix and I say you marketers should be ashamed of yourselves.

I’ll keep this rant short, but influence is and will be a contributing factor in the success of many social marketing activities. Yet, as with all things social, leveraging influence must be genuine. Blatant solicitation of influence is only adding to the derision of influencer metrics and the narcissists who work to game the system. The real value of your influencers will pay dividends when they choose to talk about your products and services unprovoked. Doing it otherwise is a surefire way to usurp the power of the influencers you’re trying to enlist.

Social Media

It's Time for a Change, and I Need Your Help

If you have any regular interaction with me on this blog, Facebook, Twitter, LinkedIn, or scads of other social media sites, then you’re used to seeing my visage as such:

It’s time for a change, and I’d like a little wisdom of the crowds to drive it. Two quick background notes.

Why the Jester Hat in the First Place?

I started using social media heavily when I started working at Bulldog Solutions. I didn’t have much in the way of digital/digitized pictures of myself. The one above was handy (my wife and our three kids made these hats for all of us for a Bulldog social event), and, before I knew it, I’d signed up for a half-dozen services and dropped the image in as my profile picture. Since then, I’ve met numerous people for the first time who have asked, “Where’s the jester hat?” (including being asked by Rudi Shumpert on the Live at eMetrics edition of the Beyond Web Analytics podcast). Who knew? I had apparently done a moderately successful job of personal branding!

Where’d ‘Gilligan’ Come From, Anyway?

In 1993, I hiked the Appalachian Trail from Georgia to Maine. It’s a tradition on the trail to adopt a “trail name” for the duration of the hike. Partly due to my lanky frame, partly due to the fact that I had a tendency to bang my head on the low beams in many of the shelters along the trail, and largely because I wore a Tilley hat, I was dubbed “Gilligan” one evening by several other hikers who were staying at the same shelter that night.

I started this blog on something of a lark, and I knew that “Tim Wilson” was entirely too common of a name to base the blog on, as I’ve written about before.

Cast Your Vote to Contribute to a Worthy Cause

I’ve come up with four options for my new standard profile picture, and I want you to help me choose the one I go with. As a moderate incentive, for every vote cast, my wife and I will contribute $1 to the Appalachian Trail Conservancy (up to $250). And, I’ll wear the chosen hat in public at the next major geek conference I attend (so spread the #measure word, people!).

A – The Hat That Started It All
It’s the same hat I wore for a 2,100-mile hike — and it still sees occasional use.
 
B – A New Jester Hat
Sticking with the jester theme, but with a new hat and a new picture
 
C – As Gilligan As I Can Be
 
D – If You’re Simply Opposed to Change

The voting is wrapped up! You can find the results in this post.

Thanks!

Analytics Strategy, Conferences/Community

WAA Elections: I Support the Slate

While the voting period is mostly over I wanted to drop a quick note and offer up some thoughts on the candidates and process for the current Web Analytics Association elections. This year is clearly different thanks to a new process, one that has the membership voting on both a “slate” of candidates and two “at large” positions. While initially I didn’t understand the need to change the process, upon further explanation and a little reflection, I believe the new process makes sense and has the best interests of the Association and it’s membership at heart.

Before you go and Tweet “he’s lost his mind …” hear me out.

As the Web Analytics Association has grown the few board positions have become less of an obligation and more of an opportunity for people. In that, in recent years, we have seen an almost staggering number of people nominated into the election process. This, in my opinion, has created a problem in that A) most of the candidates, despite qualification, are relatively unknown to the web analytics community and B) because of the relatively low number of voters, a “popular vote” has become relatively easily gamed. I have certainly thrown my weight behind individual candidates in the past and, because my blog has tens of thousands of readers worldwide (many of whom do vote in WAA elections), I believe I have been able to help folks get elected.

Yeah for us and our friends, but boo for the process in general.

The popular vote has led to some truly great people participating in the WAA — folks (and my bias here) like John Lovett, June Dershewitz, Matt Langie, Dennis Mortensen, Ed Wu, and Peter Sanborn. But the popular vote has also led to some less-than-stellar participants in my humble opinion — people who either quit the board mid-stream or who served more as obstructionists than participants.

This new process, with what I believe to be a pretty well vetted board “slate” and list of “at large” candidates, has tremendous potential to do one very important thing: allow the Association to maintain the momentum they have today. From where I sit, in the past year the Association has:

  • Hired a very qualified Executive Director in Mike Levin
  • Started a very successful “local” event in the Symposium
  • Launched a very important community initiative with the Code of Ethics
  • Held a wonderful recognition event in the Emetrics/WAA Gala

and more. Plus, while I am not privy to any greater level of detail than anyone else, my general sense is that the current board is more productive and more collegial than many (or any) past boards and that bodes well for all of us.

So when it comes to the current election cycle, the “slate” has three returning Board members in Peter Sanborn (currently the Board President), Ed Wu, and Alex Yoder plus two new members who are, in my opinion, tremendously qualified to serve in Jodi McDermott and Shari Cleary. I have faith in Peter, Ed, and Alex based on their past work, Jodi has been a passionate contributor to WAA Standards and a number of other initiatives, and Shari is one of the most intelligent, level-headed people I know in life, much less web analytics.

The “at large” positions do create some problems, to be sure. The proposed group was whittled down from a larger group of folks, several of whom were qualified, passionate, and motivated, but my understanding is that the “secret selection committee” (which I offered to help with but asked too late) made decisions based on demonstrated commitment, involvement, and a willingness to work within the processes the Association has already established for the benefit of the membership. This strategy ends up recognizing folks like Chris Berry, a huge supporter of Research and Standards, Eric Feinberg and Lee Isensee, the “Laurel and Hardy” of the WAA and critical members of the membership committee, and Bob Page and Joe Megibow, two individuals who represent the level of leadership in web analytics that many (if not all) of us aspire to. In short, a brilliant group.

This list leaves off some pretty nice people as well, and this I think is what is creating some of the recent consternation in Twitter, but from where I sit the opportunity is clear: Participate in the WAA at the level that Chris, Eric, and Lee have, or build the reputation that Bob and Joe have, and you’re a shoe-in for the “at large” slots in the future.

For the record I am voting for Joe Megibow and Bob Page for the “at large” positions. Both are brilliant, both are passionate about measurement, and both serve as an excellent example of the kind of work we should all be doing. The Association needs more practitioners to represent the real needs of our industry and I cannot  think of two better people to fill that role.

Anyway, for what it’s worth, I too was confused about the “slate” process and this election, but hopefully like me you are willing to give the process a chance.

I welcome your comments.

 

Analysis, Analytics Strategy, Reporting

In Defense of "Web Reporting"

Avinash’s last post attempted to describe The Difference Between Web Reporting and Web Analysis. While I have some quibbles with the core content of the post — the difference between reporting and analysis — I take real issue with the general tone that “reporting = non-value-add data puking.”

I’ve always felt that “web analytics” is a poor label for what most of us who spend a significant amount of our time with web behavioral data do day in and day out. I see three different types of information-providing:

  • Reporting — recurring delivery of the same set of metrics as a critical tool for performance monitoring and performance management
  • Analysis —  hypothesis-driven ad hoc assessment geared towards answering a business question or solving a business problem (testing and optimization falls into this bucket as well)
  • Analytics — the development and application of predictive models in the support of forecasting and planning

My dander gets raised when anyone claims or implies that our goal should be to spend all of our time and effort in only one of these areas.

Reporting <> (Necessarily) Data Puking

I’ll be the first person to decry reporting squirrel-age. I expect to go to my grave in a world where there is still all too much pulling and puking of reams of data. But (or, really, BUT, as this is a biggie), a wise and extremely good-looking man once wrote:

If you don’t have a useful performance measurement report, you have stacked the deck against yourself when it comes to delivering useful analyses.

It bears repeating, and it bears repeating that dashboards are one of the most effective means of reporting. Dashboards done well (and none of the web analytics vendors provide dashboards well enough to use their tools as the dashboarding tool) meet a handful of dos and don’ts:

  • They DO provide an at-a-glance view of the status and trending of key indicators of performance (the so-called “Oh, shit!” metrics)
  • They DO provide that information in the context of overarching business objectives
  • They DO provide some minimal level of contextual data/information as warranted
  • They DON’T exceed a single page (single eyescan) of information
  • They DON’T require the person looking at them to “think” in order to interpret them (no mental math required, no difficult assessment of the areas of circles)
  • They DON’T try to provide “insight” with every updated instance of the dashboard

The last item in this list uses the “i” word (“insight”) and can launch a heated debate. But, it’s true: if you’re looking for your daily, weekly, monthly, or real-time-on-demand dashboard to deliver deep and meaningful insights every time someone looks at it, then either:

  • You’re not clear on the purpose of a dashboard, OR
  • You count, “everything is working as expected” to be a deep insight

Below is a perfectly fine (I’ll pick one nit after the picture) dashboard example. It’s for a microsite whose primary purpose is to drive registrations to an annual user conference for a major manufacturer. It is produced weekly, and it is produced in Excel, using data from Sitecatalyst, Twitalyzer, and Facebook. Is this a case of, as Avinash put it, us being paid “an extra $15 an hour to dump the data into Excel and add a color to the table header?” Well, maybe. But, by using a clunky Sitecatalyst dashboard and a quick glance at Twitalyzer and Facebook, the weekly effort to compile this is: 15 minutes. Is it worth $3.75 per week to get this? The client has said, “Absolutely!”

I said I would pick one nit, and I will. The example above does not do a good job of really calling out the key performance indicators (KPIs). It does, however, focus on the information that matters — how much traffic is coming to the site, how many registrations for the event are occurring, and what the fallout looks like in the registration process. Okay…one more nit — there is no segmentation of the traffic going on here. I’ll accept a slap on the wrist from Avinash or Gary Angel for that — at a minimum, segmenting by new vs. returning visitors would make sense, but that data wasn’t available from the tools and implementation at hand.

An Aside About On-Dashboard Text

I find myself engaged in regular debates as to whether our dashboards should include descriptive text. The “for” argument goes much like Avinash’s implication that “no text” = “limited value.” The main beef I have with any sort of standardized report or dashboard including a text block is that, when baked into a design, it assumes that there is the same basic word count of content to say each time the report is delivered. That isn’t my experience. In some cases, there may be quite a bit of key callouts for a given report…and the text area isn’t large enough to fit it all in. In other cases, in a performance monitoring context, there might not be much to say at all, other than, “All systems are functioning fine.” Invariably, when the latter occurs, in an attempt to fill the space, the analyst is forced to simply describe the information already effectively presented graphically. This doesn’t add value.

If a text-based description is warranted, it can be included as companion material. <forinstance> “Below is this week’s dashboard. If you take a look at it, you will, as I did, say, ‘Oh, shit! we have a problem!’ I am looking into the [apparent calamitous drop] in [KPI] and will provide an update within the next few hours. If you have any hypotheses as to what might be the root cause of [apparent calamitous drop], please let me know” </forinstance> This does two things:

  1. Enables the report to be delivered on a consistent schedule
  2. Engages the recipients in any potential trouble spots the (well-formed) dashboard highlights, and leverages their expertise in understanding the root cause

Which…gets us to…

Analysis

Analysis, by [my] definition, cannot be something that is scheduled/recurring/repeating. Analysis is hypothesis-driven:

  • The dashboard showed an unexpected change in KPIs. “Oh, shit!” occurred, and some root cause work is in order
  • A business question is asked: “How can we drive more Y?” Hypotheses ensue

If you are repeating the same analysis…you’re doing something wrong. By its very nature, analysis is ad hoc and varied from one analysis to another.

When it comes to the delivery of analysis results, the medium and format can vary. But, I try to stick with two key concepts — both of which are violated multiple times over in every example included in Avinash’s post:

  • The principles of effective data visualization (maximize the data-pixel ratio, minimize the use of a rainbow palette, use the best visualization to support the information you’re trying to convey, ensure “the point” really pops, avoid pie charts at all costs, …) still need to be applied
  • Guy Kawasaki’s 10-20-30 rule is widely referenced for a reason — violate it if needed, but do so with extreme bias (aka, slideuments are evil)

While I am extremely wordy on this blog, and my emails sometimes tend in a similar direction, my analyses are not. When it comes to presenting analyses, analysts are well-served to learn from the likes of Garr Reynolds and Nancy Duarte when it comes to how to communicate effectively. It’s sooooo easy to get caught up in our own brilliant writing that we believe that every word we write is being consumed with equal care (you’re on your third reading of this brilliant blog post, are you not? No doubt trying to figure which paragraph most deserves to be immortalized as a tattoo on your forearm, right? You’re not? What?!!!). “Dumb it down” sounds like an insult to the audience, and it’s not. Whittle, hone, remove, repeat. We’re not talking hours and hours of iterations. We’re talking about simplifying the message and breaking it up into bite-sized, consumable, repeatable (to others)  chunks of actionable information.

Analysis Isn’t Reporting

Analysis and reporting are unquestionably two very differing things, but I don’t know that I agree with assertions that analysis requires an entirely different skillset from reporting. Meaningful reporting requires a different mindset and skillset from data puking, for sure. And, reporting and analysis are two different things, but you can’t be successful with the latter without being successful with the former.

Effective reporting requires a laser focus on business needs and business context, and the ability to crisply and effectively determine how to measure and monitor progress towards business objectives. In and of itself, that requires some creativity — there are seldom available metrics that are perfectly and directly aligned with a business objective.

Effective analysis requires creativity as well — developing reasonable hypotheses and approaches for testing them.

Both reporting and analysis require business knowledge, a clear understanding of the objectives for the site/project/campaign/initiative, a better-than-solid understanding of the underlying data being used (and its myriad caveats), and effective presentation of information. These skills make up the core of a good analyst…who will do some reporting and some analysis.

What About Analytics?

I’m a fan of analytics…but see it as pretty far along the data maturity continuum. It’s easy to poo-poo reporting by pointing out that it is “all about looking backwards” or “looking at where you’ve been.” But, hey, those who don’t learn from the past are condemned to repeat it, no? And, “How did that work?” or “How is that working?” are totally normal, human, helpful questions. For instance, say we did a project for a client that, when it came to the results of the campaign from the client’s perspective, was a fantastic success! But, when it came to what it cost us to deliver the campaign, the results were abysmal. Without an appropriate look backwards, we very well might do another project the same way — good for the client, perhaps, but not for us.

In general, I avoid using the term “analytics” in my day-to-day communication. The reason is pretty simple — it’s not something I do in my daily job, and I don’t want to put on airs by applying a fancy word to good, solid reporting and analysis. At a WAW once, I actually heard someone say that they did predictive modeling. When pressed (not by me), it turned out that, to this person, that meant, “putting a trendline on historical data.” That’s not exactly congruent with my use of the term analytics.

Your Thoughts?

Is this a fair breakdown of the work? I scanned through the comments on Avinash’s post as of this writing, and I’m feeling as though I am a bit more contrarian than I would have expected.

Social Media

One Digital Analyst’s Guide to Using Twitter

Did you come to this post via a #measure link on Twitter? If so, then fair warning: your arrival probably has more opportunity to benefit me than it does to benefit you – I’d love to get some tips in the comments section that help me evolve my own process!

Guy Kawasaki spoke at the San Francisco eMetrics this year, and one of his early statements that most people in the room seemed to agree with was, “If the first time you saw or used Twitter, you didn’t think, ‘This is the dumbest thing I’ve ever seen,’ you probably aren’t that bright.”

Anyone who actively uses Twitter has struggled to articulate how and why it brings value to their lives when discussing it with a non-user. Consistently, those of us who are active users wind up falling back on, “You really have to get in and try it out and stick with it for a couple of weeks before it will start making sense.”

This post is my attempt to provide a guide/process specifically for analysts who fall into that “skeptical non-user” camp to try to make that “try it out for a couple of weeks” as smooth and worthwhile as possible.

A second Guy Kawasaki reference: he wrote a post once where he articulated another absolute truism:

There is no right and wrong with Twitter. There’s only what works for you and what doesn’t, so telling people how to use Twitter is as laughable as telling people what kind of websites were acceptable in 1980.

I’m walking a fine line with this post then, am I not? What I’m laying out here has two critical caveats:

  • It’s how I use Twitter – what I use it for and the tools I employ to use it effectively
  • It’s how I use Twitter as of April 2011 – as the medium continues to evolve and shift, and as I pick up tips and tools from others (I’m hoping to get some such tips from comments to this post), my process evolves

I (like many others) disagree with a lot that Kawasaki has to say about Twitter, but I agree that there is no single “right” way to use it, and this post shouldn’t be taken as such. It’s how I use it — if you find a thought or two that you think would be useful, use it. If you find a thought or two that you think is inane, then don’t (but I’d like a comment on this post so I can evolve my own approach).

Let’s dive in, shall we?

What I Get Out of Twitter

As an analyst, I get a range of benefits from my use of Twitter. Trying to organize them into a list makes it seem like they fall into discrete buckets, when, in reality, they’re a bunch of fuzzy overlapping circles, but here goes, anyway:

  • Breaking news in the industry – product launches, acquisitions, hot topics
  • Useful thinking from members of the industry – blog posts with tips/tools/explanations/philosophies
  • Relationship building – tweets can lead to emails, phone calls, and in-person meetings with both recognized industry leaders as well as analysts grappling with similar issues to me, and two minds are better than one almost all the time! I’ve also had relationships with vendors seeded through Twitter – several that have led to very real and very valuable partnerships
  • Technical support…from the community – I regularly tap into the Twitterverse to confirm oddities (the disappearance of Google Analytics Benchmarking, the fact that GA shows Safari as the top mobile browser for Android devices, etc.), which is often a quick way to confirm that I’m not missing something obvious
  • Technical support…from vendors – many vendors have a formal customer service presence (username) on Twitter or monitor references to their products and will respond promptly. I’ve gotten quick and helpful responses with very targeted queries to @OmnitureCare, @OmnitureFC, @Twitalyzer, and others

Even without all of these benefits, I would still value Twitter as a useful tool in my analyst workbelt. So, let’s get onto the actual process I use with Twitter.

But First! A Critical Understanding

If you’re new to Twitter, there is one key, key thing you absolutely must understand:

98%* of the content on Twitter that you could see and that might be of interest to you…you will miss…and that’s okay.

It’s easy to get sucked into your Twitter streams and find one useful link or reference after another. You then jump into some other work for a few hours (or days), and a little voice in the back of your head starts saying, “You’re missing valuable content!”

You are…and you aren’t. There is simply too much information out there to consume it all. Don’t try. Think about the information you get out of Twitter in terms of incremental bonus information on top of your existing resources rather than an entire body of information that you should be consuming as much of as possible, and you will be much more able to go to bed at night and rest peacefully. As it happens, my own Twitter usage has been pretty light for the past couple of weeks due to travel. It’s not keeping me up at night!

NOW…to the Brass Tacks of Using Twitter

There are three aspects (plus an optional bonus) to using Twitter:

  • Twitter tools (clients)
  • Filtering and categorizing content
  • Contributing and engaging
  • (Optional) Measurement and analysis

Twitter Tools

I use Hootsuite. It’s web-based, has all the functionality I need (the key one being the ability to show multiple “streams” of content at once), and has a mobile app that works fairly well. And, it’s got a nice bookmarklet (“hootlet”) that is a persistent button in my preferred browser, Google Chrome, so I can quickly tweet any link I find.

My setup at work and at home is to use an external monitor with my laptop. I use the monitor as my primary workspace, and my laptop off to the side with a browser maximized with Hootsuite running in it at all times. That way, I engage with the various streams I set up (discussed below), simply by glancing off to the side – where my laptop sits.

There are other clients, for sure. You can look at the tweets of people you follow (or ones you don’t follow) to see what clients they use.

‘nuf said. I’d be surprised if I was still using Hootsuite a year from now, but maybe not terribly surprised.

Filtering and Categorizing Content

One of the great things about the evolution of Twitter is that there is little harm in having a high count of people you are following. You may occasionally scan the timeline that intermingles all of their tweets, but, in practice, that’s going to be an unmanageable sea of information. I have the following “streams” that I set up to drastically filter and organize the content:

  • Replies – I have a stream for people who reference me in a tweet; this is one of the more important ones, because anyone who says something to me or about me can reasonably expect a timely response or acknowledgement
  • #measure hashtag – this is a search of “#measure,” basically, and it’s the widely adopted convention that web analysts (or, really, digital marketing analysts) use for tweets relevant to the field
  • Other searches – during eMetrics, I also followed the #emetrics hashtag; if I were working exclusively with a single web analytics platform, I might follow a search for that tool’s name or the hashtag…but I don’t do that currently (and there are a finite number of streams that I can reasonably follow at once)
  • Lists – I have several lists of people I follow; most notably, a “web analytics” list that I add people to when I see them tweet something of interest in #measure, a “Resource Interactive” list that contains current (and former) co-workers of the agency where I work, and even a private “client” list where I add employees of clients with whom I’ve had some interaction

I have a stream for Twitter direct messages…but I seldom look at it. I have DMs set up to send me a text message so that I’ll be more likely to get them even if I am away from my computer.

Contributing and Engaging

Some people use Twitter solely as a one-way communication vehicle – so-called “lurkers.” They use a combination of the techniques above, but they seldom actually tweet anything themselves. This really cuts down on the overall value that can be gained from the medium.

My personal strategy for contributing/engaging goes something like this:

  • Using the various streams described above, I retweet information that I genuinely find valuable and try to include a few words as to what struck me about the tweet or link it referenced
  • I get emails with interesting content – often people send me a note because they found content that they thought might be of interest to me, and, when I check it out, I feel like it’s actually content that might be of interest to the larger #measure community, so I tweet it (and credit the person who emailed it to me, if they’re on Twitter and I know their username).
  • I reply to people when I’ve got something to contribute – either a humorous response that might make them chuckle, a helpful link that I remember/can track down, or actual information that answers a question they’re asking or furthers a conversation
  • I have a list of “Measurement and Analytics” bloggers that I’ve built a feed for in Google Reader. This is my own version of something Stéphane Hamel (@immeria) set up years ago, and I used to use Yahoo! Pipes for, but which I recently cut over to Google Reader. I regularly add additional blogs to this feed, and I go beyond the pure “measurement” blogs that I find – pulling in both some data visualization and presentation tips blogs as well. This is an information resource for me in its own right, but, I start my day by scanning the new entries in that feed, and, if I see anything that might be of interest to the #measure community or to a specific person…I tweet it.

None of these are time-consuming activities for me. They’re either 5-10 seconds tacked on to whatever I’m already doing (to share the content), or they’re micro-interruptions throughout the day when I need to glance away from whatever work is currently at hand for a quick mental break.

(Optional) Measurement and Analysis

I wonder if it might be a bit controversial to say that measurement is optional. But, I don’t measure my e-mail use or my phone use, and, in many respects, all Twitter is is another channel along those lines.

Having said that, Twitter is also a key tool I use to build and evolve my personal brand. I use it to promote new blog posts I’ve written (this one, for instance, was auto-tweeted when it was published), as well as, I hope, to elevate awareness of who I am and what types of expertise I haven, and even some degree of my personality.

So, for me, it is important for me to measure whether my contributions and use of the platform are having a positive impact on @tgwilson as a Twitter presence. I use Twitalyzer for this…and you can read about how (in near-excruciating detail) in a post from a few months back.

Some Closing Thoughts

In the end, I try to hook into a few different dimensions of my social graph and engage with each of those dimensions. At times – fairly often, actually – I find content from one dimension of my social graph (say, my non-analyst co-workers) and port it over to share with the #measure community.

Both Twitter and various Twitter tools will continue to evolve. In a medium that is inherently micro and choppy, relatively small nuisances (being limited to 4 streams showing concurrently on my laptop screen, for instance) can really start to grate on my nerves over time. But, time and again, both Twitter and Twitter tool vendors continue to innovate and improve the user experience.

Looking back over the past 3 years, I realize that I’m now consuming and engaging much more, garnering more value, and spending the same or less actual time in the medium than I did when I started out. That’s partly from improvements in the tools, partly from the organic evolution of my personal process, and partly from…practice.

What Do You Think?

Chime in! What else do you do to efficiently leverage Twitter as an analyst? What frustrates you or is holding you back?

 

*Completely unsubstantiated, mostly defensible estimate.

Conferences/Community, General, Social Media

Announcing "Demystified Days"

UPDATE MAY 6, 2011: Under threat of litigation we have decided to postpone Demystified Days for the time being. You can read more about this decision here.

I am incredibly excited to let all of you know about something that Adam, John, and our friends at Keystone Solutions will be doing this coming September that builds on our long-standing commitment to local web analytics communities and our more recent efforts to support nonprofits around the world … something we are calling “Demystified Days!”

Check out the mini-site for Demystified Days right now!

For years we have been helping local web analytics communities around the globe connect with each other as part of Web Analytics Wednesday, and by every measure, Web Analytics Wednesday works. Thanks to current and past sponsors — great companies like I.Q. Workforce, Coremetrics (an IBM Company), SiteSpect, and hundreds of other companies who have hosted regional events — Analytics Demystified has brokered more personal introductions (and served more beers) than any other organization or group in our industry.

This past year we have been trying to leverage our connections in the industry to do something truly good and solve bigger problems. The result was, of course, the Analysis Exchange — the world’s only effort to provide free analytics support to nonprofits and nongovernmental organizations — which thanks to the efforts of great people like Wendy Greco, Emer Kirrane, Jason Thompson and our mentors and students has changed how people learn how to tells stories with data.

Now we are taking it to the next level, one city at a time.

Starting September 12th in San Francisco we will be bringing a day long educational and networking event to cities across the globe.  The format will be one you are all familiar with — great presentations in the morning and great conversations in the afternoon, of course followed by drinks and networking at Web Analytics Wednesdays in the evening.

We could easily do these events for free … but we aren’t going to. Instead we are going to find awesome sponsors to help us offset costs and ask everyone who participates to buy a $99 ticket to the event. Then, at the end of the day, we are going to add up all of the revenues, subtract out all of the costs, and donate every penny that is left to two local charities decided on by the event participants.

Our hope is to be able to donate a total of $50,000 to six charities in the United States. You can help us achieve that goal by doing three very easy things:

  1. Helping us spread the word about Demystified Days within your social network. We have created a short URL http://bit.ly/demystifieddays and you can tag tweets about these events with #demystifieddays.
  2. Joining us in San Francisco, Atlanta, and Boston. We are finalizing venues right now and will post ticket purchasing information in the next few weeks so watch for that!
  3. Email us and let us know you are interested in Demystified Days. The mini-site has a form at the bottom that will let you indicate your interest. Fill out the form and we will keep you in the loop!

On behalf of the teams at Analytics Demystified and Keystone Solutions we sincerely hope you are excited about what Demystified Days can become. We welcome your questions in comments or directly via email.

Help spread the word!

 

Adobe Analytics

Time Zone Trick [SiteCatalyst]

EDITOR’S NOTE:
Since joining Analytics Demystified, the most common email/comment I have received goes something like this:

“When are you going to get back to blogging about cool, advanced stuff you can do in SiteCatalyst?”

While in my new role, I am vendor-agnostic, I will do my best to keep sharing the SiteCatalyst tips & tricks I used to on my old blog. My hope is that as I work with clients using all web analytics vendors, I will branch out and share tips & tricks for all technologies. However, as I always tell people, the goal of my blog posts are to introduce concepts that can be applied to all web analytics tools…

Now on with a new tip/trick…

Dealing With Time of Day (Time Parting)

One of the analyses that I have done from time to time is Time Parting Analysis. Time Parting Analysis consists of looking at the time of the day (or day of week) that website success takes place in order to better understand its importance. While I don’t usually put a whole lot of stock into the time of day, there can be times where websites do much better/worse in the morning vs. evening. Knowing this can be used when planning advertising so you can “strike while the iron is hot,” so to speak.

If you think Time Parting might be important to your business, you should capture the time of day in some manner into variables in your web analytics tool. For example, if you use Omniture SiteCatalyst, you might use the Time Parting Plug-in to pass the time of day (in half-hour increments) to an eVar or sProp. Doing this allows you to look at a report that might resemble the one shown here:

As you can see, this report allows us to see what the action is taking place on our website down to the half-hour increment. If you are not already doing this type of analysis, it may be worthwhile since you can glean some new insights and use this data point for visitor segmentation.

Time Zone Hell!

However, inevitably you will run into a few problems with the above report. First, someone at your organization will ask you which time zone the above report is related to. Therefore, the first thing I recommend is that you clearly label your Time Parting reports with the time zone that the JavaScript file is using to capture the data. In the example above, the “Hour of Day” report was labeled do be PST (Pacific Standard Time) so it can be easily interpreted by everyone using it.

The next problem you will encounter is that of multiple time zones. If you work at a global organization and have people focusing on business in various locales, the above report is pretty much useless to many of your internal customers. If they happen to be good at math and can calculate time zone differences in their head, then you’ll be ok, but most people have trouble interpreting web analytics reports without the added labor of doing on-the-fly time zone translation!

Want to see this problem in action? Take a closer look at the report above. Do you notice anything strange? If you look closely, most of the Visits and Form activity took place in the evening. People might like your product(s), but not so much that they are willing to spend their evenings looking at them! The reason the above report looks strange is because it is for an Australian website, but the time zone is Pacific Standard Time. If you are a web analyst in Australia, seeing your website success events in the Pacific Time Zone is not super-helpful!

So how do we fix this? All it takes is a bit of creativity and meta-data. Keeping in mind that there is a direct relationship between time zones, you can take the above report and apply meta-data to it to adjust for alternative time zones. If you are using Omniture SiteCatalyst as in the example above, this means using SAINT Classifications. By applying a different SAINT Classification for each time zone you care about, you can create new reports for each time zone. Here is an example of what the SAINT file might look like for a few additional cities:

As you can see here, we took the data that was already being collected (the Key column, which in this case is PST) and added meta-data for four additional cities. You can add as many cities as you want and each column you add will create a new report for that city time zone. Once you have done this, you can see a new version of the report above adjusted for each time zone. Now if we look at the same report above, but use the Sydney Time Zone classification report, we see a report like this:

You will notice that now we are seeing the same exact data as the first report, but now the times of the website successes are adjusted for the Sydney time zone. This makes the report look a bit more normal for the Australian web analyst as the success events are now shown as taking place during more realistic business hours. The best part of this solution is that anyone using the standard Time Parting plug-in Omniture provides can use the same SAINT Classification file. It just needs to be adjusted so the “Key” column is the time zone for which you are collected the data. If you are using the PST time zone, you can download the file I showed above. If you are in a different time zone, you can still download the file and adjust it as necessary.

Caveats

As always, there are a few caveats with any “hack,” so here are mine:

  • I take no responsibility for daylight savings time which can wreak havoc on time zone translations, but even in that worst case, your data will be an hour off…
  • Time Parting reports can also be used to track Day of Week. This is harder to adjust for than is time zone unless you are time stamping using the actual date and are willing to have a massive, multi-year SAINT Classification file. This is not a bad approach, but is much more involved. Contact me if you’d like to explore this.
  • It is possible to collect time zone data using different time zones for each report suite. For example, it may be better for you in the long run to have your Sydney data collected in the Sydney time zone and your London data in the London time zone, but I have often seen clients have issues with this and if you don’t start doing this from the onset, you can have issues going to it later. Please consult your account manager for more details.

Final Thoughts

So there you have it, a few thoughts on Time Parting and a fun trick to make it more useful if you do business in multiple time zones. Give it a whirl and let me know what you think…

If you have any questions or want to learn more, feel free to contact me for more information.

Analytics Strategy, Social Media

eMetrics San Francisco 2011 — Recap by the Tweets

Note: There’s a lot of gushing I could do about how great it was to meet a lot of people in person whom I’d only known via Twitter prior, to see people I’ve met before, and to meet new people…but I’ll save some of that for a later post. This is the “content recap” post.

The last 3-4 conferences I’ve gone to, I’ve used Twitter in lieu of a notepad for my note-taking. What I realized after the first time I tried this was that it forced me to be succinct and to be selective as to what I noted. Now, for better or worse, my thumbs have gotten a bit more nimble at the same time that the input mechanisms on my mobile devices have improved. So, some might say I’m not as selective as I should be!

But, after eMetrics in D.C. last fall, I realized that there’s another benefit of tweet-based note-taking at conferences — it enables crowdsourcing the key takeaways. In theory, at least! Given that, I decided to organize this recap based on one thing: the top most retweeted tweets during the conference, as reported by a TweetReach tracker I set up. Scroll to the end of this post to download a CSV with the raw data, if you’re interested in crunching it yourself.

With that, here are my six summary takeaways:

The Data Isn’t Actionable without Storytelling

Hands, down, the most retweeted tweet (14 times) from the conference was this from Wendy Ertter:

Several presenters touched on the fact that one of the key challenges in our industry is communicating what the data means. As analysts, it can be easy to get absorbed in the data to the point that we intuitively can interpret our analyses. All too often, though, we forget that the business users we’re supporting are neither “wired for data” nor have they been as immersed in it as we have. So, rather than getting stamp-your-foot irritated that your brilliant insights have not led to action, take a look at how those insights are being communicated.

Now, not discussed was the fact that “tell a story with the data” can easily come across as “torture the data until it tells the story you want it to.” It’s a fine line, really, that means transparency has to come along with the storytelling. Storytelling must be merely a means of “effectively communicating the truth” — conveying what the data really is saying, but in a digestible manner.

Social Media, Social Media, Social Media

Ken Burbary’s tweet during Guy Kawasaki’s closing keynote (which garnered quite a bit of ire from the attendees, but that’s potential fodder for a future post) was retweeted 11 times:

Social media was a hot topic at the conference, with the sessions devoted to it concluding: “It’s tough to analyze.” In general, there was consensus that Performance Measurement 101 still applies — if you want to have any hope of measuring social media, you darn tootin’ better have clear objectives for your investment in the channel. Now, because social media isn’t the same as longer standing channels, there are different measures to work with.

One of the more intriguing sessions I attended was a panel, moderated by Michele Hinojosa, that featured Gary Angel of Semphonic and Michael Healy. The subject was sentiment analysis. Specifically, sentiment analysis of short-form text messages — Twitter and the like. Both by illustrating examples and talking through some of the advanced machine learning algorithms that have been applied to the challenge, they made a pretty strong case that trying to discretely quantify sentiment in a Twitter world is a fool’s errand.

Gary also made a distinction between “monitoring” and “measurement” and, later in the discussion, postulated that social media may be one case where you actually need to do analysis first and then set up your measurement. This makes sense, even in light of my “Performance Measurement 101” comment above. It does make sense to sift around in the conversation that is going on around a topic or a brand a bit to get a human and qualitative sense of the lay of the land before determining exactly what to measure and how.

[Update: I just realized that Gary wrote up a pretty detailed post about his key points in the session over on his blog last week — it’s worth a read.]

Attitude Is As Important As Behavior

This tweet from @SocialMedia2Day during Larry Freed’s opening day keynote was retweeted 10 times:

Foresee Results was the Diamond Sponsor for eMetrics, and the company continues to push the web analytics industry to recognize attitudinal data as being every bit as important as behavioral data. Interestingly, VOC vendors overall had a much more prominent presence than web analytics vendors (only Google Analytics and Yahoo! Web Analytics were exhibitors at the event — Webtrends, Adobe/Omniture, and Coremetrics were nowhere to be seen in the exhibit hall).

I have to credit Chris Dooley from Foresee Results for initially introducing me to (read: pestering me about) the rightful place of attitudinal data as a companion to behavioral data. He was right when he started preaching it, and he’s still right today. Another VOC vendor noted during his presentation that, when his company surveyed the top 500 retail sites and the top 500 overall trafficked sites, they found that only 15% were running on-site surveys. That is both surprising and alarming! OpinionLab also impressed a number of people with their presentations in the exhibit hall theater, and iPerceptions provided a bit more detail about their coming 4Q Premium product (which, seeing as how they announced it was coming back in October, is somewhat underwhelming given the price tag).

In short, lots of reinforcement that the voice of the customer matters and shouldn’t be ignored!

comScore’s Silver Bullet (A Bit Tarnished, IMHO)

Since I said I’d go with the most retweets, I have to include this one from John Lovett, which was retweeted 10 times:

The key here is that comScore announced all the problems they were solving. The main differentiator, as best as I can tell, is that comScore is combining web analytics capabilities with its rich demographic/audience-based data. That might be slick, although it seems that they’re overpromising a bit when it comes to the flexibility of the tool and the completeness of the demographic data. I trust John…a lot…so maybe I’m being unduly and prematurely cynical. We’ll see.

Consumers Are Cross-Channel — So Should Be Your Analysis

At the risk of inflating John’s ego (which I’m not all that worried about, but if, ages hence, he’s turned into a pompous ass, I’ll dig up this post and claim credit for starting a perfectly pleasant guy down that path!), the next tweet and the last one are all Lovett-related. Lovett. Love it! 🙂 This next one was Eric Peterson quoting John and was retweeted 10 times:

Data integration and cross-channel analytics were covered by a number of presenters. With the exception of one vendor (who shall remain nameless…but who announced a name change to his company at the conference), the overwhelming agreement was that cross-channel integration is hard, tedious, expensive…and necessary. That one vendor had a video that showed it as being simply a technology issue (and they had the technology!). I’ve dabbled in the customer data integration (CDI) world enough to know that doing this integration at the individual person level is a bear.

But, because customers are living in multiple channels — offline, digital, mobile, social — and are switching freely between them, it’s dangerous to narrow in on a single channel and draw too many conclusions. This challenge isn’t going to go away any time soon.

Several times, both in sessions and in hallway discussions, it came up that both “WAA” and “eMetrics” have quickly become misnomers. Most of the attendees at the conference have responsibilities well beyond simply “web site analytics,” and simply “digital metrics.” I put a plug in that we could start considering “eMetrics” to be “everywhereMetrics,” which is a shameless ripoff of Resource Interactive‘s stance that “eCommerce” has become “everywhereCommerce.”

Fun times to come!

Consumer Privacy — the Regulations, the Law, the Ethics of It

Covered briefly in several sessions, touched on in the WAA Member Meeting, and then covered in depth in a panel was the challenges our industry is facing with regards to consumer concerns about privacy:

John has been the face of a multi-person effort to craft a code of ethics that individuals can sign that lays out how we will treat customer data. What became evident at eMetrics is that there simply is no easy answer to “consumer privacy.” And, the fact that the FTC covers the U.S. and has taken differing stances from the EU, and the EU will get to “one policy…implemented and enforced by country,” just makes my head hurt.

The good news, it seems, is that there seems to be an emerging philosophical consensus as to what is “good/okay” and what is “bad” when it comes to user tracking. The kicker is that it’s really, really hard to write that down in an unambiguous, loophole-free way.

If anything, I took away a sense of empowerment when it comes to really living the Code of Ethics and speaking up if/when I see an initiative starting to get into a gray area — it’s not just a “do the right thing because it’s ethical” case at this point. It’s a “do the right thing…or it might come out that you didn’t, and your brand can get burned severely.”

The Tweets Themselves…

As promised at the beginning of this post, if you want to download the data file with all of the tweets from 14-Mar-2011 to 16-Mar-2011 (Eastern time) that came out of my Tweetreach tracker, you can do so here. If you do anything interesting with them, please leave a comment here as to what that was.

Social Media

eMetrics Day 1 — Let's Look at the Tweets!

Update: I misstated @johnlovett’s follower count in the initial post. This was a fatigue-driven user error on my end — not bad data coming from either tool employed in this analysis and has been corrected!

Picking up on Michele Hinojosa’s quick analysis of tweets from the first day of the Omniture Summit, I thought I’d take a quick crack at Day 1 of eMetrics. I used TweetReach and a “tracker” (query) I set up a couple of weeks ago for that.

Now, I was a bit short-sighted, in that I set up the tracker on Eastern time. But, we still cover the main bulk of the tweets by selecting March 14th for the analysis range, so I’m not going to lose any sleep over it. The high-level summary:

Let’s take a look at some of the more interesting tweets, as identified using a few different criteria.

Just looking at raw exposure of the tweets, @SocialMedia2Day really dominated with their tweets. Now, @SocialMedia2Day has over 59,000 followers, which means every tweet gets recorded as that many impressions — even before anyone retweets (and there are more followers who might retweet). According to Twitalyzer, @SocialMedia2Day has an effective reach of 175,226, which puts the account in the 98.2nd percentile. The top 3 tweets, just based on raw exposure:

Notice that the top tweet had 10 retweets — 10 people in @socialmedia2day’s network thought it worth repeating. And, it’s a pretty good point content-wise.

@comScore also has a high follower count — more than 24,000, and an effective reach from Twitalyzer of 46,474 (94.3rd percentile). So, after all of the @socialmedia2day tweets comes a list of all of the @comScore tweets. Jumping beyond those as anomalies, of sorts, we get the top tweets by “individual” contributors:

John’s Code of Ethics tweet was retweeted 9 times and garnered almost 30,000 impressions. Nice! We care about acting responsibly! John’s tweet generated its exposure through retweets, as he has around 2,500 people following him…which is a lot of people, but only 1/4 of Ken, who has 10,000 people following him (and he’s following 10,000 people), so his tweets generate ~10,000 impressions just from him tweeting them.

So, looking at raw retweet volume is an indication of how naturally interesting and repeatable a user’s followers (and any followers who retweeted) found the tweet to be. The top retweeted tweet was retweeted 11 times:

Again…a pretty sharp observation.

Shifting around to the top contributors, TweetReach again provides a list based on the exposure generated by each user. The top 35:

We covered that @SocialMedia2Day, @comScore, and @kenburbary have a very high follower count, so let’s take a look at the next two. First, @michelehinojosa, who has just under 1,000 followers, an effective reach in Twitalyzer of 18,852 (89.7th percentila), and tweeted about eMetrics 127 tweets over the course of the day (tweet detail sorted by highest to lowest exposure):

Note the top two tweets were retweeted multiple times…and they’re worth sharing!

And, finally, yours truly — a bit under 1,200 followers, and a Twitalyzer effective reach of ~3,000  (although it jumped up to north of 89,900 starting on March 9th, which is twice what @comScore’s effective reach is, and they have 20X the followers; I need to ping the Twitalyzer folk to help me understand how that happened). My top 5 highest exposure eMetrics tweets for the day:

The second tweet — which was just a humorous observation — was interpreted as a “reply” to @jimsterne…but it showed up as the second-highest exposure tweet. That’s not exactly high-value content — more of a chuckle for those in the room who were watching the #emetrics stream. And, interestingly, I got a direct message from a follower midway through the day that they were unfollowing me as I was clogging their stream. I’m somewhat sensitive to that, but, with tweets being, essentially, public note-taking for me at conferences (and the enticing opportunity to then analyze and summarize those tweets after the conference, so it’s actually shared public note-taking), I suppose I’m okay with that.

Overall, this (very quick) analysis seems to reveal that the most engaging (egad! scary word!) tweets were one that stated, succinctly and eloquently, truths about our profession. I also  I would’ve liked to generate a word cloud of all of the tweets (appropriately cleansed)…but that’s simply not as quick and easy as I wish it was!

What do you think?

 

 

Excel Tips

Excel Dynamic Named Ranges Redux — Multiple Series in One Chart

In one of the more consistently popular posts I’ve written, I went into detail about how to set up charts that would update based on a value selected from a couple of dropdown menus – specifically geared towards a dropdown menu that allows the selection of a date such that the chart(s) would update to reflect the data up to that date.

One of the commenters asked how to include multiple data series in a single chart using that same technique. I did a very quick example via email, but I mentally committed to documenting the specifics on the blog, so here we go (file download at the end of this post).

Add Some Data

I could, of course, just use the data I was already working with, but none of that fictitious data made sense as a stacked bar chart. So, the first step is to add a couple of data series that might reasonably belong in a stacked column chart – an easy one is to break out the web traffic into “New Visitors” versus “Returning Visitors.”

Following the same technique as described in the original post, I name the top cells NewVisitors_Current (Column E) and ReturningVisitors_Current (Column F) and copy the formula from the Web Traffic column into those two columns (it’s the same formula in all cells in row 1, and they can be copied without modification due to the use of “COLUMN()” in the formula).

Then, create NewVisitors_Range and ReturningVisitors_Range named ranges by going to Formulas » Name Manager, copying the formula for WebTraffic_Range, and then creating the two new named ranges using the same formula, except swapping out “WebTraffic” in the formula with “NewVisitors” and “ReturningVisitors.”

Note: This may seem like a complicated setup. It’s actually pretty quick and simple, and can even be achieved using a macro if there are a slew of metrics that need to be set up. One tip, though, is to establish a consistent naming convention for the different aspects of each metric.

So, enough with the seup. How do we put multiple series into a stacked bar chart?

Copy One of the Line Charts

The easiest way to get our base chart is to simply hold down <Ctrl>-<Shift> and click and drag one of the existing charts straight down on the worksheet. I’m a fan of copying charts rather than making new charts from scratch for two reasons: 1) It’s easier to keep them aligned and exactly the same size, and 2) It’s easier to keep the formatting the same (the formatting in this example is horrid, but that was for the sake of simplicity in the initial tutorial).

So, now we have two charts (I copied the date and “current total” cells as well, but we’re pretty much done there now – in this case, the current total uses the “Web Traffic” value, and it’s the sum of the New Visitors and Returning Visitors):

Change the Chart Type

Select the chart and then go to Chart Tools » Design » Change Chart Type and select the Stacked Column chart type:

You will now have a chart that looks like this:

But, this is still only one data series, and it’s the overall web traffic – not the breakout of new visitors and returning visitors. So…

Update the Data Series

Click on the columns in the chart, and a formula will appear in the formula bar (it’s not you…it’s a small image; image width constraints I apply to this blog, but you get the idea):

There are other ways to update the data, but this is the fastest when it’s a viable option. Simply change the first “Web Traffic,” which is the name of the data series, to “New Visitors.” Then, change “WebTraffic” later in the formula to “NewVisitors”. What you’re really doing with this second change is changing the data source from “WebTraffic_Range” to “NewVisitors_Range”.

The chart will update and will look like this:

Now, since we’re going to have two series on this chart, let’s go ahead and click on the column title and change it to “Web Traffic” manually (when you changed “Web Traffic” to “New Visitors” in the formula bar, you were changing the series name — Excel just noticed that you had only one series and no legend, so it decided to make that the chart title, too; you’ll still want the series name to be “New Visitors,” though; the reason should become apparent shortly…like…after the next sentence!). And, while we’re at it, let’s add a legend and make the chart a bit taller to make room for it!

Add the Second Series

Now, here’s where the fun happens. Right-click in the chart and select Select Data. Then, select New Visitors and click the Edit button. You’re not actually going to edit that data series, but it’s the fastest way to get the second series set up. In the Edit dialog box, select the entire contents of the Series values field and select <Ctrl>-<C> to copy the formula:

Click Cancel.

Click Add.

For the Series name enter “Returning Visitors” and then paste the formula (<Ctrl>-<V>) you just copied into the Series values field. Then, scroll to the end of that formula and replace “New” with “Returning”:

Click OK and then click OK again on the next screen.

Voila!

Still, as before, you can change the Report Period and the Report Range dropdowns to alter what data gets displayed on the chart.

You can download the spreadsheet with the full example if you want to fiddle around with it without starting from scratch.

Happy charting!

 

 

Social Media

Omniture Announces SocialAnalytics

Omniture’s SocialAnalytics offering won’t be publicly available until summer of this year, but the early glimpses show big promise for the burgeoning field of SocialAnalytics. What makes this tool different from the many capable tools already out on the market is the tight integration of web analytics data with social brand or keyword mentions. This means that you can collect and analyze data from major social media channels like Facebook, Twitter, YouTube (45 data social media sources in total) and perform web analytics style slicing and dicing on the results.

Yet, the beauty of this solution is that users can trend and analyze social metrics against any metric within the SiteCatalyst interface. Further, the SocialAnalytics offering allows users to correlate data from social media with SiteCatalyst metrics and even offers a percentage of statistical confidence. This exceeds what I’ve seen in any other social analytics offering currently on the market. To illustrate with a hypothetical example, the Omniture SocialAnalytics capabilities will allow you to imbed traditional SiteCatalyst campaign ID codes into a your social media marketing on Twitter, YouTube and Facebook, which could all be monitored for activity within the SiteCatalyst interface. You could then trend the social data from campaigns and mentions against any metrics that you currently use within SiteCatalyst such as visitors or conversions. Thus, you could monitor the impact of your social marketing as a driver for website traffic and determine what percentage of that traffic actually purchased online as a result of the social campaign. The tool does this by making a correlation (versus actually pinning causation), but the statistical confidence will deliver assurance as to the validity of the correlation. This is magical. It actually enables users to quantify ROI from social marketing activities with a degree of statistical confidence. No one else has this that I’m aware of today.

Further, one of my pet peeves with today’s social analytics tools is the inability to create custom metrics. In most cases, you have to deal with the formulas and calculations that vendors deliver. The exception here is firms like Radian6 that allow users to weight factors for calculated metrics like Influence, whereby users do have some controls over their metrics. Yet, Omniture’s SocialAnalytics allows users carte blanche ability to create custom metrics and report on them within SiteCatalyst and even leverage in report builder and other Omniture functions. This is a revolutionary step in controlling the way that social is currently measured because it introduces a level of customization that was formerly absent.

While, it’s still early days and this was only my first glimpse at the product, you can probably tell that I’m bullish already. It’s currently in private beta for a few lucky Omniture customers who will undoubtedly bang away at it and help to shape the future of this product. However, there’s still a long way to go before this new tool is street legal, so most Omniture users will have to wait until the general release this summer. I’ll also say that this tool currently does not offer a wholesale replacement for Radian6 or other enterprise social analytics vendors on the market. The primary reason for this is that there is no engagement capability from the interface (i.e., can’t send Tweets or respond to Facebook comments directly). Additionally, there is no workflow built into the SocialAnalytics solution either. Thus, while social is about the interaction between a brand and its customers, Omniture is still leaving its clients to work that out using other means. They do however deliver some of the most robust analysis and reporting capabilities of anyone out there. If you’re looking to make sense of social media and measure the impact it has on your business operations; I suggest you give Omniture’s new SocialAnalytics tool a good look.

Adobe Analytics

Welcome to SiteCatalyst v15

Among the many announcements Adobe made at the 2011 Omniture Summit (#omtrsummit), probably the most anticipated was the release of version 15 of the flagship SiteCatalyst product. Those of us who follow SiteCatalyst regularly know that this release has been a long time in the making. Unfortunately, Omniture didn’t provide much detail in the keynote about specific enhancements so in this post, I will try to highlight some of the key things that I have heard about this new release (but in the interest of sharing info in semi-real-time, forgive me if I am not 100% correct and keep in mind I am writing this between sessions!). Since version 15 isn’t scheduled to be released right away (April?), not all features listed here are set in stone and as more details emerge about the release, I will follow-up with additional information/corrections…

Instant Segmentation Segmentation
The ability to segment data has always been a two-step process in SiteCatalyst. You could segment your data by passing values into eVars and sProps, utilize DataWarehouse/ASI, but if you wanted real-time segmentation you had to pay additional $$ for Discover or Insight. Unfortunately, most of the available options required you to wait for your segmented data which is not ideal from a web analytics perspective. However, when Google’s free analytics product released the ability to segment data in real-time, it became apparent that SiteCatalyst’s segmentation capabilities needed to be improved. The masses asked why they were getting less functionality than a free product?

With version 15, Omniture will now provide the ability to segment data in real-time. This will go a long way to appeasing those who realize that segmenting data is almost as critical as collecting the data. Instant segmentation will allow casual users to slice and dice website data without having to go through power users and then wait for the data to process. As you might expect, when business users have questions, they usually want the answer NOW! Forcing them to wait causes you to lose momentum and prohibits adoption so I think this feature will really help create more data-driven cultures. While this feature will be a big hit with the SiteCatalyst community, I expect that other web analytics vendors will position this as Omniture gaining parity with what they have already had.

One outstanding question I have is what this release means for ASI? Does this product/feature go away? Do customers who have paid for it, get some $$$ back?

New Architecture
So why did it take so long to introduce instant segmentation? Well the answer lies in the next big item Omniture discussed – a next-generation architecture. While I am not privy to all of the details, Omniture has stated that they redesigned the entire back-end of SiteCatalyst so that it could scale better and provide additional functionality like instant segmentation. Unfortunately, since most end-users won’t ever see the “back-end” of SiteCatalyst it will be hard to appreciate what went into it, but if this new architecture is as described, it should allow for more features and faster product improvements in the coming months/years.

However, there is one important catch to this v15 architecture. End-users will not be able to upgrade to v15 be themselves, but instead will need to work with Omniture to upgrade. This is due to the fact that once you upgrade, there is no going back. This is because v15 processes data differently than its predecessor. In general, v15 will process data in a manner that is more similar to Discover so users of both products should find that their data between SiteCatalyst match much more closely going forward. However, this means that SiteCatalyst v15 will approach things in a slightly different manner which could result in key metrics like Visits being slightly different than they were in previous versions. This means that looking at YoY data could show some variances, but SiteCatalyst will have an alert that tells you when you are comparing pre v15 data to v15 data so you are at least aware of this potential anomaly.

While it will remain to be seen how the SiteCatalyst community reacts to this, my hunch tells me that most clients will bite the bullet and upgrade to v15 and deal with this one time data discrepancy and take the benefits that v15 provides with respect to functionality.

More eVar Subrelations
Power SiteCatalyst users will rejoice in the fact that hey can now have full subrelations on more (all?) eVars. This means that you can break down more eVars by other eVars. In the past, you could only select a few conversion variables for which you wanted to see breakdowns, but this limitation will be reduced (abolished?) in v15. This is huge news and is another example of why the new back-end architecture is so vital.

UPDATE: Brett’s closing session suggested that ALL eVars and sProps could be broken down by each other. I have heard conflicting things on this so stay tuned!

Trend Multiple Metrics!
While it may not sound super-sexy, one new feature of the v15 release is the ability to trend more than one metric at the same time. To date, you can view multiple metrics in a “Ranked” report, but as soon as you switch to the trended view, only the 1st metric is trended. This has been a real bottle-neck and forced people like me to create additional reports in ReportBuilder to get this functionality. Version 15 solves this and I am told that it will continue to improve over time.

Visits & Visitors in all Reports
Another sticking point for SiteCatalyst users was that you could not see Visit/Visitor metrics in all reports. There were numerous workarounds, but most had an additional cost associated with them, but in v15 you can see both metrics in most reports. This will be a welcome addition, especially in conversion reports where they are needed the most. I have not confirmed whether Visits and Visitors will have full subrelations or not.

Ad-Hoc Unique Visitors
Currently in SiteCatalyst you can see unique visitors for set time periods, such as day, week, month, but if you choose an ad-hoc date range, you cannot see an accurate unique visitor count. In version 15, Omniture has rectified this like it had done in the Discover product. This feature will bring SiteCatalyst to closer parity to other web analytics vendors who have been providing similar unique visitor counts for any timeframe.

UPDATE: Brett’s closing session mentioned that you can also see ad-hoc unique visitors for Pages as well. That might mean that arbitrary time frame ad-hoc unique visitors might be available for all sProps?

Bounce Rate
After many requests from customers, Bounce Rate will finally be a standard metric in SiteCatalyst. Initially it sounds like it will be limited to a few reports, but it looks like it will be more pervasive in the future. While it has been possible to create Bounce Rate work arounds to compensate for not having Bounce Rate as a default metric, I think the gerenal population will be happy to have this baked into the product.

Video Enhancements
Previously, video data was relegated to video-specific reports only. Those clever enough would add custom metrics and eVars to get around this, but now it appears that you no longer have to do this to see video data in all SiteCatalyst reports.

New iPad App
In v15, the SiteCatalyst iPad app is getting a huge overhaul and will allow for more advanced web analysis on-the-go:

General UI Enhancements
Mixed into this release are a bunch of UI enhancements that people will probably like. These include searchable menus, report-specific default metrics, some new dashboard stuff and some more hand-offs between multiple Omniture products (like sharing segments with Test&Target). I think most will notice that that Omniture spent some cycles thinking about how an analyst uses the tool on a daily basis.

Long Live the Idea Exchange!
Lastly, I wanted to take a moment to thank Omniture for listening to its customers via the Idea Exchange. Many of the items above were highly voted upon by the Omniture community through the Idea Exchange. Omniture has done a great job of listening to its clients throughout the year (in addition to Brett’s fun Summit session!) so that it can focus its development efforts on what the majority of people are saying they want. It takes real courage as an organization to open up and ask customers what they want, interact with them, let all customers see this and then deliver the top items. While it sounds like common sense, there are very few vendors doing this today and I applaud Omniture for being forward-thinking about it. A special shout-out goes to Bill, JD and Ben who worked hard to champion this effort and I hope that v15 and beyond are the better for it…

UPDATE – ADDITIONAL FEATURES MENTIONED AT BRETT’S CLOSING SESSION

Dashboard Segmentation
In v15 it will be possible to apply real-time segments to SiteCatalyst Dashboards which will change all reportlets on the dashboard

Default Metrics by Report
In current versions of SiteCatalyst, you can set default success event metrics for conversion reports, but it is an all or nothing proposition. In v15, it sounds like you will be able to assign different default metrics for different conversion reports.

Data Warehouse Improvements
It sounds like v15 will provide more information about pending DataWarehouse requests and possibly allow for re-running (or “Save As”) of DataWarehouse requests. The latter will be a huge time-saver since today, you have to re-create each from scratch to make any changes…

Adobe SocialAnalytics
In addition to the new version of SiteCatalyst, another related product release is the Adobe SocialAnalytics product. This new product will compliment SiteCatalyst and will allow companies to monitor all social media activity. This product will be positioned as a competitor to Radian 6 and others in the social media measurement space. Key parts of this product are measurement for Twitter, Facebook and YouTube. Personally, I am excited that Omniture has formalized some of the cool social media tracking things I spoke about a few years ago and delivering on past promised features like viral video measurement. This new product will allow you to see Social Media metrics side by side, but and filter on specific influential users, but doesn’t appear to show if it is the same people who are coming from Social Media sites and then converting (if that is even possible!). Unfortunately, I believe this new product won’t be available to everyone until Q3, but it is interesting to see this new product, especially in the context of what was announced by Webtrends last week around social dashboards.

Final Thoughts
While I will defer final judgement until I learn more about all of the new v15 features, at a high level, I think that version 15 is a big step forward for Omniture. While there are not hundreds of new features, they have hit some really big ones that will have a real impact for power users. I predict that Omniture’s competitors will discount this release by saying that SiteCatalyst is now providing functionality they have had for years. While I could see that argument (and don’t disagree with it), I will offer the following perspective. SiteCatalyst was a product that experienced tremendous growth over a very short time frame as they went from a vendor that no one had heard of ten years ago, to one of the most popular web analytics tools used in the enterprise. With that growth, it was likely hard for SiteCatalyst to change its back-end architecture during this growth spurt, whereas other tools have either been around longer (and had more time), or come around afterwards (and had the ability to start with a clean slate). It is in that context that I still believe that Omniture is taking a big step forward and that the move to a new architecture is probably the right move for the SiteCatalyst product. I will be curious to hear your thoughts as you start seeing more about the product this week and beyond…

So those are a few of my favorite new features of SiteCatalyst v15 and some thoughts on SocialAnalytics and the new platform. What are your favorites? Have you heard of others? Are there any I listed that are not true? Which features were you hoping for that didn’t make the release?

As always, if you have any questions about SiteCatalyst or migrating to v15, feel free to contact me to learn more. Thanks!

Analytics Strategy, Reporting, Social Media

A Framework for Social Media Measurement Tools

Fundamental marketing measurement best practices apply to social media as much as they apply to email marketing and web site analytics. It all begins with clear objectives and well-formed key performance indicators (KPIs). The metrics that are actually available are irrelevant when it comes to establishing clear objectives, but they do come into play when establishing KPIs and other measures.

In a discussion last week, I grabbed a dry erase marker and sketched out a quick diagram on an 8″x8″ square of nearby whiteboard to try to illustrate the landscape of social media measurement tools. A commute’s worth o’ pondering heading home that evening, followed by a similar commute back in the next morning, and I realized I might have actually gotten a reasonable-to-comprehend picture that showed how and wear the myriad social media measurement tools fit.

Here it is (yep — click on the image to view a larger version):

‘Splain Yourself, Lucy

The first key to this diagram is that it makes a distinction between “individual channel performance” and “overall brand results.” Think about the green box as being similar to a publicly traded company’s quarterly filing. It includes an income statement that shows total revenue, total expenses, and net income. Those are important measures, but they’re not directly actionable. If a company’s profitability tanks in any given quarter, the CEO can’t simply say, “We’re going to take action to increase profitability!”  Rather, she will have to articulate actions to be taken in each line of business, within specific product lines, regarding specific types of expenses, etc. to drive an increase profitability. At the same time, by publicly announcing that profitability is important (a key objective) and that it is suffering, line of business managers can assess their own domains (the blue boxes above) and look for ways to increase profitability. In practice, both approaches are needed, but the actions actually occur in the “blue box” area.

When it comes to marketing, and especially when it comes to the fragmented consumer world of social media, things are quite a bit murkier. This means performance measurement should occur at two levels — at the overall ecosystem (the green box above), which is akin to the quarterly financial reporting of a public company, and at the individual channel level, which is akin to the line of business manager evaluating his area’s finances. I use a Mississippi River analogy to try to explain that approach to marketers.

Okay. Got It. Now, What about These “Measurement Instruments?”

Long, long, LONG gone are the days when a “web analyst” simply lived an breathed a web analytics tool and looked within that tool for all answers to all questions. First, we realized that behavioral data needed to be considered along with attitudinal data and backend system data. Then, social media came along introduced a whole other set of wrinkles. Initially, social media was simply “people talking about your brand.” Online listening platforms came onto the scene to help us “listen” (but not necessarily “measure”). Soon, though, social media channels became a platform where brands could have a formally managed presence: a Facebook fan page, a Twitter account, a YouTube channel, etc. Once that happened, performance measurement of specific channels became as important as performance measurement of the brand’s web site.

When it comes to “managing social media,” brand actions occur within a specific channel, and each channel should be managed and measured to ensure it is as effective as possible. Unfortunately, each of the channels is unique when it comes to what can be measured and what should be measured. Facebook, for instance, is an inherently closed environment. No tool can simply “listen” to everything being said in Facebook, because much of users’ content is only available to members of their social graph within the environment, or interactions they have with a public fan page. Twitter, on the other hand, is largely public (with the exception of direct messages and users who have their profile set to “private”). The differing nature of these environments mean that they should be managed differently, that they should be measured differently, and that different measurement instruments are needed to effectively perform that measurement.

Online listening platforms are not a panacea, no matter how much they present themselves as such. Despite what may be implied in their demos and on their sites, both the Physics of Facebook and the Physics of Twitter apply — data access limited by privacy settings in the former and limited by API throttling in the latter. That doesn’t mean these tools don’t have their place, but they are generalist tools and should be seen primarily as generalist measurement platforms.

Your Diagram Is Missing…

I sketched the above diagram in under a minute and then drew it in a formal diagram in under 30 minutes the next morning. It’s not comprehensive by any means — neither with the three “social media channels” (the three channels listed are skewed heavily towards North America and towards consumer brands…because that’s where I spend the bulk of my measurement effort these days) nor with the specific measurement instruments. I’m aware of that. I wasn’t trying to make a totally comprehensive eye chart. Rather, I was trying to illustrate that there are multiple measurement instruments that need to be implemented depending on what and where measurement is occurring.

As one final point, you can actually wipe out the “measurement instrument” boxes and replace those with KPIs at each level. You can swap out the blue boxes with mobile channels (apps, mobile site, SMS/MMS, mobile advertising). I’m (clearly) somewhat tickled with the construct as a communication and planning tool. I’d love to field some critiques so I can evolve it!

Analytics Strategy

Web Analytics Tools Comparison — Columbus WAW Recap Part 2

[Update: After getting some feedback from a Coremetrics expert and kicking around the content with a few other people, I rounded out the presentation a bit.]

In my last post, I recapped and posted the content from Bryan Cristina’s 10-minute presentation and discussion of campaign measurement planning at February’s Columbus Web Analytics Wednesday. For my part of the event, I tackled a comparison of the major web analytics platforms: Google Analytics, Adobe/Omniture Sitecatalyst, Webtrends, and, to a certain extent, Coremetrics. I only had five minutes to present, so I focussed in on just the base tools — not the various “warehouse” add-ons, not the A/B and MVT testing tools, etc.

Which Tool Is Best?

This question gets asked all the time. And, anyone who has been in the industry for more than six nanoseconds knows the answer: “It depends.” That’s not a very satisfying answer, but it’s true. Unfortunately, it’s also an easy answer — someone who knows Google Analytics inside and out, has never seen the letters “DCS,” referenced the funkily-spelled “eluminate” tag, or bristled at Microsoft usurping the word “Vista” for use with a crappy OS, can still confidently answer the, “Which tool is best?” question with, “It depends.”

And You’re Different?

The challenge is that very, very few people are truly fluent in more than a couple of web analytics tools. I’ve heard that a sign of fluency in a language is that you actually think in the language. Most of us in web analytics, I suspect, are not able to immediately slip into translated thought when it comes to a tool. So, here’s my self-evaluation of my web analytics tool fluency (with regards to the base tools offered — excluding add-ons for this assessment; since the add-ons bring a lot of power, that’s an important limitation to note):

  • Basic page tag data capture mechanics — 95th percentile — this is actually something pretty important to have a good handle on when it comes to understanding one of the key differences between Sitecatalyst and other tools
  • Google Analytics — 95th percentile — I’m not Brian Clifton or  John Henson, but I’ve crafted some pretty slick implementations in some pretty tricky situations
  • Adobe-iture Sitecatalyst — 80th percentile — I’m more recent to the Sitecatalyst world, but I’ve now gotten some implementations under my belt that leverage props, evars, correlations, subrelations, classifications, and even a crafty usage of the products variable
  • Webtrends — 80th percentile — I cut my teeth on Webtrends and would have put myself in the 95th percentile five years ago, but my use of the tool has been limited of late; I’m actually surprised at how little some of the fundamentals change, but maybe I should
  • Coremetrics — 25th percentile — I can navigate the interface, I’ve dived into the mechanics of the different tags, and I’ve done some basic implementation work; it’s just the nature of the client work I’ve done — my agency has Coremetrics expertise, and I’m hoping to rely on that to refine the presentation over time

So, there’s my full disclosure. I consider myself to be pretty impartial when it comes to tools (I don’t have much patience for people who claim impartiality and then exhibit a clear bias towards “their” tool — the one tool they know really well), but, who knows? It’s a fine line between “lack of bias” and “waffler.”

Any More Caveats Before You Get to the Content?

My goal with this exercise was to sink my teeth in a bit and see what I could clearly capture and explain as the differences. Ideally, this would also get to the, “So what?” question. What I’ve found, though, is that answering that question gets circular in a hurry: “If <something one tool shines as> is important to you, then you really should go with <that tool>.” Two examples:

  • If enabling users to quickly segment traffic and view any number of reports by those segments is important, then you should consider Google Analytics (…or buying the “warehouse” add-on and plenty of seats for whatever other tool you go with)
  • If being able to view clickpaths through content aggregated different ways is important, then you should consider Sitecatalyst

These are more of a “features”-oriented assessment, and they rely on a level of expertise with web analytics in order to assess their importance in a given situation. That makes it tough.

Any tool is only as good as its implementation and the analysts using it (see Avinash’s 10/90 rule!). Some tools are much trickier to implement and maintain than others — that trickiness brings a lot of analytics flexibility, so the implementation challenges have an upside. In the end, I’ll take any tool properly implemented and maintained over a tool I get to choose that is going to be poorly implemented.

Finally! The Comparison

I expect to continue to revisit this subject, but the presentation below is the first cut. You might want to click through to view it on SlideShare and click the “Speaker Notes” tab under the main slide area — I added those in after I presented to try to catch the highlights of what I spoke to on each slide.

Do you see anything I missed or with which you violently disagree? Let me know!

http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6= http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6=

http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6=http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6=http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6= http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6=

http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6=
http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6=

http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6=
http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6=http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6=http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6=

http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6=

Reporting

Campaign Measurement Planning — Columbus WAW Recap Part 1

We tried a new format at last week’s Columbus Web Analytics Wednesday, in that we had three completely unrelated presentations, and we kept the entire presentation period to right at a half hour. Mathematically, that gave us 10 minutes per presentation, and we split the time between formal presenting and Q&A. The event was sponsored by Resource Interactive (population 320-ish and growin’; SA…LUTE! </heehaw>), and it was our first “presentation included” WAW since last November. Apparently, we had some pent-up WAW demand, as we had right around 45 attendees.

The three presentations of the evening were:

Dave’s presentation was the most informal and focussed on the various developments across Facebook/Bing and Google when it comes to incorporating social graph and social profile data into search results. The Google video he showed was pretty interesting, and he illustrated how rapidly the space is evolving. But, overall, I took lousy notes, so I don’t know that I’ll manage to get a full blog post up on the subject.

As for Bryan’s presentation, I had the benefit of previewing the material and, as such, getting to have a mini-Q&A with Bryan via e-mail.

Campaign Measurement Planning

Bryan and I are both pretty passionate about measurement planning. His presentation really nails some key points about the topic and has a fantastic list on slide 8 as to elements to consider including in a measurement plan:

In addition, Bryan provided a (click to download) measurement plan example Word document (it’s an auto insurance company example, so it’s obviously grounded in reality, but he worked it over pretty thoroughly on several fronts in preparation for the presentation, so it is an entirely fictional example).

I asked Bryan a couple of questions offline about his approach prior to the event:

Q: Under “Targets and Benchmarks,” you note, “Don’t be afraid to put ‘TBD’ or ‘No Data’ for some benchmarks.” If that is the case, do you support not setting a target, or should you still try to set a target (even noting that it is a bit of a swag) in the absence of a benchmark?

Bryan’s response: I try to set a target no matter what because it gets people at least thinking about it and TRYING to set up some kind of expectation.  It makes sure that people are at least estimating.  Maybe they don’t know the CPC for the search terms yet, aren’t sure on the demand, and aren’t sure on the completion rates, but it’s at least a start.  We were completely off for one of our last campaigns because we had no idea on all those factors.  It still gave the agency something to report on for their % of goal and it drove an informed discussion mid-campaign.

[I, of course, loved this answer…because I totally agreed with it]

Q: You’re at a company that uses agencies for much of the campaign execution, and, clearly, you have put in a process whereby you develop this sort of plan partly as a tool to drive clarity and alignment with the agencies and their work. As an agency analyst, we are increasingly including “measurement planning” as a non-optional part of the scope of our engagements. In those cases, we (the agency) actually do the discovery and documentation of the measurement plan (which clients provide input to, review, and approve). I actually would love to have clients coming to us with this level of forethought, but, in the absence of that, what are your thoughts on having the accountability for the creation of a measurement plan reside with an agency?

Bryan’s response: I think this varies on the relationship between the agency and company.  For us, we’re very capable, we all know how to do the campaign execution, but we just don’t have the time or bodies to do it.  I’m sure there are many companies that have no clue how to do it, so the agency does both the execution and the strategy, or the execution, strategy, tracking, plus reporting, or whatever else. It really depends on the analytics maturity of the client as to whether it makes more sense for the agency or the client to own the creation of the plan.  If it’s the agency, you’d have to be absolutely be sure to talk to the client in depth about all of it and make sure they’re on board with all the points.  In the end, the outcome should be the same, the only difference really being the author of the document would be the agency instead of the business, and I’m sure some of the reporting responsibilities would change based on that.

Bryan joked during his presentation about how “exciting” the topic of measurement planning is. Obviously, it can seem like a pretty dry topic, but, in both of our experiences, measurement planning can drive some tough and interesting discussions. More importantly, it’s a foundational element of marketing — without it, you wind up looking back after the fact and wondering if what you executed was successful, whether you captured the right data, and whether you learned anything that can be meaningfully applied to the next initiative.

Hey…I also cleaned up my “sharing” options on my blog this weekend. Go ahead. Give it a try! See how easy it is to Like or Tweet (or…er…whether I really got those implemented and functioning correctly). Who knows, maybe Facebook Insights will start giving me some interesting web site data!

General, Technical/Implementation

Need A Checkup? The Doctor Is In!

When it comes to your health, most doctors say that having a regular checkup is the easiest way to prevent major illness. By simply going to see your doctor once a year, you can get your vitals evaluated and see if your blood pressure is too high or low, check your cholesterol, etc… If you happen to be sick at the time you have your checkup, you can find out if it is serious or not and if you feel fine, the checkup is a way to confirm that you are in good shape.

However, when it comes to web analytics implementations, it isn’t always easy to know how “healthy” you are. You might wonder the following:

  • Is my organization capturing the right data to ensure it can do the analysis needed to improve conversion rates?
  • Do the configuration settings of our web analytics tool make sense?
  • Are we maximizing the use of our web analytics tool or are we only using 20% of its capabilities?
  • How does our web analytics implementation compare to that of my peers/competitors?

Over the past decade, I have been associated with hundreds of web analytics implementations, and the above questions were ones that often kept my clients awake at night. And, truth be told, based upon my experience, many of them had reason to be worried. More often than not, when I crack open a client’s web analytics implementation, I am shocked by what I see. Here are a few examples of problems I encounter repeatedly:

  • Unusable pathing reports due to inconsistent page naming practices
  • Unusable campaign reports due to inconsistent tracking code naming conventions
  • Web analytic variables/reports defined, but with no data
  • Cookie settings that don’t line up with business goals (i.e. Cookie using Last Touch when Marketing uses First Touch)
  • Data inconsistencies resulting in reports that are highly suspect or untrustworthy
  • Incomplete meta-data or look-up tables
  • Lack of critical KPI’s and best practices specific to the industry vertical the website serves
  • Lack of appropriate usage of key web analytics tool features that could improve overall analytic success

The remainder of this post will discuss a new service offering Analytics Demystified will be providing to address the preceding concerns. If you are interested in knowing the “health” of your organization’s web analytics implementation, please read on…

Introducing the Web Analytics Operational Audit

So how do you know if you are doing well or poorly? Like anything, the best way to know where you stand is to perform a checkup or audit. In this case, I am referring to an audit that reviews which web analytic tool features you are utilizing and what data your web analytics implementation is currently collecting.

Since there is no official “doctor” when it comes to web analytics, we at Analytics Demystified have created what we believe is the next best thing. Taking advantage of our depth of experience in the web analytics arena, we have created a Web Analytics Operational Audit scorecard that encompasses the best practices we have seen across all company sizes and industry verticals. This scorecard is vendor-agnostic and has over 100 specific items and categories that allow you to see where your current web analytics implementation excels and where it is lacking.

Over the years, I have done this type of scoring informally, but the Operational Audit framework we have created at Demystified takes this to a whole new level. Here is a snapshot of what the scorecard looks like so you can see the format:

Our goal in creating this Operational Audit project is to have a simple, yet powerful way to objectively score any web analytics implementation from a functionality point of view. Knowing where your organization stands with respect to its web analytics implementation is beneficial for the following reasons:

  • If you think you have a robust implementation, but it turns out that you do not, you may be making poor business decisions today based upon faulty data and/or incorrect assumptions
  • What if your implementation is worse than you thought? You can try and hide it, but I have found that in the long run, bad web analytics implementations are eventually found out…usually at the worst time when an executive needs something critical and you have to come back and say “sorry, we don’t have a way to know that…” Wouldn’t you like to know sooner, rather than later, what shape you are in so you can get your web analytics house in order?
  • Maybe you have an awesome web analytics implementation, but your boss doesn’t know it! What would it do to your job/career if your boss was told by an independent 3rd party that all of the time and money they invested in your web analytics implementation have paid off! What if your web analytics implementation was in the top 10% of the general web analytics population? Promotion anyone?
  • Your organization doesn’t have unlimited time and budget for web analytics implementation projects. When the stars align and you do get resources or budget, wouldn’t it be great to be armed and ready with the top things you should be doing so you don’t miss these golden opportunities?

These are just a few of the many reasons that auditing your implementation makes sense. One important note: this Operational Audit does not include a technical audit of JavaScript tagging (which can be equally as important!).

Go Forth and Audit!

As I stated earlier, the unfortunate truth is that there is more bad than good out there. People change roles, priorities change, people leave your company, companies merge. There can be any number of reasons contributing to the devolution of web analytics implementations, but regardless of how you got to where you are, if you want to be successful, you need to grab hold of the reins of your current web analytics implementation and take ownership of it.

For example, when I joined Salesforce.com, I could have spent my time blaming our implementation shortcomings on my predecessors, but that wouldn’t help me get to where I needed to go. Instead, I chose to audit our implementation and identify what was worth keeping and what had to go! In the end, our company was better for it, and the audit led to an implementation roadmap for the next year, allowing me to know how long it would take to turn things around and what type of resources I would need.

It is based upon this recent experience that I highly encourage you to consider this Operational Audit service for your organization. Long term, one of my hopes is that I can audit enough companies, across various company sizes and verticals to enable me to create a benchmark of web analytics implementations so I can let you know how your scores compare to others like you. This way, even if most companies score poorly, you can possibly claim to be the best of what is currently out there (can you tell I liked being graded on a curve in high school?). I am also looking forward to re-scoring companies next year so they can see how their implementation has improved year over year.

Intrigued? Interested? Scared?

If you’d like to learn more about having your web analytics implementation audited, please contact me and I’d be happy to answer any questions. Thanks!

 

Reporting

The Ugly Truth About Benchmarks

Why Do We Want Benchmarks in the First Place?

As Garrison Keillor says every week, in Lake Wobegon, “all the kids are above average.” If we can simply be “above average,” then we know we’re pulling away from mediocrity. And that’s what we want with benchmarks — we want to know what “average” is so that we know the exact height of the measurement bar that, if we clear it, we can claim success (if not necessarily supremacy). It’s something to aim for that must be attainable, because others have attained it.

We’re surrounded with benchmarks in our personal lives, too: doctors tell us how our weight, blood pressure, and cholesterol compare to benchmarks for healthy people of the same age, gender, and height; standardized testing in schools are compared to statewide benchmarks; salary surveys tell us (generally in a flawed way) benchmarks for pay for others in our field. We’re used to benchmarks, and we want to use them to set targets for the key performance indicators (KPIs) for our marketing initiatives.

Benchmark = Target…right?

All too often, I run up against someone who equates a benchmark with a target. That’s dangerous for two reasons:

  • Benchmarks are a reasonable sanity check, but targets should be driven by what success will really look like — where does a particular metric need to be in order to justify the investment required to get there?
  • If targets are solely driven by benchmarks, then it’s an easy (if faulty) deductive leap to believe that, in the absence of a benchmark, no target can be set

So, resolved: benchmarks are not targets.

The Benchmarks We Most Want Are the Ones We Can’t Realistically Have

The easiest, and, in most cases, most relevant and useful benchmarks generally come from your own historical data. If you’re considering an initiative that will improve a certain metric, then your track record with that metrics is a fantastic baseline input into target-setting. Since that data is usually readily available, it gets used. It’s when a totally new initiative is launching — a Facebook page, a mobile app, a community contest — that we get the most anxious about what a “reasonable target” is and, therefore, launch a quest to find benchmarks.

The problem is that these are most often the benchmarks that are least likely to be available. Or, if they are available, there is so much variability inside the data set that it’s hard to put much stock in the data.

Even with something as massively established as email marketing, getting a reasonable benchmark for something as common as open rate has a lot of underlying variables mucking up the data:

  • The type of e-mail — newsletter vs. general promotion vs. targeted promotion vs. something else
  • The target of the e-mail — internal house list vs. rented list, for instance
  • The specific industry and consumer type the emails target
  • The email platform in use and how it captures and calculates open rate
  • The basic deliverability of the emails included in the benchmark, as driven by content, email platform, and user type

If all of these factors are at work with something as established as e-mail, then what does that mean for a relatively knew and evolving medium like social media or mobile? Almost every time we launch a new Facebook page, we get asked what the “benchmark is for new fan growth.” In that case, the single biggest driver of fans — outside of brands that have a massive number of rabidly enthusiastic customers — is the promotion of the page, be it through Facebook advertising, through channels the brand already owns (email database, web site, TV advertising, etc.), or through paid promotion elsewhere. It’s an unsatisfactory reality…but it’s reality nevertheless.

Should We Just Abandon All Hope, Then?

There are some cases where relevant and appropriate benchmarks are available. For instance, Google Analytics provides benchmark data for common web metrics based on sites of “similar size” and in a user-selectable site category/industry. Twitalyzer can be used to gather benchmarks using all of the tracked users who fall into a given “community.” Email marketing platforms often do provide benchmark data by industry, but they can fall short on the critical “e-mail type” front. When benchmarks are available, by all means use them as an input!

In the absence of available benchmarks, meaningful targets can absolutely still be set. It’s just largely a matter of ferreting out stakeholder expectations. Expectations always exist, even if they are claimed to not:

Expectations almost always exist. In the (real) example illustrated above, I pointed out that, if there truly were no expectations, then there would have been no “shock.”

The expectations that exist may not be precise , but, with a little bit of probing, you can generally find a range, below which the initiative will undoubtedly be judged as disappointing, and above which the initiative will certainly be judged a success. Starting with that range and then narrowing down as best you can and getting agreement of this target range from all of the key stakeholders is just smart performance measurement.

Analytics Strategy, Conferences/Community

Guest Post: Success in The Analysis Exchange!

Since Analysis Exchange has been honored with a nomination in the Web Analytics Association Gala Awards, while our community is considering their votes I figured it was a good time to share some of the great email we get from Exchange participants.  This one is from David Schuette who started as a student and has already graduated to mentor!  You can follow David on Twitter @TheCakeScraps and thanks to David and everyone who has benefitted from Analysis Exchange!

If you are in the WAA please consider that a vote for Analysis Exchange is a vote for EVERYONE who contributes to the effort around the world.

A Tale of Two Projects

In the middle of 2010 – 2.5 years into my career as a web analyst – I made one of the better decisions on my journey through the field of web analytics.  A friend of mine, active in the community for some time, pointed me to a project called the Analysis Exchange; he encouraged me to check it out and to sign up as a student.  I did some research and it seemed like a great match.  I would get to help nonprofits and learn a lot in the process.

I’ll be honest; it took a while for me to secure my first project.  I wasn’t sure what the problem was until Eric pointed out (to all members) that a complete profile greatly contributed to the likelihood that a student or mentor would be selected for a project.  I filled it out and started applying again.  At the time there were only a few projects available, in contrast to the 5+ open right now thanks to the hard work of Wendy and team, so it took some time but I was picked to work with Kids Matter, Inc. – an organization supporting foster children.

The experience couldn’t have been better.  Megan, the partner at Kids Matter, was filled with excitement and ambition.  She had done some great work for her organization and wanted to learn more.  She wanted to let the data take away some of the guessing and let it do part of the work for her.

I dove right in and, before long, I had a great presentation that I was able to tweak based on the feedback from my mentor.  The presentation went smoothly and the people at Kids Matter were extremely appreciative of the work.  I even got a thank-you card that was hand-made by one of the kids.  It really made me stop and appreciate just how much good can come from a little time given.

While I was busy working on my first project, the Analysis Exchange kept improving.  The Google Group, a bit quiet recently, contributed in a huge way to make small but important improvements to the Exchange site.  It is cool to look at some of the discussions from just a few months ago and see the ideas already implemented into the site.  It made it all the easier to sign-up for my next project, at the Apalachicola Bay Chamber of Commerce.

The second project went as well as the first.  My mentor and I provided a high-level usability driven analysis to Anita, our partner at Apalachicola Bay.  The analysis focused on opportunities to draw visitors deeper into the site so they could really see what the Apalachicola Bay area had to offer.  Again, our partner was excited about the results and was genuinely appreciative of the work we put in.  It was our pleasure.

And now I have transitioned myself to a Mentor on The Exchange.  If my next experiences are half as good as the first two I would be thrilled.  I’m excited, even anxious, to have the chance to help another organization and provide some coaching to an upcoming analytics ninja.  But I also view this change to a mentor as a re-upping of my commitment to The Exchange; I have made it my goal to bring at least 1 local non-profit to The Exchange this year and hopefully more!

Everything about my experience has been wonderful.  If you have thought about joining, or perhaps have not participated in awhile, go check it out.  You won’t regret it.

Social Media

Twitter Influence — Still Searching for the Perfect Answer

[Updated on 2/17/2011 — added the last section with additional information about Twitalyzer’s Community measurement and a little additional nod to TweetReach.]

A pretty intriguing post from Michael Healy came across the Twitterverse yesterday: #Measuring in 2010 — Analyzing the #measure Data of the Twitterati. What Michael did was take all of the Tweets that used the #measure hashtag in 2010 and run them through an “influence” formula he developed. The tweets data was courtesy of Kevin Hillstrom, who had set up a Twapper Keeper archive of #measure tweets. I’ve set up a couple of Twapper Keeper archives (hopefully, the #emetrics one will continue to function through the upcoming eMetrics conference, as I smell another juicy data set for us to play around with)  in my day and have been a bit frustrated with the quality of the exports — they required quite a bit of cleanup, especially of the timestamps — and, I’ve been a little skeptical of the completeness of the data. But, maybe that’s just because I’ve been working with TweetReach of late, and it’s just so darn clean and robust that the free services really do start to pale in comparison.

I digress.</statementoftheobvious>

Michael’s stated goal was pretty simple:

I wanted to know who were the most influential members of the #measure Twitterverse.

This was an exercise he did as prep work for the Web Analytics Association Spring Awards Gala, which, if you’re going to be in the area, you should plan to attend, as it should be a really good time.

I read the post and had three immediate thoughts:

  • “Influence” is one of the Mid-Major Holy Grails of social media management
  • “#measure” could be replaced by any brand or topic in, and the ability to achieve Michael’s stated goal would come in damn handy in all sorts of situations
  • Michael is one of those Brains with a capital “B,” so it’s worth taking a close look at what he produces when he claims he’s “just having fun.”

The final result was a big ol’ diagram (click on the image to jump over to Michael’s original post and a link to the full-sized image):

Michael’s formula focussed on both the volume of tweets and the “original content” in the tweets (using an Entropy calculation and a slight dampening of the score based on the volume of retweets).

Like Michael, I was surprised to see Szymon Szymanski (@ulyssez) as the dominant circle in the diagram. I’d certainly seen his tweets in the #measure stream, but I would have been more likely to guess @aknecht or @KISSMetrics (which is the good-sized circle off to the right of the diagram, as it so happens) would have had the dominant slot based on volume/variety.

So…Influence, You Say?

Seeing as I’ve been spending a lot of time with Twitalyzer of late, the next thought that popped into my mind was, “I wonder how Michael’s analysis of the top influencers would line up with Twitalyzer’s?”

It doesn’t take much to push that thought further and immediately hit a wrinkle:

  • Michael’s definition of influence was oriented towards tweet volume and tweet content
  • Twitalyzer’s definition of influence is based on an assessment of how likely a user is to be referenced or retweeted

Now, logically, if you have a high tweet volume, and your tweets contain a lot of original content (and, presumably, it’s not navel-gazing content, as that would rarely warrant the inclusion of the #measure hashtag), you’re more likely to be referenced or retweeted. Okay, there’s a logical link there, so maybe that’s not a huge wrinkle.

“Oh, bother,” said Pooh almost immediately, “I think we have a second wrinkle.” That being:

  • Michael’s analysis was based solely on tweets that included the #measure hashtag and the users who tweeted those tweets
  • Twitalyzer’s definition is more “user-based,” and takes into account the user’s direct and indirect network

And, a third wrinkle, just to round out a nice list:

  • Michael’s analysis was based on all #measure tweets from 2010
  • Twitalyzer operates more on a last day, last 7 days, last 30 days mode (with historical data going back much further…but it’s all based on when the user got plugged in as a daily-updated account)

For this third wrinkle, it seems reasonable to assume that the most influential #measure tweeters in 2010 are likely still fairly influential as of the last month.

In the end…does it matter? There’s only one way to know! Let’s take a look!

A Semi-Random Comparison

I’m not going to go through every bubble in Michael’s diagram. But, I am going to hit the “big” bubbles, look up their Twitalyzer Influence scores for the last 30 days, and then do the same for a smattering of small bubbles. For the “small bubble” users I”ve only included users where it looks like Twitalyzer has been doing daily tracking for the last 30 days. And, these bubbles were also a random selection of users that I readily recognized (which, I realize, very likely introduced some sample bias).

Let’s see what we see:

Username Bubble Size Twitalyzer Influence
@ulyssez Ginormous 1.0%
@immeria Huge 1.0%
@analyticscanvas Huge Not available*
@kissmetrics Damn Big 24.0%
@mongoosemetrics Big 5.0%
@thebrandbuilder Big Not available*
@cjpberry Pretty Big Not available*
@usujason Pretty Big 2.0%
@corryprohens Pretty Big 0.0%
@jdersh Pretty Big 1.0%
@minethatdata Pretty Big 3.0%
@hkwebanalytics Pretty Big 1.0%
@johnlovett Pretty Big 2.0%
@jimsterne Small 2.0%
@analyticspierce Small 1.0%
@ericjhansen Small 1.0%
@aknecht Small 4.0%
@tgwilson Small 2.0%
@jojoba Small 1.0%

* These accounts had not yet been Twitalyzed. As such, while I Twitalyzed them, a reliable 30-day average was not available, so I have not included their reported scores here.

So, what does this tell us? Well…seems like we don’t have a perfect correlation (we never do, do we?). From my own use of Twitter, the Twitalyzer scores square pretty well with what I would expect, although, yowza!, I wouldn’t have expected @kissmetrics to be running away from the pack like that!

I don’t think either one of these is “right” in any absolute way. Both approaches were developed with different purposes. Michael’s exercise was, I think, a couple of idle thoughts taken to a logical conclusion. Twitalyzer’s score is one metric inside a measurement platform that offers a whole suite of metrics and that has been evolving and maturing a couple of years.

Does Any of This Really Matter?

I’m drawn to these sorts of exercises because I think they do matter. As web analysts, we got to a pretty consistent definition of a “page view” and a “visit” (“different tools calculate differently” be damned — the basic definition is the same), left things a little loose on “unique visitors,” and never really reached closure on “engagement” (philosophical debates as to whether it even matters notwithstanding).

As social media continues to gain traction with consumers and as social media platforms continue to evolve and mature, we absolutely need to be thinking about measurement within those platforms and we need to keep scrambling to keep up. And, hopefully, maybe we’ll be able to influence the evolution of those platforms so that they’re at least somewhat measurement-friendly. As long as we’ve got analysts pushing the tools and experimenting with new approaches (another example: @jojoba’s oxygenating alter ego posted her Social Media Masters Twitter Analytics presentation over the weekend — lot’s o’ tools out there!), we’ll get there!

So, yeah, it matters. The fact that it’s pretty interesting to watch (and maybe even help) some really, really sharp minds in our space try to crack some pretty hard nuts is just added gravy.

Update: Twitalyzer Community Scores

One of the few benefits of being based in the Eastern timezone with a lot of the heavy analytics work occurring on the west coast is that I got to get up this morning with an inbox and comments on a post that went up shortly before I retired for the evening!

Eric Peterson sent me some of his thoughts and pointed me to the Community area under Tweets and Tags in Twitalyzer:

So…now I need to go do some more Twitalyzer exploration and thinking — from the list above, I need to think through the relationship between Participation, Influence, and Attention, methinks. One of the real draws, for me, of Twitalyzer, is that it enables picking a set of appropriate metrics that, together, measure the effectiveness of any particular Twitter engagement approach. The kicker is nailing down which of those metrics are the right fit in any given situation.

Jenn Deering Davis of TweetReach also sent me some TweetReach data on #measure that covers 2011 to date. TweetReach focusses on reach and exposure of tweets (the difference being that reach is “unique people exposed” and exposure is more “raw impressions”). From that perspective:

TweetReach’s approach has a more direct tie to traditional advertising measurement when it comes to tracking “impressions.” But, it also has a lot of other features that can help sniff out influential people on a particular topic — a major differentiator is that its trackers can use boolean logic, which they showcased in the work they did around the Super Bowl ads. It doesn’t really show this off when we’re looking at a community that is defined as tightly as the #measure hashtag.

Again, I say… so many tool…!

 

Social Media

Is Social Media Encouraging Narcissism?

I’m a little worried about us. At first I was really psyched to see a tweet about my friend and business partner Eric appearing the the WSJ for his not-so-small side project, Twitalyzer. I eagerly clicked through from Tweetdeck to read all about the great strides that Twitalyzer was making in the marketplace only to be massively disappointed by the article, Wannabe Cool Kids Aim to Game the Web’s New Social Scorekeepers. This article is all about gaming the social system to increase influence scores from services like Klout and Twitalyzer and to personally benefit from doing so. Is this what we’re training kids to aspire towards today?

Have you Googled yourself lately…?

Okay, just admit it. At one point of another you’ve typed your own name into to Google just to see what shows up. Or perhaps, if you’re like me you’ve even created a proactive alert that informs you every time you or your business is mentioned in media outlets? It’s not that I’m vain, but I want to know when something or someone publishes about me or about our brand. Isn’t this the cost of putting yourself out there today? Social media has accelerated this exponentially.

I don’t fault people like the ones described in the WSJ article for working to improve their social influence scores as long as they’re genuine. It’s smart to understand how rankings are formulated and how you can improve your scores. That makes the difference between individuals who are building their personal brands with an entrepreneurial drive and those who simply aren’t tuned in enough to know how. Done right, that’s commendable. But understanding the system and rigging it to your favor is potentially where we’re headed in this age of social media. It’s an environment where your potential employer will check your Facebook page prior to extending that job offer; and they definitely will follow your Tweets after that offer is extended; and you can bet on the fact that they’ll be watching your social escapades after you’re hired to ensure that you don’t misconstrue ideas that are yours alone with those of your employer. Or heaven forbid you’re passed over for a consulting job because of a low Twitalyzer score, like the story Shel Israel foretells. But, this is business today, I just wonder if we’re encouraging an unhealthy level of narcissism?

What’s your Social Media Credit Score…?

One of the topics I’ve been researching lately is Social Media Profile Management. This started with the whitepaper that I authored for Unica called, True Profiles: A Contemporary Method for Managing Customer Data (download the paper next week) where I explored what it takes to integrate data streams from disparate sources. Yet, while that’s happening on the business side, consumers are in desperate need of managing their own social profiles. Services like Rapleaf, PeerIndex, Klout and Twitalyzer all reinforce the need to know how you’re portrayed as an individual in social circles and how much personal information about you is floating around out there.

Brian Solis talks about this as well in his compelling Lift presentation where he describes the sociology and psychology behind what we do in social media. He mentions that debt collectors are now visiting individual’s Facebook pages to track them down and sometimes publicly humiliate them into paying their debts. That’s absolutely frightening! But it’s a reality of the world we live in.

Managing your social credit score is important and undoubtedly we’ll see a burgeoning slew of services like Identity Mixer and others that allow you to manage what appears in the databases of companies like Spokeo.com and whitepages.com for all to see. You’re already being indexed, ranked and reported on whether you like it or not. I just can’t help from wondering if the way we (or at least some people) operate with the aid social profile management technologies is disingenuous?

What Should You Recommend To Your Business…?

Those of you who know anything about Analytics Demystified recognize that we’re not ones to take data and simply gaze at it in wonderment. We use data to make recommendations. More importantly, we encourage you to do this as well. So for all the measurers of social media out there, take into deep consideration the value you place on influence. I do believe that it’s a meaningful metric and I am optimistic about < foreshadowing > new developments on the horizon from Social Analytics vendors in this area < /foreshadowing >, yet you have to understand what your metrics are made of and how they’re calculated.

That’s the thing that irked me most about the WSJ article was that it implied in the subtitle that all the vendors out there keep their influence rankings secret. Twitalyzer doesn’t do this, in fact they expose all of the factors that go into their calculated metrics for all to see. While some metrics within the Twitalyzer dashboard do rely on scores from other technologies like PeerIndex and Klout, they’re labeled as such with nothing secret about them. I’m not bringing this up to tout the greatness of Twitalyzer, but more so to call out the fact that transparency in the metrics you use and rely on is critically important.

Hopefully, most of you are migrating away from counting measures like fans and followers that offer little more than a measure from an uncalibrated yardstick and adopting business value metrics that actually mean something to your organization. If you are working toward this end — and if influence is a measure that will factor into your marketing efforts — then take the time to see through inflated scores and popularity hounds that are gaming the system. It’s likely that you don’t want these people doing your bidding anyhow. Instead, use measures of success like Impact to correlate influence to action. When you begin to look at your social marketing efforts in this way, you may just find that those with the most “popular” profiles aren’t actually good for your business.

Reporting

Pocket Guide to Identifying Great KPIs

Here’s a quick post sharing a printable reference for establishing clean, clear, and appropriate KPIs for a project. This was something that Matt Coen, one of my peers at Resource Interactive, and I developed in response to some internal requests coming out of a measurement class that we teach both internally and for some of our clients. But, we agreed it was worth sharing with the broader measurement community. The goal was to put something in the hands of analysts or marketers that would actually give a practical guide to the questions to ask when heading into a project to ensure the establishment of effective KPIs up front.

I see this as a complement to one of my favorite Avinash Kaushik posts, which I think I’ve been referencing almost since he wrote it…and I now realize that was over three years ago! The meat of the post is his list of “four attributes of a great metric” (they’re Uncomplex, Relevant, Timely, and “Instantly Useful”). I see this guide as a guide to how to ask the right questions such that you wind up with great KPIs (it works for non-KPI measures, too, but the focus is on KPIs specifically). It’s not rocket science by any means, but it’s handy! Click on the image for a larger version, but see below if you want to print it.

Guide to Great KPIs

A small (half-a-page), black-and-white version of the guide is available in this PDF. The PDF actually has the same diagram twice. Print it, cut it in half, and pass the second diagram along to a colleague who might find it handy!

What do you think? What’s missing?

General, Social Media

Measuring the Super Bowl Ads through a Social Media Lens

Resource Interactive evaluated the Super Bowl ads this year from a digital and social media perspective — how well did the ads integrate with digital channels (web sites, social media, mobile, and overall user experience) before and during the game. I got tapped to pull some hard data. It was an interesting experience!

A Different Kind of Measurement

This was a different kind of measurement from what I normally do. I definitely figured out a few things that we’ll be able to apply to client work in the future, but, while, on the surface, this exercise seemed like just a slight one-off from the performance measurement we already do day in and day out, it actually has some pretty hefty differences:

  • Presumption of Common Objectives — we used a uniform set of criteria to measure the ads, which, by definition, means that we had to assume the ads were all, basically, trying to reach the same consumers and deliver the same results. Or, to be more accurate, we used a uniform set of criteria and then made some assumptions about the brand to inform how an ad and it’s digital integration was judged. That’s a little backwards from how a marketer would normally measure a campaign’s performance.
  • Over 30 Brands — the sheer volume of brands that advertise at the Super Bowl introduces a wrinkle. From Teleflora to PepsiMax to Kia to Groupon, the full list was longer than any single brand would normally watch as its “major competitors.”
  • Real-Time Assessment — we determined that we wanted to have our evaluation completed no later than first thing Monday morning. The reality of Marketing, though, is that, even as there is a high degree of immediacy and real-time-ness…successful campaigns actually play out over time.  In this case, though, we had to make a judgment within a few hours of the end of the game itself.
  • No Iterations — I certainly could (and did) do some test data pulls, but I really had no idea what the data was going to look like when The Game actually hit. So, we chose a host of metrics, and I laid out my scorecard with no idea as to how it would turn out once data was plugged in. Normally, I would want to have some time to iterate and adjust exactly what data was included and how it was presented (certainly starting with a well-thought-out plan of what was being included and why, but knowing that I would likely find some not-useful pieces and some additions that were warranted).

It was a challenge, for sure!

The Approach

While the data I provided — the most objective and quantitative of the whole exercise — was not core to the overall scoring…the approach we took was pretty robust (I had little to do with developing the approach — this is me applauding the work of some of my co-workers).

Simply put, we broke the “digital” aspects of the experience into several different buckets, assigned a point person to each of those buckets, and then had that person and his/her team develop a set of heuristics against which they would evaluate each brand that was advertising. That made the process reasonably objective, and it acknowledged that we are far, far, far from having a way to directly and immediately quantify the impact of any campaign. Rather, we recognized that digital is what we do.  Ad Age putting us at No. 4 on their Agency A-List was just further validation of what I already knew — we have some damn talented folk at RI, and their experience-based judgments hold sway.

For my part, I worked with Hayes Davis at TweetReach, Eric Peterson at Twitalyzer, and my mouse and keyboard at Microsoft Excel to set up seven basic measures of a brand’s results on Twitter and in Facebook. For each measure, there were either two or three breakdowns of the measure, so I had a total of 17 specific measures. For each measure, I grouped each brand into one of three buckets: Top performer (green), bottom performers (red), all others (no color). My hope was that I would have a tight scorecard that would support the core teams’ scoring — perhaps causing a second look at a brand or two, but largely lining up with the experts’ assessment. And, this is how things wound up playing out.

The Metrics

The metrics I included on my scorecard came from three different angles with three different intents:

  • Brand mentions on Twitter — these were measures related to the overall reach of the “buzz” generated for each brand during the game; we worked with TweetReach to build out a series of trackers that reported — overall and in 5-minute increments — the number of tweets, overall exposure, and unique contributors
  • Brand Twitter handle — these were measures of whether the brand’s Twitter account saw a change in its effective reach and overall impact, as measured by Twitalyzer; Eric showed me how to set up a page that showed the scores for all of the brands we were tracking, which was nifty for sharing.
  • Facebook page growth — this was a simple measure of the growth of the fans of the brand’s Facebook page

The first set of measures were during-the-game measures, and we normalized them using the total number of seconds of advertising that the brands ran. The latter two sets of measures we assessed based on a pre-game baseline. We used Monday, 1/31/2011, as our baseline date. Immediately following the game, there was a lot of manual data refreshing — of Facebook pages and of Twitalyzer — followed by a lot of data entry.

As it turned out, many of the brands came up short when it came to integrating with their social media presence, which made for a pretty mixed bag of unimpressive results for the latter two categories above. Sure, BMW drove a big growth in fans of their page, but they did so by forcing fans to like the page to get to the content, which seems almost like having a registration form on the home page of a web site in order to access any content.

The Results

In the end, I had a “Christmas Tree” one-pager: for each metric, the top 25% of the brands were highlighted in green and the bottom 25% were highlighted in red. I’m not generally a fan of these sorts of scorecards as an operational tool, but, to get a visual cue as to which brands generally performed well as opposed to those that generally performed poorly, it worked. It also “worked” in that there were no hands-down, across-the-board winners.

What Else?

In addition to an overall scoring, we captured the raw TweetReach data and have started to look at it broken down into 5-minute increments to see which specific spots drove more/less social media conversations:

THAT analysis, though, is for another time!

General

Should Google Offer a Paid Version of Google Analytics?

Recently there has been some rumor buzz about Google releasing a “paid” version of Google Analytics (beyond what is currently available through Urchin). Assuming, for a second, that something like this is coming in the future, the real question is whether this is a good or bad idea. In this post, I’ll examine some of the pros and cons to this potential move by Google.

Why Google Should Offer a Paid Version

So what are some of the reasons that Google should offer a paid version of its web analytics offering? I can think of the following:

  • There will always be a group of web analytics users that want advanced functionality and are willing to pay for it. These advanced features are often resource-intensive and I could see Google wanting to recoup some money to enable these features or the additional data storage they necessitate.
  • There are millions of websites using Google Analytics for free and if Google can extract even a small amount of revenue from these, it can add up quickly. Since I don’t think Google is hurting for revenue, I assume that the money generated would be filtered back into the product which would mean even more enhancements to a product that pretty robust already.
  • One of the reasons Google may be thinking about offering a paid version of the product is to open the door to its sales team to cross-sell other Google products and services. By being free, Google Analytics has infiltrated millions of websites which creates an easy entrée for a Google sales rep to say: “I see that you are using Google Analytics, did you know that Google also offers Google Ad Words, Google Apps, etc…” While they can already do this, if a company has already started paying for Google Analytics (and it has made it through procurement!), that makes the cross-sell so much easier. It also helps weed out the companies that are serious, which will often be the ones willing to pay.
  • Services baby! It is no secret that professional services are a huge money maker. When I was at Omniture, we had a sizable consulting group and there are a host of other firms (including Analytics Demystified of course!) offering services around web analytics. While I am not sure if it would be a good move or not, Google could offer paid-for services around a paid-for web analytics tool itself or through its certified partners.
  • Competition! I love competition. I think it helps drive innovation. In my opinion, the consolidation of the web analytics industry over the last few years has reduced the amount of innovation and I think Google having a paid product will ultimately mean that everyone in the industry gets more.

Why Google Should Be Careful About Offering a Paid Version

So what are the pitfalls that Google might want to look out for? Here are a few worth considering:

  • Too much functionality! One of the strengths of Google Analytics is its simplicity. Since it is a free tool for most users, it has not been beholden to the axiom that more features must always be added to continue justifying the investment. Like all software products, as time goes by, more features are added to meet the needs of the most advanced users, which often results in casual users leveraging 10% of the functionality. While it looks like Phil & Nick have done a great job adding the features their users want to date, once someone is paying you money, the balance of power tends to shift in a big way (think difference between privately held vs. publicly traded company). I hope that Google will not lose its simplicity “mojo” that got it to where it is today.
  • Customer Support? One of the biggest expenses for software products is the cost associated with supporting its customers. When I worked at Omniture, we had a massive customer support organization of account managers and client care that grew exponentially. If Google has paid clients, I would imagine that it would need to provide support at a level that far exceeds what it is offering today. This is not an easy task and Google is known for being somewhat hands off for most of its products. When your product is free, people accept that they are going to be on their own more than when they are paying for something and if support isn’t good, I could see Google Analytics losing a bit of its current luster. I also imagine that Google loses quite a bit of money on Google Analytics (which I assume it makes up for on the AdWords side), and this will be even worse once it has to staff up to support users unless it can find a way to get its partners to offer that support.
  • SLA’s (Service Level Agreement). Paid-for vendors have legal requirements around the availability of the product and the handling of product issues. To date, it is my understanding that Google Analytics has not had SLA’s since it is a free product, but I would imagine Google would need to provide a reasonable SLA for the paid side. SLA’s are never fun and usually end up costing time and money…
  • What happens if no one buys it? Google has done a lot of things that have changed the market and some that have not done quite as well (i.e. Google Wave). Google shook up the web analytics industry in a huge way with free Google Analytics, but what would it say if only a small % of companies decide to pay for its product? Does this serve as a boost to its paid competitors? I guess the real question comes down to this. If I am a Fortune 500 company and am currently using Google Analytics and a paid product from Omniture, Webtrends, Coremetrics or Unica (which is very often the case!), what features will Google Analytics add to its paid product that will get me to only use Google Analytics and get rid of my other paid vendor? I would guess that the things I would be looking for are 1) my own dedicated servers so I know my data is really my data and can be kept as long as I want, 2) knowledge that Google is not seeing any of my data and using it in its search algorithms, 3) support and SLA’s at the same caliber I am getting from my other paid vendors and 4) 90% of the features I can get from my other paid vendor. If Google can deliver on these items (and I am sure it can), I think it will make a compelling case as to why companies should standardize on Google Analytics, but I don’t think this will be something that happens overnight.

Obviously, all of this is still speculation, but I, for one, look forward to seeing what Google does and how they address some of the items I have described here.

I highly recommend you check out this YouTube video on disruptive innovation. I think it is very cool to watch this and think about Google being the “entrant” and the other paid web analytics vendors as being the “incumbents” described in the video. This video talks about what Google has done to the other paid vendors and how Google could one day become the incumbent and fall prey to even newer entrants (or reincarnations of the old incumbents!). Fascinating stuff!

So what do you think? Will they do it? Will people buy it? What things do you think Google needs to do to make it successful? Please share your thoughts by adding a comment here…

Analytics Strategy, Conferences/Community, General

A few thoughts on the upcoming WAA Awards

I got a nice note this morning from Mike Levin at the Web Analytics Association:

“CONGRATULATIONS! You have been nominated for a WAA Award of Excellence in the category of: Most Influential Industry Contributor (individual) Your nomination recognizes the contributions you and/or your company have made to the web analytics industry. It is an honor to be nominated and the WAA congratulates you on your success. “

While I am honored by the recognition and delighted to have been nominated I told Mike that I am declining to participate in the voting.

Mike wrote me back and seemed surprised but my thinking is very simple: I have been very fortunate in my web analytics career and have received lots of recognition from my peers, my clients, and the press. I’m not one to bang my own drum and brag about my accomplishments … I prefer to just do my thing, help my clients and the community, and build a strong company for my partners and associates.

So I humbly and politely decline the honor and instead will cast my vote for folks I believe to be truly deserving of an industry honor. Here are the people I will be voting for:

  • Web Analytics Rising Star: Jason Thompson.  Jason is still a bit rough around the edges but I love his style and commitment to getting things done.  If I can vote twice I am voting for Michele “Jojoba” Hinojosa … her passion is palpable and her enthusiasm is infectious.
  • Most Influential Industry Contributor: John Lovett. I’m not sure John is actually eligible because he is on the WAA Board but his work on the WAA Code of Ethics is a monumental achievement and one that has the potential to shape our industry for years to come.  If I can vote twice my second nod goes to Jim Sterne … who has done more for this industry than Jim Sterne?  Damn right, nobody!
  • Most Influential Vendor: Google.  Most of the positive changes we have seen in the past two years in web analytics can be derived either directly or indirectly to the work that Brett Crosby and the team at Google Analytics put out there.  Second vote goes to Omniture given the critical mass they have been able to create and the big strides they made since the Adobe acquisition on customer support and overall focus.

UPDATE: OMG I didn’t realize that Corry Prohens was running a shameless and ruthless campaign to win the “Influential Agency/Vendor” award.  You should read his “shameless campaign” blog post and consider voting for Corry.

  • Client/Practitioner of the Year: Best Buy. Difficult to not vote for one of your own favorite clients but I hope you will all come to my keynote presentation with Lynn Lanphier at Emetrics and hear why I cast this vote.  Second vote? Dell, for taking the advice I gave them last year to heart and who are now kicking ass and taking names for testing and optimization. Bravo!
  • Technology of the Year: Analysis Exchange. Now, of course, I’m not really going to vote for something I helped create, but I am pretty damn proud of the work we have done and with Wendy Greco at the helm things are only getting better.  If I could vote twice … I wouldn’t, because I’d be tempted to vote for Twitalyzer LOL!

Again, I do appreciate the nod from the WAA and am looking forward to the party — the Analytics Demystified and Keystone Solutions crews will be there in force. I wish everyone nominated for the WAA awards the best of luck and, as a native of Chicago, remember to vote early and vote often!

Don’t forget to nominate your favorite web analytics superstar!

Adobe Analytics, Analytics Strategy, Conferences/Community, General

Conference Season is Upon Us

Wow, I just got done looking more closely at the Analytics Demystified team calendar for the next few months and it is a doozy! Chances are if you live in the U.S. and do any type of digital measurement, analysis, or optimization professionally we are going to see you between now and the end of March.

If that is the case, we’d like to buy you a drink!

Despite each of us presenting, often multiple times, we are always happy to make time for our clients and potential clients when we are out-and-about.  If you realize you’re going to be at one of the following events why not drop us a line and we’ll see if we can connect. Who knows, maybe we’re planning a great party or something …

After all that the three of us are going to slink home to our loved ones and try and convince them we are in fact their fathers, husbands, and sons.

Seriously, though, we never get enough opportunities to meet with partners, friends, and prospects at these events so if you’d like to meet with any or all of us please drop us a line sooner than later so that we can block time and make plans.

Reporting

How Marketing is Like Homelessness

I’ve officially succumbed to the Blog an Intriguing Title Syndrome (BITS). My payback for that, I suppose, is that I’ve blown the SEO power of my <h2> tags, my <title> tag, and keywords in the URL such that almost certainly no one who would actually be interested in this post will find it via Google or Bing. So it goes.

But, the title isn’t a pure gimmick. It’s the outgrowth of one of those, “I bet I’m the only person sitting in this church hall with 50+ other people at 3:30 AM who is having this thought,” moments. We all have those moments occasionally, right? Right?!

Gilligan, Are You Drunk? WHAT Are You Talking About?

The basic thought: Marketing is like homelessness, in that they face similar challenges when it comes to measurement.

Earlier this week, I participated in an annual homelessness count in downtown Columbus coordinated by the Community Shelter Board (CSB), which is an organization that drives coordination, collaboration, and consistency across the various homeless shelters in the area. It’s been nationally recognized as a model for how communities can efficiently and effectively meet the basic needs of the homeless. As it turns out, they’re also an organization that does a great job of measurement (which, I now realize, I’ve discussed before).

One of the questions that CSB tries to answer for the community is, “Are we reducing the number of people who are homeless over time?” It turns out that that is a pretty tough question to answer. CSB can certainly track how many beds are filled each night in the various shelters they work with, but those shelters tend to pretty much run at capacity and find creative ways to adjust their capacity as needed so they seldom turn people away. And, the weather affects how many people seek shelter on any given night. So, it’s messy to measure the true change in overall homelessness. That’s sort of like measuring marketing.

Whopper of a Disclaimer: I’m going to spend the rest of this post comparing measuring homelessness to measuring marketing. I’m pretty passionate about both, but the latter pays the bills, while the former actually has a degree of Noble Purpose attached to it. I am in no way comparing the the marketing profession to the group of underpaid and overworked people whose careers are dedicated to reducing homelessness. There is simply no contest there.

Marketing and Homelessness, Huh?

The way that we measure marketing at the highest level is often by measuring revenue, profitability, brand awareness, brand affinity, etc. These are all messy to measure in one way or another, and some of them are expensive to measure, too! It turns out, measuring homelessness is the same way.

So, there I was at 3:15 AM in a church hall waiting for all of the volunteers to arrive. Each team in the hall was made up of 5-6 people, and each team was assigned a different area of the city to physically walk around counting the homeless in that area. It’s not that it takes 5-6 people to do the counting, but there is safety in numbers. I was pretty much just one of “the numbers.” My team’s leader was Dave S., who I’ve known for several years, and whose team I explicitly asked to be assigned to. I mean, if your pre-count pep talk includes flashlight under the chin, how can you not be inspired?

Scary Leader Dave

The Outcome Alone Isn’t All That Helpful

So, we headed out and did our counting. For our group, our total homeless count was: zero. Does that mean that we’re solving homelessness in Columbus? Of course not. That might be the case (we certainly hope it is), but we were only providing one input to an overall count that included the other teams, a shelter census, and self-reported “homeless-but-not-somewhere-you-could-count-me” (i.e., a car) data. And, there was a lot of construction under way in our area, which doesn’t make for conducive overnight outdoor stays, so we were not all that surprised with what we found (two of the members of our team had covered the same area last year, and they did count a handful of people).

Marketing Analog: It’s messy to measure overall marketing outcomes, and it’s almost always impossible to draw a meaningful conclusion from a single data set. In the world of digital and social media, we don’t want to go crazy and try to assess an unorganized and overwhelming sea of data, but we do want to deliberately plan and measure using different tools and sources as appropriate to get as clear a picture as possible of a messy world.

Regardless of how the final tally turns out on the homeless count, having a solid annual measure of the key outcome we’re hoping to change is just the frame around a rather intricate and involved picture. Homelessness, like marketing results, are impacted by myriad  underlying factors. The most commonly recognized causes of homelessness are:

  • The economy — when a local economy is down, there is less prosperity, and the “barely keeping our heads above water” populace become the “drowning” populace
  • The availability of jobs with a living wage — related to the economy, but includes issues such as job skills and quality of the local public education system
  • Mental illness — without access to mental health services and medications, it can be impossible for many people to maintain a stable life
  • Drug and alcohol abuse — often, this goes hand in hand with mental illness, but, even when it doesn’t, once an addiction has set in, wheels can rapidly fall off the steady income wagon
  • Personal catastrophe — a health crisis of the individual or a family member often wrecks limited savings and can draw a person away from his/her job, which triggers a spiral that, ultimately, ends on the streets

It’s a daunting challenge to address all of these, and it’s even more daunting to try to disentangle which of these issues are interrelated and to what degree — both for an individual and at a macro level.

Marketing Analog: Trying to tease apart how economic factors, cultural trends, competitor activity, TV advertising, print advertising, radio advertising, web site content, SEO and SEM, Facebook, and Twitter all interact with each other to affect marketing outcomes is daunting and messy. The fact is, we need to effectively use multiple channels, and we need to identify cross-channel effects and measure those as best as we can. But, it’s not easy, and it’s definitely not perfect.

I’m starting to feel a little silly making this comparison, but, having volunteered on a number of “basic needs” (the lower levels of Maslow’s Hierarchy) committees over the years, it’s been interesting to watch how often the question gets asked: “What’s the single root cause that we can address to have the biggest impact?” The answer? There isn’t one. It’s got to be a multi-faceted approach. And, it’s also a fact that no one person or group can address all of the facets at once.

As it happens, one of the other volunteers on my count team was Matt K., who is the United Way of Central Ohio staff member now responsible for the main United Way committee on which I’ve been a member for the past few years. As we walked along the Scioto River early Tuesday morning, we chatted about how hard it was to identify a clean set of “leading indicators” as to whether we were making progress in our assigned community impact area: emergency food, shelter, and financial assistance. I told Matt that he and I were living in similar worlds — we both are supporting people with expectations and desires for easy, accurate, and accessible measures of something that is very complicated and messy!

Planning Is Important

One final point: our assigned area was a mile or two away from the church where we started…and no one had a car that would easily fit six people. Luckily, it was relatively warm (mid-30s), and the back of my truck was relative dry, so four of us piled into the front, while Matt K. and  Joe M. (also of United Way) climbed into the back of my truck:

Matt and Joe Ready to Ride to the Count Area

Now, had I known we would need to transport six people ahead of time, I easily could have swapped vehicles with my wife for the day and brought along a minivan rather than a small truck. But, we didn’t coordinate that up front. We didn’t fully plan for our measurement.

Marketing Analog: Planning does matter. It doesn’t mean that, without planning, you can’t gather some data, but you may not get the data you want, and it may be a little more painful (or at least chilly) to get the data you need.

I guess, as an analyst, I see data challenges all around me. I also have lower expectations for the quality and completeness of data, I’m more comfortable with wildly imperfect proxy measures, and I expect gathering meaningful data to be a messy process.

Whether counting the homeless or counting web site conversions, though, it’s definitely a whole lot more pleasant to do it with fun and interesting people. I’ve been pretty fortunate on that front!

Analytics Strategy

The Web Analyst’s Code of Ethics

The Web Analyst’s Code of Ethics is a reality! This Code represents an industry effort to promote ethical data practices and treat consumer data with the respect and attention it deserves.

I’m writing this on the eve before the official launch announcement of the Web Analyst’s Code of Ethics here at the WAA Symposium in Austin Texas. As you can see in the video above, this effort is the culmination of a ton of hard work by a community of contributors.

Yet, the conversation isn’t a new one. My partner Eric has been writing about the fact that We are our own worst enemy since August and our internal conversations about privacy regulation and public opinion of tracking practices have been going on long before that. The issue received mainstream attention from the Wall Street Journal in their What They Know series, which took a bias view in our opinion. Anything that starts out with the phrase; Marketers are spying on Internet users… is FUD in my opinion.

So, in September of last year we decided to do something about it. I must say that Eric never fails to amaze me in his ability to make things happen, because not 24 hours after our conversation about launching a Code of Ethics, he had one drafted and in my inbox. We decided that the best avenue for getting this code out to the community was to work in conjunction with the WAA, where I am a member of the board. Thus, I shopped it around to my fellow board members and we all agreed that it was something that our industry needed. The issue was brought before the WAA Standards Committee and a sub-committee was formed to hash out the details. And the Code was offered to the community for public comment. After numerous iterations and literally dozens of comments and contributors, we arrived at the final Code you see here.

It’s important to recognize that this Code is a pledge for individuals and not organizations. We created it as such because we know that not every individual will be able to enforce policy within their company, but every individual can inform and educate their peers. Yet, as we state in the pledge itself, “I recognize that we are far stronger as a community…”. And this effort is about a community showing it’s commitment to ethical data collection and utilization practices.

Momentum for this project has been incredible thus far, but our work is far from over. It’s just beginning. Like any good analyst, I’ve created goals and success metrics for the code of ethics that I’ll be tracking and reporting on over time. The video above is the first effort to share a glimpse of the metrics, but ultimately I’m shooting for the following goals:

      1) Gain 1,000 Pledges to the Code of Ethics in 2011

 

      2) Attract mainstream media attention to this community effort within the first 90 days of launch (e.g., recognition by @WhatTheyKnow)

 

    3) Ensure that our collective voice is heard by legislators and policy makers before regulation is forced upon us

Let us know what you think about the Code of Ethics here by leaving comments and joining the conversation. Or simply show your support by pledging to follow the Web Analyst’s Code of Ethics.

Adobe Analytics, Analytics Strategy, General

Free webcast on Tag Management Systems on Jan 25th

Given the considerable buzz in the marketplace regarding Tag Management Systems and vendors like Ensighten, TagMan, and BrightTag I wanted to call your collective attention to a free webcast I am participating in next week on “The Myth of the Universal Tag.” On Tuesday, January 25th at 1:00 PM Pacific time I will presenting with Josh Manion, CEO of Ensighten and Brandon Bunker, Senior Manager of Analytics at Sony, detailing some of the advantages I see in the adoption of a tag management platform.

What’s more, the nice folks at Ensighten have taken the registration form off of my white paper on tag management systems and so everyone is free to read all of my thoughts on Tag Management without prompting a sales call.  How cool is that?

Spread the word:

“The Myth of the Universal Tag” free webcast sponsored by Ensighten
Tuesday, January 25th, 1:00 PM Pacific / 4:00 PM Eastern
Register online now at GoTo Meeting!

Don’t forget to download that free copy of my white paper on tag management systems!

Adobe Analytics, Analytics Strategy, Conferences/Community, General

Want to meet Adam Greco? Go to OMS 2011 in San Diego!

By now I hope you have heard that Adam Greco is joining John and I as a Senior Partner in Analytics Demystified. While his official start date isn’t still for a few weeks he’s already on the road as part of the Demystified team. If you’d like to meet Adam in person and talk with him about the practice he is building there are a few places I just happened to know he will be in the coming months:

  • Adam will be participating in the Web Analytics Association (WAA) Symposium in Austin, Texas on Monday, January 24th. Adam is talking about integrating web analytics and CRM which is core to his practice area given his past work at Salesforce.com and Omniture.
  • Adam will also be presenting at the Online Marketing Summit in San Diego, California on Tuesday, February 8th. He’ll be giving the same presentation on web analytics and CRM, discussing how to move marketing analytics from the server room to the board room.
  • Adam will also be joining me in Minneapolis on Wednesday, February 16th for a special Web Analytics Wednesday sponsored by our good friends at SiteSpect and with the generous help from our friends at Stratigent.  We don’t have the details on the site yet but the event will be downtown Minneapolis and Adam and I will be doing some prognostication and fielding questions from Twin Cities locals.

Adam will also be at Webtrends Engage, Adobe’s Omniture Summit, and the Emetrics Marketing Optimization Summit but we’ll post more on that when additional details emerge.  Suffice to say Adam will be busy in his first few months on the job.

If you haven’t met Adam I would encourage you to head out to one of these events and introduce yourself. Especially if you’re a marketer and are considering the Online Marketing Summit — if you haven’t been to OMS you really need to go.  Every year I am absolutely blown away by the job that Aaron Kahlow and the OMS team do bringing that conference together.  OMS draws amazing speakers, amazing sponsors, and most importantly amazing conference participants and delivers an absolute fire-hose of information.

I’m sincerely bummed that Adam is taking my place at OMS this year — I haven’t actually missed a big OMS event in California ever — but I am confident that the audience will benefit greatly from Adam’s message about CRM integration, his direct experience at Salesforce.com, and his distinct presentation style.

Social Media

Dear Facebook: As an Analyst, It’s Hard to Be Your Friend

Update: More Facebook Insights updates rolling out, and, so far, they are buggy: Facebook Quietly Updates Insights to Show Real-Time Data on Page Posts, Bugs Appear.

Dear Facebook,

I really want to stay your friend, you see, but I’m an analyst. I’m someone who daily gets asked by marketers: “How do I know if my Facebook investment is paying off?” They want to be your friend, too, but you sure don’t make it easy.

For starters, I couldn’t figure out where to send this note. And, honestly, I’ve never been able to figure out how to actually contact you. It seems kinda’ silly — you do a fantastic job of helping people interact with each other, and you do a lot to enable brands to engage with consumers. Yet, you don’t make it very easy for us data types to engage with you. It would be one thing if there were a handful of uber-analysts who had an “in” with you, and if those folk were out there chiming in to the myriad aborted threads of frustrated analysts trying to extract meaningful data from your systems, that would be one thing. But there aren’t.

Alas! This note is destined to be a bit of a pissy rant. You can accuse me of using social media in all the wrong ways, of launching a Festivus-style airing of grievances. All I can say is that I’ve been around long enough to know that I should not post in anger, but, rather, should pen this note and then let my heels cool overnight. I did. Over a couple of nights, actually. And it still seemed like the right thing to do.

You see, I’ve been doing this web analytics thing for a while now. I lived through the maturing of the industry from “counting hits” all the way to “measuring conversion, segmenting traffic, and testing and optimization experiences.” As an industry, we’ve learned a lot on that front, but, Facebook, you seem hell-bent on reinventing the wheel, and, so far, your wheel looks like a square drawn by a drunken monkey. I want to be able to cleanly measure who’s interacting with my brand within Facebook and how they are engaging with my content. I don’t want to know names and e-mail addresses, but I sure would like to know how first-timers with my brand engage with me as compared to long-time fans. I want to be able to segment my fans and analyze their behavior by segment. Each successive Facebook Insights rollout gets a lot of buzz, but that buzz tends to turn out to be a swarm of horseflies…circling a fresh cow pie. And that stinks.

I don’t know if you know this about me, Facebook, but I was a technical writer early in my career. That’s made me a few things: 1) a pretty fast typist, 2) a guy who occasionally does RTFM, and 3) someone who expects formal documentation to be pretty pristine and comprehensive. With that in mind, let me show you what happens when I go to Facebook Insights and click the Export button. This is the box that pops up:

Seperated? SepErated?!! Maybe you don’t have copy editors on your dev team, but that’s just embarrassing, and, it turns out, an indication of deeper issues. I killed several hours trying to get Excel 2010 + PowerPivot + OData working to “sync data with Excel,” and I overcame several hurdles before running into a brick wall a hundred feet from the finish line. One of these days, maybe I’ll fully crack the code and can write a nice guide on how to make that work — it would be a handy capability — but your single page of documentation simply blithely points to dead ends! Never mind that Excel 2010 is far from a mass-adopted tool (and let me point out that all major web analytics vendors, including, Google Analytics, which is every bit as free as Facebook, have Excel integrations that are well-documented and work with multiple versions of Excel). I’m an analyst — not a programmer. That means I’m reasonably technically savvy, I can generate, hack up, and even do a little debugging of VBA and Javascript here and there. But, I’m not really equipped to jump into a poorly documented API to start extracting data. I need a little help, and you simply do not provide it.

Let’s say I just export an Excel file manually, though. At least, finally, you provide daily data for a few more metrics so that I can do some roll-ups and trending. Now, mind you, I still can’t get trended data for individual custom tab traffic without jumping through a painful number of scroll-and-click hoops, and that’s a pretty run-of-the-mill need. But I digress. I’ve exported my Excel file and I’m checking out the Key Metrics tab. I’m not going to even bother to quibble with how on earth you could know what my key metrics are, or the fact that you provide 18 “key” metrics. Let’s put that aside and, instead, just take a close look at the first three columns of data and the metric names and descriptions provided:

  • Daily Active Users — Daily 1 day, 7 day, and 30 day counts of users who have engaged with your Page, viewed your Page, or consumed content generated by your Page (Unique Users)
  • Weekly Active Users — Weekly 1 day, 7 day, and 30 day counts of users who have engaged with your Page, viewed your Page, or consumed content generated by your Page (Unique Users)
  • Monthly Active Users — Monthly 1 day, 7 day, and 30 day counts of users who have engaged with your Page, viewed your Page, or consumed content generated by your Page (Unique Users)

Wha…?!!! Keeping in mind that each of these metrics has a single column of data that has a value for the metric for each date…what the HECK is a “Monthly 1 day count of users?” I guess I can make an assumption that this was just some of the sloppiest bit of documentation ever written (maybe it was those drunken monkeys again?), and that Daily Active Users are, for each day, the number of unique users who “engaged with the Page” (more on that in a minute) on that one day; that Weekly Active Users are, for each day, the number of unique users who engaged with the page over the prior 7 days (so it’s a rolling 7-day count); and that Monthly Active Users, for each day, are the number of unique users who engaged with the Page over the prior 30 days (so it’s a rolling 30-day count).

Unfortunately, that’s not what the definitions say. What the definitions say is…gibberish.

But wait! There’s more! Let’s look at “…or consumed content generated by your Page.” That’s, like, three multi-syllable words put back to back, which, seemingly, indicates a coherent command of the language. Alas! It’s actually a pretty vague statement. Again, I have to make an assumption that this means, “any user who generated an impression by having a status update by the page render in their news feed.” If that’s what it means, then why not say so? And, if that’s what it means, should that really count as an “active” user? Sure, “engaging with your Page” (my assumption being that that is a Like of or a Comment on content from my page) is a sign of “Active,” as is visiting the page itself (“viewed your Page”), but an impression? Hardly. Unfortunately, I can’t carve that out and use a metric definition that makes sense for me.

The vagueness of the documentation points to a larger issue of transparency as to the mechanics of how you capture and report data. With Google Analytics, Sitecatalyst, Coremetrics, Webtrends, Twitalyzer, Localytics, Flurry, and other analytic tools, I can roll up my sleeves, dive into the documentation and the interwebtubes, do a little experimentation, and wind up with a fundamental understanding of how bits and bytes are flying around to capture the data. While these sorts of underlying mechanics aren’t something that the  business users I support need to understand, it’s critical for my ability to translate the business questions they ask into the interpretation of the reporting and analysis I do. If I had a nickel for every time I had to say, “Well, it’s pure guesswork as to how Facebook is actually capturing and counting that (video views is a biggie there),” I’d have a nice chunk o’ change that I could transfer to an offshore account and then buy a little piece of Facebook. I don’t have those nickels, though, so I’ll settle for just having you pull back the covers a bit and share your data capture mechanisms and data model.

That actually leads to a real head-scratcher on some data you don’t provide. Call me crazy if you must, but I actually care if people are spreading my content to their social graph. You know how they do that? Of course you do! You were instrumental in bringing the concept of “Share” into the mainstream! Yet, you provide no native reporting on share volume (much less segmentation of who shares, or any indication of the lifecycle of a share)! I can get basic content Share counts for content that I manage through Vitrue, but I’m not running Vitrue on all of the pages I work with.

Don’t even get me started on the random nonsensical holes in your data — “my page plummeted from hundreds of thousands of fans to zero fans for two days and then mysteriously returned to its pre-plunge levels!” — or the firm commitment you’ve made to have data available within 48 hours <choke!> of the activity occurring. 48 hours? It’s a real-time world, baby, and, even if “real-time” doesn’t truly need to mean absolute zero latency, 48 hours is ridiculous.

Now, a workaround that occurred to me back in late 2009 was to simply give up on Facebook when it came to getting the data I wanted and, instead, to just deploy web analytics code on my pages. But, you made it clear from the get-go that you had no interest in me bringing anyone else into this relationship, even if they could totally offer something that you’re not interested in providing. Javascript only runs in the narrowest of circumstances, your image caching stymies many workarounds to that limitation, and, even when I am successful, you manage to make me feel vaguely dirty about my success, like I’m doing something wrong. I’m not. I just want to understand how people are engaging with my pages!

I could go on and on. Unfortunately, I don’t have time to make this note shorter, and I do apologize for that. I’m going to go hang out with some Twitter data for a bit to calm down. Maybe, while I’m out, you could take a good hard look at the way you’ve been treating me? A few months ago, Brian Clifton predicted that, in order to survive, Webtrends needs to get acquired, and he suggested that Facebook would be be a good suitor. When I initially read that, I thought it was a pretty “out there” idea. I don’t think that any more. You need to get help. You need a friend, and having some seasoned web analysts and web analytics developers sharing their thoughts and ideas with you would really help your and my relationship with each other.

Facebook, as a user, I am your friend. And I’m loyal. You give me a lot. It’s as an analyst that I’m being forced to remain your friend, even though I soooo Unlike how you reciprocate.

Best regards,

Gilligan on Data

General

weThink Podcast — Digital Trends and What They Mean for Marketers

Matthew Santone, Dan Shust, Chris BarcelonaLast fall, I started listening to the weThink podcast (here’s the iTunes link) that is unique in that I personally know all of the people who work on it. They’re some of my co-workers at Resource Interactive, and most of them are part of the RI Lab — our “R&D” wing: Matthew SantoneDan Shust, and Chris “Barce” Barcelona, with Lisa Richardson as the moderator. They’re go-to folk when it comes to what’s hot and happening in the digital and social space, and what those happenings mean for consumers and for marketers. Seeing as how I’m both a consumer AND a marketer, I pick up great info from every episode. Even better, the format and style of these bi-weekly chats are entertaining and engaging.

The most recent episode was a bit longer than usual, but it’s a good sample of the breadth of material they cover.

Predictions for 2011

Lisa asked the guys to complete the statement: “2011 will be The Year of…” and she got a range of responses:

  • Barce: Facebook Credits and the superphone
  • Matthew: data — the year we actually start making sense of and great experiences out of all of the data we’re collecting from consumers
  • Dan: the year of “the internet of things” and the year of Kinect-like technology (using motion to deliver great experiences)

CES 2011 Recap

Dan attended CES, while Matthew and Barce followed the event closely from afar. The highlights they discussed:

  • Tablets — the Motorola Xoom, which runs Google’s Android Honeycomb OS; the RIM Playbook, the Samsung Galaxy Tab, the Razer Switchblade, and all of the questions and issues around how the myriad form factors and applications will evolve (and how marketers and developers will deliver content to such a wide range of devices)
  • Superphones — the Motorola ATRIX made a splash at the show, but the larger discussion was around how a single device would truly become the centerpiece of a consumer’s digital life
  • 3D — 3D experiences are here to stay, but there was some debate as to whether this is really going to be driven more by consumers or more by manufacturers (and not just device manufacturers — Oakley and other sunglasses manufacturers are now introducing 3D glasses). Glassless group viewing may never happen (lenticular displays, even as they evolve, are still reliant on the viewer being in a small sweet spot to get the 3D effect), and what kind of human interaction barriers do 3D glasses introduce that limit the practical application of 3D?
  • Automotive — Audi’s attempts to deploy vehicle-to-vehicle communication such that vehicles can automatically collect data about weather and road conditions and share that information with other vehicles. This, I believe, is one example of “the internet of things” — all sorts of devices floating around the world that have both data collection and network connectivity capabilities
  • Motion — centered around Microsoft as the lead press conference at the event and Steve Ballmer discussing what’s next for the Kinect — controlling both Netflix and Hulu Plus using hand gestures, as well as Kinect-based avatars interacting in a virtual space (the Second Coming of SecondLife, perhaps?). And, the gang discussed how Kinect-like technologies can make for richer and more relevant consumer experiences both in-home and in-store.

The Mac App Store

Apple has now released an app store for the Mac — think iTunes, but for Mac laptops and desktops rather than just for iPhones. This appears to be a harbinger of a future that sounds a little funny: a future where laptops and desktops run apps. But, these are apps in the smartphone/superphone/tablet paradigm, rather than the “heavy overhead installed software applications” that have been a mainstay of computers for years. These apps will have much more of a platform-agnostic and cloud-centric orientation — enabling cross-device usage of an app in a seamless manner. The Chrome Web Store is another example of this shifting paradigm, with the Tweetdeck, Mashable, and Amazon Window Shop apps available there being examples of where it appears this world is heading.

The iPhone on Verizon

The consensus was that the announcement that, as of February 10th, the iPhone will be available on Verizon, rather than solely with AT&T, will be one of the biggest non-news events of the year. While iPhone users are frustrated with the dropped calls they get with AT&T, they’re going to be equally frustrated by the fact that they cannot simultaneously make a phone call and maintain a data connection with their iPhone when they switch to Verizon. AT&T’s 3G service is GSM-based, which allows data and phone service simultaneously…but is prone to call dropping. Verizon’s 3G service is CDMA-based, which is less prone to dropped calls, but which cannot run data and phone at the same time. Both AT&T and Verizon are migrating to the GSM-based 4G LTE technology, so users, presumably, will have similar experiences and similar limitations once that happens.

One way to look at this announcement is that it is a further leveling of the playing field for a 2-horse race </mixedmetaphor> between the iPhone and Android-based phones: Windows Mobile 7 is awesome, but it’s wayyyyy too late to the game, and RIM just can’t seem to get out of its own way.

Picks of the Week

  • Barce: personal hotspots coming to all iPhones in March (Verizon and AT&T)
  • Matthew: over the holidays, he purchased and installed a Filtrete WiFi Enabled Programmable Thermostat — controllable via an iPhone app or an web interface — and is loving it
  • Dan: the new eBay Fashion iPhone app — a very cool augmented reality app whereby you put your eyes between a couple of markers and you can then “try on” sunglasses

Pretty cool stuff. If you listen to podcasts, it’s worth subscribing (iTunes link)!

Analytics Strategy, Conferences/Community, General

Big Changes at Analytics Demystified

I suspect by now many of you have noticed but this week we made two pretty amazing announcements here at Analytics Demystified. Now that the dust is settling I have some time to take a step back and offer up some comments on the announcements and what I believe they mean for our clients, our prospects, and the web analytics industry in general.

On Tuesday we announced that respected industry veteran Adam Greco had joined John and I as a Senior Partner. Adam is well-known to many in our community thanks to his high-visibility work during his tenure at Omniture, his popular “Omni-Man” blog, and his fine, fine work on the Beyond Web Analytics podcast series.

For John and I bringing Adam on board was a no-brainer. The guy is as bright as they come, he is articulate, and most importantly he knows how to squeeze every last drop of value out of the most widely deployed digital measurement solutions in use today — Adobe SiteCatayst and Google Analytics. Adam is committed to extending that expertise to all of the popular platforms as quickly as possible, and our hope is that by mid-year he will be providing the same great insights he has for SiteCatalyst to Webtrends, Unica, Coremetrics, Nedstat, and other customers.

Adam will be running our Operational Use Audit and Framework Development practice as well as providing custom training and generally supporting the rest of the Demystified service offerings.  Which brings me to our second announcement …

On Wednesday we announced an exclusive partnership with tactical and technical consulting practice leaders Keystone Solutions. Keystone is a slightly better-kept secret than Adam Greco, although their current clients certainly know who they are. Founded years ago by former Omniture super-star Matthew Gellis, Keystone has grown into a talent magnet comprable to, well, Analytics Demystified.  Matt Wright from HP, Kurt Slater from Expedia, Rudi Schumpert from Ariba, and a host of other amazing analytics technicians.

We have doubled-down with Keystone for one simple reason: in our experience they are the best of the best when it comes to providing fundamental and foundational support for any digital measurement practice. Especially against those same two “most popular” solutions — Google Analytics and Adobe SiteCatalyst — Keystone delivers in a way that few others out there are capable, and that is the kind of talent we prefer to work with in the field.

Through this partnership Analytics Demystified clients will be able to benefit from a dramatically expanded set of web analytics consulting service offerings ranging from on-the-ground implementation support to ongoing reporting and analysis to some pretty amazing custom solutions. They will also be taking the lead on our Tag Management Systems Audit and Deployment practice, an offering I expect to be red-hot in 2011 and beyond.

Now, unfortunate as it is, we were not able to pursue this type of relationship with Keystone without some cost. The immediate fall-out is that Analytics Demystified will no longer be participating in the X Change conference. While this breaks my heart after having put three years of sweat equity into the event, relationships change and so it is time to move on.

I do, however, promise every one of the hundreds of consultants, vendors, and practitioners we have personally invited to this conference over the past three years that we will be back, live and in-person, with something far more “Demystified” in nature. Based on our work with Web Analytics Wednesday, the Analysis Exchange, and hundreds of other events around the globe, we have a pretty good idea of what is truly missing from the web analytics event landscape … and now, thanks to Adam and the team at Keystone, we have the means to deliver.

I welcome your comments and questions about both pieces of news, and I hope you’ll keep your eyes open in the coming few weeks for even more news from our growing company. It is exciting times, indeed.

Reporting, Social Media

The Future of Advertising Is Clear — Measurement, not So Much

Fast Company published a lengthy article last November titled The Future of Advertising, and it’s a good read. It traces the evolution of the advertising industry over the past 50 years, and it does a great job of assessing the business model(s) that have worked over time and why. That all serves as a backdrop for how the author posits digital and social media, and the crowdsourced-fragmented-wiki world we now live in is blowing those models up. And, it highlights a number of examples of agencies that are successfully shaking up the ways they operate. It’s a great read.

As I read through the article, I was eager to see what, if anything, came up regarding measurement and analytics. The sole mention turned up on the fourth page of the article (bold/underline added by me):

Every CEO in the [advertising agency] business…wants to be financially rewarded for performance, and thanks to all those new data-analytics tools, for the first time ever, their effectiveness can be measured. Says IPG chairman [Michael] Roth: “We should get higher [compensation] if it works and lower if it doesn’t. That’s how this industry can return to the profitability level.” It’s a nice thought, but those tools aren’t infallible: While Wieden’s innovative Web campaign for P&G’s Old Spice garnered tons of publicity, Ad Age speculated that the boost in sales may well have been due to a coupon.

So much for the silver bullet.

First off, the “every CEO in the business wants” statement is a little odd. Unquestionably, every CFO would love to be able to pay for performance, both the leaders in the company’s own marketing organization, as well as every agency with whom the company works. And, sure, every agency executive would agree that it is fair and reasonable to be paid based on performance. But, I don’t exactly think the advertising industry is flush with agencies wishing they could have performance-based compensation. Sure, agencies want to be able to measure the business impact of their work, but that’s so they can demonstrate their value to their clients, so, in turn, they can retain and grow those clients.

Just as my dander was good and raised, I hit the second sentence that I bolded and underlined above: “It’s a nice thought, but those tools aren’t infallible.” <whew> The voice of reason. But, the “new data-analytics tools” wording implies that there is some whole new class of business impact measurement platforms, and there simply is not. There are scads of emerging tools for measuring new channels like blogs and Facebook and Twitter, and there are lots of really smart people trying to build models that can supplement or supplant the broken reality of marketing mix modeling. But, we’re far, far, far from simply having “tools that aren’t infallible.”

Finally, the snippet above brings up the Old Spice campaign that featured Isaiah Mustafa in an eye-popping number of clever and consumer-engaging videos. No rational marketer would look at that campaign and try to judge it solely based on near-term sales. Word-of-mouth impact, consumers talking positively about the brand, existing customers quietly puffing out their chests because “their” brand is making a splash. How can that not lead to increased awareness of the brand, a positive shift in brand perception, and, I would think, 12-24 months of lingering positive effects? Is all of that worth $100 million or $1 million? I don’t know. But, from a “results based on what the conceivers of the campaign hoped to achieve,” it’s hard to argue that it delivered. But, I’m really not going to continue that debate — just want to point out that “immediate sales impact” is, well, the same sort of old school thinking that the rest of the article takes to task.

I still liked the article, but the brief measurement nod was a bit bizarre.

General

Adam Greco, Demystified

I am extremely excited to begin this next chapter in my career and wanted to get started at Analytics Demystified by describing my past, the present, and what I hope to do in the future.

The Past

I began my career in consulting working for Arthur Andersen’s technology group, first in the field of Customer Relationship Management and eventually in Marketing where I ended up managing the website of the Chicago Mercantile Exchange. It was there, as one of Omniture’s first customers, that I began to learn about web analytics and the data behind websites.

For some reason web analytics came naturally to me, and I loved figuring out fun, new ways to use the technology at my disposal to answer interesting business questions. As I had done in my consulting days, I dug into every Omniture training manual available and soon found that I knew the technology almost as well as those at Omniture. I think that is what led me to ultimately apply to work at Omniture in their newly founded Omniture Consulting group.

While at Omniture, I had the pleasure of working on many different clients, large and small, and helping them get the most out of Omniture products. As I showed Omniture clients how to use the technology, I often heard clients say “I had no idea you could do that…” so I decided to start a blog to teach people what they really ought to know.

Fast forward a few years and I decided that I wanted to go back to the “practitioner” side of the house and was given a great opportunity to head up web analytics at Salesforce.com. In the role of Senior Director for Web Analytics over the past two years at Salesforce I have had a great time reviving the web analytics program. More importantly, thanks to the generosity of the Salesforce organization, I have been able to continue writing about Omniture technology, speaking at industry events, and helping to establish and grow the Beyond Web Analytics podcast series.

The Present

While working at Salesforce.com has been one of the highlights of my career, when you are used to working on ten or more web analytics projects at a time as a consultant, sometimes working on just one for a few years in a row can be tough. Plus, despite being fully employed these past two years, I have been constantly approached by great people asking how to get the most out of their web analytics efforts — requests that I more often than not had to turn down.

Having been in consulting for most of my career, I had to face the inevitable truth; I have always been destined to become a consultant again … to work with dozens of companies at a time, sharing my knowledge of Omniture with companies large and small working to maximize their investment in web and digital analytics.

The Future

Once I decided that I wanted to go back into web analytics consulting, the difficult part was figuring out which organization would be the best fit for me. There are so many great web analytics consultancies, and over the years I have become friendly with many of the leaders of these firms. However, in my mind, Analytics Demystified was my first choice because of the brand that they have built, the caliber of their clients, and the overall thought-leadership their principals have displayed throughout the years.

When I think about the web analytics industry as a whole, I think about the books that helped launch the industry, the Yahoo Discussion Forum, the camaraderie found on Twitter, Web Analytics Wednesdays and most recently, the Analysis Exchange, and the new Web Analysts Code of Ethics. The common theme in all of these pillars is Eric Peterson and the Analytics Demystified brand he has created.

It was only last year when I first met Eric face-to-face but I have always been impressed by his ability to understand where our industry has been and, more importantly, where it is going. When John Lovett joined the organization, I enjoyed reading about all that he was doing in helping clients with their strategy, vendor evaluations, and social media efforts. In the last year, I got to spend some time with John and have been equally impressed with his passion for the industry and work he has done as a board member for the WAA.

I know there are any number of organizations that I could have joined where I could have helped teach people what I knew and managed teams of consultants. But at Analytics Demystified I believe that I am in a position to be both teacher and also a student, working with two of the smartest, most respected people in the field.

At Analytics Demystified, my hope is to help as many clients as possible get the greatest value from their investments in web analytics technology. I am excited to continue helping clients using Omniture technologies, but also look forward to branching out into other vendor tools and help their clients as well. It is my hope that the combination of Eric and John’s strategic work with my design and architecture background will have a synergistic effect, helping Analytics Demystified clients to further achieve their digital business objectives.

I look forward to hearing from all of you here in my new blog, and by all means if I can help your business, please don’t hesitate to contact me.

Adobe Analytics, General

Form Submit Button Clicks

At the end of last year, I spent a bunch of time showing how you could dissect your website forms to see which were performing well and not so well. While this post will be different from those, it is still related to website forms. In this post, I am going to share a concept that will let you determine which of those visitors seeing your forms have the intention to complete them and which do not. This information can be very valuable as I hope to show.

Which Forms Get Visitors to Take Action?
If you have forms on your website, I hope that you are at least doing the basics and tracking how many people View each Form and how many Complete each Form like this:

This will allow you to have a rudimentary view about how each website form is performing. However, one short-coming of this is that you only have two points of comparison. As a web analyst, I always like to have more data points to slice, dice and analyze. The report above answers the question: “How many people who see each form decide to complete it?” What if you wanted to know how many people who see each form try to complete it? That might be an interesting data point, since sometimes when you do a lot of Paid Search or Display Advertising you could be driving less qualified traffic to your website. Therefore, what I like to do is to create a new metric that I call Form [Submit] Button Clicks. This Success Event is set when website visitors click the button that you place on your form (duh!). By doing this, you have essentially created a wedge between the Form Views and Form Completes metrics shown above such that you can create a report that looks like this:

As you can see here, in the first report above we knew that only 786 of the 2,246 Form Views turned into Form Completions. However, with the second report, we now know that visitors to that specific form clicked the Form Submit button 830 times. That means that 44 times they tried to complete the Form, but were unable to for one reason or another (maybe Form Errors).

Dig Deeper With Calculated Metrics
Once you have this cool new Form Button Clicks metric, you can then create some fun new Calculated Metrics that let you dig even deeper. Here are two that I suggest: Form Button Click Rate & Form Button Click Fail Rate. The Form Button Click Rate is the number of Form Button Clicks divided by the number of Form Views. This metric shows you what percent of people viewing the Form actually click the button as shown here:

In this report you can see which forms on your website are doing a good job at getting visitors to click the button. Forms with low percentages might indicate that there are too many fields, poor content or a bad offer. You can use this report to zero in on which forms represent the biggest opportunity for improvement. I like to bubble-chart this data such that the forms with the most Form Views and the lowest Button Click Rate move to the “magic quadrant.”

The next Calculated Metric is the Form Button Click Fail Rate. This represents the percentage of times visitors click the Form Submit button, but fail to have a Form Complete. These people represent your “lowest hanging fruit” as by clicking the button, they have implicitly told you they are somewhat interested in you! You create this metric by dividing the difference between Form Button Clicks and Form Completes by the number of Form Button Clicks as shown here:

In this case, for the first form, about 5% of people who click the button don’t make it to a Form Complete, but the last form shown in the report seems to have some issues since 62% of Form Button Clicks don’t make it to a Form Complete. You may want to start doing some testing on that form!

As is always the case, whenever you create new Calculated Metrics you can see them as general metrics in addition to using them in eVar reports. Therefore you can set Alerts and see trends for both of the metrics described above:

What I like about these two metrics is that one shows you how good you are at getting people to click the button on the form (how good your offer/content is) and the other tells you how good you are at closing the deal once a visitor has decided to give you a chance. Those who have managed websites realize that there are very different tactics used to solve these two very different questions so having these metrics can really help you focus and use your precious website resources as efficiently as possible.

Don’t Forget Your Other Reports!
While the above reports hopefully get you excited, don’t forget that you already have many reports that can be combined with the information above to get even more value. For example, one of the reports I use a lot is the Traffic Driver (Unified Sources) report which shows me how each visitor got to my website. Wouldn’t it be cool if I could see Form Button Clicks and the above two new Calculated Metrics by Traffic Source? Well…you can! All you have to do is add these metrics to your existing Traffic Sources report like this:

Now you can see how each channel is doing! Looks like Paid Search (SEM) is generating lots of Form Views, but only gets 12% of these to turn into Form Button Clicks. If they do get someone to click the button, it looks like 55% of them don’t end up successfully making it to a Form Complete. This can be contrasted by SEO which seems to fare a bit better by getting 30% of its Form Viewers to click the button and of those 75% make it through to Form Completion. You can imagine how powerful this data could be and how you could use a product like Test&Target to come up with ways to improve these conversion rates by traffic source.

If you want to get even more granular, you can break this report down by the root traffic driver so you can take specific actions. In the following report, I can see the Paid Search ID’s that make up the Form Views and the other metrics and see how each performs individually:

Here we can see that there are some Paid Search keywords that are doing well (get people to click on the submit button over 20% of the time) and others that are under-performing (less than 15%). You can use these metrics to help drive your Paid Search strategy or possible automate this using SearchCenter. Finally, in this fictitious example, I have made row three have zero Form Completes, but a 32% Form Button Click Rate, which would indicate a major issue with the form that should be addressed.

One last example of leveraging an existing report would be the Visit Number report:

Here we can see that the Form Button Click Rate is pretty consistent, but up a bit in the 3rd visit, but interestingly, our Form Button Click Fail Rate appears to decrease over time. Perhaps the more time visitors take to get to know us, the more likely they are willing to deal with all of the information we are asking for on our forms!

Final Thoughts
Well there you have it. I always find it so amazing that adding one simple Success Event in the right place can open up so many new web analysis opportunities. If you have forms on your website, I hope this will help you learn more about your users and how they are interacting with your forms. Let me know if you have any questions…

Analytics Strategy, General, Social Media

It's not about you, it's about the community …

Happy New Years my readers! I hope the recent holidays treated you well regardless of your faith, persuasion, or geographic location. I wanted to take a quick break from all the heavy privacy chatter these past few months and tell a little story about the generosity of our community and one individual in particular.

If you follow me on Twitter you may have noticed me cryptically tweeting “it’s not about you, it’s about the community” from time to time. I started sending this update as a subtle hint to a few folks who harp on and on about their accomplishments, products, and “research” in the Twitter #measure community … but sadly those folks never got the hint (so much for being subtle, huh?)

Over time the tweet became something larger — it became a reminder about what we all are capable of when we think about more than our own little world.  “It’s not about you, it’s about the community” is about some of the greatest contributors in the history of web analytics, people like:

  • Jim Sterne, who years ago realized that we needed a place to gather, and who wisely picked the Four Seasons Biltmore in Santa Barbara, California.  While Emetrics may have become a profit-generating machine, those of you who know Jim and know history understand that the conference is as much about and for the community as it is anything else;
  • Jim Sterne, Bryan Eisenberg, Rand Schulman, Greg Drew, Seth Romanow, and others who founded the Web Analytics Association years ago when it was clear that we needed some type of organizing body, committing themselves to hundreds of hours of work without thinking about how they would make money off of the effort;
  • Jim Sterne (again!!!!) who has been making sure that we all know who is doing what where and when via his “Sterne Measures” email newsletter for as long as I can remember;
  • Avinash Kaushik, Google’s famed Analytics Evangelist, who has long committed the profits from his books on web analytics to two amazing charities;
  • Super-contributors to the Web Analytics Forum at Yahoo Groups, folks like Kevin Rogers, Yu Hui, Jay Tkachuk, and dozen more who still take the time to answer questions from newer members of this rapidly expanding community;
  • Past and current Web Analytics Association Board members and super-volunteers, folks like Alex Yoder, Jim Novo, Raquel Collins, Jim Humphries, and so many more who give their time and energy every month to make sure the Association continues to evolve and grow;
  • Activists and evangelists like my partner John Lovett, who in the midst of writing his first book on social media analytics has taken the time to shepherd our Web Analysts Code of Ethics effort through the Web Analytics Association Board of Directors;
  • Everyone who has ever hosted a Web Analytics Wednesday event, including luminaries like Judah Phillips, June Dershewitz, Tim Wilson, Bob Mitchell, Emer Kirrane, Perti Mertanen, Alex Langshur, Anil Batra, Ruy Carneiro, Dash Lavine, Jenny Du, David Rogers, and way too many more folks to list who contribute their valuable time to help grow organic web analytics communities locally;
  • All of the over 1,000 members of the Analysis Exchange, many of whom have contributed to multiple projects to make sure that nonprofit organizations around the world have access to web analytics insights;
  • Dozens of others I am forgetting, and probably hundreds more I have never even met …

When I think about this list of people and their individual contributions to the web analytics community it is almost overwhelming — how lucky we are to have such considerate and giving friends!  Still, people have been giving back for years and so it is rare that I see something or someone in the community that really blows me away …

Until recently.

Not everyone knows Jason Thompson, and I suspect he would be the first to admit that not everyone who knows him actually likes him, but if I had to pick one “web analytics super-hero” for 2010 Jason would be my hand’s-down, number one choice.  See, Jason was smart enough to not just get the web analytics community to give back to our community, he managed to get our community to help provide clean water to an entire community in a developing nation.

Having worked repeatedly as a volunteer with Analysis Exchange Jason was introduced to charity:water, a nonprofit organization who’s vision is very simple: to provide clean, safe drinking water for everyone on the planet.

Water.

Not a great blog or free books, not data or solution profilers, but water that mothers can bring to their children. Clean, pure water that I would venture each and every one of the members of the web analytics community takes for granted and rarely even considers the source and its availability.

But Jason thought about it, and what’s more, Jason did something about it. Thanks to some cool new technology Jason was able to donate his 36th birthday to help raise $500. By leveraging Twitter and his web analytics community he was able to raise that $500 by December 18th.  Having met his goal before his birthday Jason didn’t stop and settle, he set the bar higher, working first to raise $1,000, then $3,000, and finally $5,000, enough to provide water for an entire village – 80 people for 20 years.

Jason’s effort brought out the best in our community again, collecting donations from luminaries and lay-users alike … hell, he even got money from his mom! Some of the biggest names in web analytics helped Jason along, and donations large and small rolled in right up until Ensighten’s Josh Manion put in the last $300 on Jason’s birthday, putting him over the top and completing his final goal.

Honestly I don’t know Jason very well, but I do know passion and greatness when I see it. Jason once again served as a reminder that “it’s not about you, it’s about the community” and he did more than just tweet obnoxiously … he put his time and money where his mouth is and did something real.

Bravo, Mr. Thompson.  Bravo.

If you don’t know Jason I highly recommend following him in Twitter (@usujason, if you’re into Twitter) and, if you see him at a conference or event do like I will and buy the man a drink. I for one am going to let Jason be an example of how I can work even harder to make a difference both inside and outside of the web analytics community in 2011 and beyond.

Hopefully some of you will do the same.