Analysis, Analytics Strategy, Social Media

What I Learned About #measure & Google+ from a Single Blog Post

Quite unintentionally, I stirred up a lengthy discussion last week with a blog post where I claimed that web analytics platforms were fundamentally broken. In hindsight, the title of the post was a bit flame-y (not by design — I dashed off a new title at the last minute after splitting up what was one really long post into two posts; I’m stashing the second post away for a rainy day at this point).

To give credit where credit is due, the discussion really took off when Eric Peterson posted an excerpt and a link in Google+ and solicited thoughts from the Google+/#measure community. That turned into the longest thread I’ve participated in to date on Google+, and subsequently led to a Google+ hangout that Eric set up and then moderated yesterday.

This post is an attempt to summarize the highlights of what I saw/heard/learned over the past week.

What I Learned about the #measure Community

Overall, the discussion brought back memories of some of the threads that would occasionally get started on the webanalytics Yahoo! group back in the day. That’s something we’ve lost a bit with Twitter…but more on that later.

What I took away about the group of people who make up the community was pretty gratifying:

  • A pretty united “we” — everyone who participated in the discussions was contributing with the goal of trying to move the discussion forward; as a community, everyone agrees that we’re at some sort of juncture where “web analytics” is an overly limiting label, where the evolution of consumer behavior (read: social media and mobile) and consumer attitudes (read: privacy) are impacting the way we will do our job in the future, and where the world of business is desperately trying to be more data-driven…and floundering more often than succeeding. There are a lot of sharp minds who are perfectly happy to share every smart thought they’ve got on the subject if it helps our industry out — the ol’ “a rising tide lifts all boats” scenario. That’s a fun community with whom to engage.
  • Strong opinions but small egos — throughout the discussion that occurred both on Google+ and on Twitter (as well as in several blog posts that the discussion spawned, like this one by Evan LaPointe and Nancy Koon’s inaugural one and Eric’s post), there were certainly differing points of view, but things never got ugly; I actually had a few people reach out to me directly to make sure that their thoughts hadn’t been taken the wrong way (they hadn’t been)
  • 100s of years of experience — we have a lot of experience from a range of backgrounds when it comes to trying to figure out the stickiest of the wickets that we’re facing. That is going to serve us well.
  • (Maybe) Agencies and vendors leading the way? — I don’t know that I learned this for sure, but an informal tally of the participants in the discussion showed a heavy skewing towards vendor and agency (both analytics agencies and marketing/creative/advertising agencies) representation with pretty limited “industry” participation. On the one hand, that is a bit concerning. On the other hand, having been in “industry” for more of my analytics career than I’ve been on the agency side, it makes sense that vendors and agencies are exposed to a broader set of companies facing the same challenges, are more equipped to see the patterns in the challenges the analytics industry is facing, and are being challenged from more directions to come up with answers to these challenges sooner rather than later.

These were all good things to learn — the people in the community are one of the reasons I love my job, and this thread demonstrated some of the reasons why that is.

Highlights of the Discussion

Boiling down the discussion is bound to leave some gaps, and, if I started crediting individuals with any of the thoughts, I’d run the serious risk of misrepresenting them, so feel free to read the Google+ thread yourself in its entirety (and the follow-up thread that Eric started a few days later). I’ve called out any highlights that came specifically from the hangout as being from there (participants there were Adam GrecoJohn LovettJoseph StanhopeTim WilsonMichael HelblingJohn RobbinsEmer KirraneLee IsenseeKeith Burtis, and me), since there isn’t a reviewable transcript for that.

Here goes:

  • Everyone recognizes that a “just plug it in and let the technology spit out insights” solution will likely never exist — the question is how much of the technical knowledge (data collection minutia, tool implementation nuances, reporting/analysis interface navigation) can be automated/hidden. A couple of people (severalpublicly, one privately) observed that we want (digital) analytics platforms to be a like a high-performance car — all the complexity as needed under the hood, but high reliability and straightforward to operate. Pushing that analogy — how far and fast it runs will still be highly dependent on the person behind the wheel (the analyst).
  • Adobe/Omniture and Google Analytics had near-simultaneous releases of their latest versions; both companies touted the new features being rolled out…but both companies have stressed that there was a lot more about the releases that were under-the-hood changes that were positioning the products for greater advances in subsequent releases; time will tell, no? And, several people who have actually been working  with SC15 (I’ve only seen a couple of demos, watched some videos, and read some blog posts — the main Omniture clients I support are over a year out from seeing SC15 in production), have pointed out that some of the new features (Processing Rules and Context Data, specifically) will really make our lives better
  • There was general consensus that Omniture has gotten much, much better over the years about listening to customer feedback and incorporating changes based on that feedback; there is still a Big Question as to whether customer-driven incremental improvements (even improvements that require significant updates on the back end) will get to true innovation — the “last big innovations” in web analytics were pointed out as being a decade ago (I would claim that the shift from server logs and image beacons to Javascript-based page tags was innovative and wasn’t much older) — or whether “something else” will have to happen was a question that did not get resolved
  • Getting beyond “the web site” is one major direction the industry is heading — integrating cross-channel data and then getting value from it — introduces a whole other level of complexity…but the train is barrelling along on a track that has clearly been laid in that direction
  • We all get sucked into “solving the technical problem” over “focusing on the business results” — the tools have enough complexity that we count it a “win” when we solve the technical issues…but we’re not really serving anyone well when we stop there; this is one of those things, I suspect, that we all know and we constantly try to remind ourselves…and yet still get sucked into the weeds of the technology and forget to periodically lift our heads up and make sure we’re actually adding value; John Lovett has been preaching about this conundrum for years (and he hits on it again in his new book)
  • Marketing/business are getting increasingly complex, which means the underlying data is getting more complex (and much more plentiful — another topic John touches on in his book), which means getting the data into a format that supports meaningful analysis is getting tougher; trying to keep up with that trend is hard enough without trying to get ahead!
  • Tag management — is it an innovation, or is it simply a very robust band-aid? Or is it both? No real consensus there.
  • Possible areas where innovation may occur: cross-channel integration, optimization, improved conversion tracking (which could encompass both of the prior two areas), integration of behaviora/attitudinal/demographic data
  • [From the hangout] “Innovation” is a pretty loaded term. Are we even clear on what outcome we’re hoping to drive from innovation?
  • [From the hangout] Privacy, privacy, privacy! Is it possible to educate the consumer and/or shift the consumer’s mindset such that they are informed about why that “tracking” them isn’t evil? Can we kill the words “tracking” and “targeting,” which both freak people out? Why are consumers fine with allowing the mobile or Facebook application access to their private data…but freak out about no-PII behavioral tracking (we know why, but it still sucks)?
  • [From the hangout] How did a conversation about where and how innovation will occur devolve into the nuts and bolts of privacy? Why does that happen so often with us? Is that a problem, or is it a symptom of something else?

Yikes! That’s my attempt to summarize the discussion! And it’s still pretty lengthy!

What I Learned about Google+

I certainly didn’t expect to learn anything about Google+ when I wrote the post — it was focusing on plain ol’ web (site) analytics, for Pete’s sake! But, I learned a few things nonetheless:

The good:

  • Longer-form (than 140 characters) discussions, triggered by circles, with the ability to quickly tag people, are pretty cool; Twitter sort of forced us over to blog posts (and then comments on the posts) to have discussions…and Google+ has the potential to bring back richer, more linear dialogue
  • Google+ hangouts…are pretty cool and fairly robust; we had a few hiccups here and there, but I was able to participate reasonably well from inside a minivan traveling down the highway that had the other four members of my family in it (Verizon 4G aircard, in case you’re wondering); and, as the system detects who is speaking, that person’s video jumps to the “main screen” pretty smoothly. It’s not perfect (see below), but we had a pretty meaty conversation in a one-hour slot (and credit, again, to Eric Peterson for his mad moderation skills — that helped!)

The not-so-good:

  • Discussions aren’t threaded, and the “+1” doesn’t really drive the organization of the discussion — multiple logical threads were spawned as the discussion continued, but the platform didn’t really reflect that, which many discussion forums have supported for years
  • Linking the blog post to the discussion was a bit clunky. Who knows what long tail search down the road would benefit from seeing the original post and the ensuing conversation? I added a link to the Google+ discussion to the post after the fact…but it’s not the same as having a string of comments immediately following a post (and if Google+ fizzles…that discussion will be lost; I’ve made a PDF of the thread, but that feels awfully 2007)
  • Google+ hangouts could use some sort of “hand-raising” or “me next” feature; everyone who participated in the hangout worked hard to not speak over anyone else, but we still had a number of awkward transitions

So, that’s what I took away. It was a busy week, especially considering I was knocking out the first half of John Lovett’s new book book (great stuff there) at the same time!

Adobe Analytics, Analytics Strategy, Conferences/Community

Announcing ACCELERATE 2011!

We are incredibly excited to announce that registrations are open for our newest community initiative designed for digital measurement, analysis, and optimization professionals, ACCELERATE!

The first event will be held this year in San Francisco on Friday, November 18th at the Mission Bay Conference Center at UCSF, and thanks to generous support from Tealeaf, OpinionLab, and Ensighten, ACCELERATE 2011 is completely free.

Our agenda is still being finalized, but we will have thought- and practice-leaders from amazing companies including Nike, Symantec, AutoDesk, Salesforce.com and, of course, Analytics Demystified. Also, since we recognize that some of the brightest talent in our field works for solution providers, we’ve invited a few practice leaders from the vendor community to present as well, thusly ensuring great content across the board.

The format at ACCELERATE is completely new, and we believe our “Ten Tips in Twenty Minutes” style will create the maximum number of insights possible for attendees of all backgrounds. What’s more, we have a dozen ten open slots for new speakers in our “Super Accelerator” session to showcase up-and-coming talent — and we’re having those folks compete for a $500 gift card from Best Buy based on audience votes.

Did I mention that ACCELERATE 2011 is completely free?

If you’re interested in joining us we encourage you to visit the ACCELERATE 2011 mini-site sign-up register today. Space is limited to the first hundred or so folks who sign up and we’ve already had registrations from New York, Boston, Seattle, Portland, Columbus, and San Francisco.

Go to the ACCELERATE 2011 site and register to attend right now!

EVENT DETAILS:

Location: Mission Bay Conference Center, San Francisco
Date: Friday, November 18, 2011 from 9:00 AM to 4:30 PM
Registration: Open now, limited to the first hundred or so folks who sign up

If you have any questions about ACCELERATE 2011 please leave comments below or email us directly.

Analytics Strategy, General, Social Media

Massive Web Analytics Throw-down in Google+

Much to my chagrin, having been outed by the local newspaper for my original dismissal of Google+, it appears that the web analytics community is prepared to go “all in” in the social network. What’s more, because we’re no longer bound by 100-odd characters (after we @respond and #measure tag), suddenly some incredibly bright minds are able to rapidly contribute to an emerging meme.

Interested? I knew you would be.

Head on over to my stream at Google+ and catch up on the conversation stemming from Tim Wilson’s recent critique of Adobe SiteCatalyst 15. Certainly the thread has diverged somewhat but if you’re in web analytics and on Google+ we would all welcome your contribution.

>>> Web Analytics Platforms are Fundamentally Broken

If you’re not on Google+ click on this link as I have bunches of invites I can share.

Analytics Strategy

Web Analytics Platforms Are Fundamentally Broken

Farris Khan, Analytics Lead at ProQuest and Chevy Volt ponderer extraordinaire, tweeted the question that we bandy about over cocktails in hotel bars the world over during any analytics gathering:

His tweet came on the heels of the latest Beyond Web Analytics podcast (Episode 48), in which hosts Rudi Shumpert and Adam Greco chatted with Jenn Kunz about “implementation tips.” Although not intended as such, the podcast was skewed heavily (95%) towards Adobe/Omniture Sitecatalyst implementations. As the dominant enterprise web analytics package these days, that meant it was chock full of useful information, but I found myself getting irritated with Omniture just from listening to the discussion.

My immediate reply to Farris’s tweet, having recently listened to the podcast, reflected that irritation:

Sitecatalyst throws its “making it much harder than it should be” talent on the implementation side of things, and I say that as someone who genuinely likes the platform (I’m not  a homer for any web analytics platform — I’ve been equally tickled pink and wildly frustrated with Google Analytics, Sitecatalyst, and Webtrends in different situations). I’m also not criticizing Sitecatalyst because I “just don’t understand the tool. ” I no longer get confused by the distinction between eVars, sProps, and events. I’ve (appropriately) used the Products variable for something totally separate from product information. I’ve used scView for an event that has nothing to do with a shopping cart. I’ve set up SAINT classifications. I’ve developed specs for dynamically triggering effectively named custom links. I’ve never done a stint as an Adobiture employee as an implementation engineer, but I get around the tool pretty well.

Given that I’ve got some experience there, I’ve also worked with a range of clients who have Sitecatalyst employed on their sites. As such, I’ve rolled my eyes and gnashed my teeth at the utter botched-ness of multiple clients’ implementations, and, yes, I’ve caught myself making the same type of critical statements that were rattled off during the podcast about companies’ implementations:

  • Failure to put adequate up front planning into their Sitecatalyst implementation
  • Failure to sufficiently document the implementation
  • Failure to maintain the implementation going forward on an on-going basis
  • Failure to invest in the people to actually maintain the implementation and use the data (Avinash has been fretting about this issue publicly for over 5 years)

In the case of the podcast, though, I wasn’t participating in the conversations — I was simply listening to others’ talk. The problem, though, was that I heard myself chiming in. I jumped right  on the “it’s the client’s fault” train, nodding my head as the panel described eroded and underutilized implementations. But, then a funny thing happened. As  I stepped back and listened to what “I” would have been saying, I got a bit unsettled. I realized I’d been seduced by the vendor. Through my own geeky pride at having cracked the nut of their inner machinations, I’d crossed over to vendor-land and started unfairly blaming the customer for technology shortcomings:

If the overwhelming majority of companies that use a given platform use it poorly…shouldn’t we shine a critical light on the platform rather than blaming the users?

I love digital analytics. I enjoy figuring out new platforms, and it’s fun to develop implement something elegantly and then let the usable data come pouring in that I can feed into reports and use for analysis. But:

  • I’ve been doing this for a decade — hands-on experience with a half-dozen different tools
  • It’s what I’m most interested in doing with my career — it beats out strategy development, creative concepting, campaign ideation, and any and every other possible marketing role
  • I’m a sharp and motivated guy

In short…I’m uniquely suited to the space. I’m neither the only person who is really wired to do this stuff nor even in the 90th percentile of people who fit that bill. But the number of people who are truly equipped to drive a stellar Sitecatalyst implementation are, best case, in the low thousands, and, worst case, in the low hundreds. At the same time, demand for these skills is exploding. Training and evangelization is not going to close the gap! The Analysis Exchange is a fantastic concept, but that’s not going to close the gap, either.

There is simply too much breadth of knowledge and thought required to effectively work in the world of digital analytics for a tool to have a steep learning curve with undue complexity for implementation and maintenance. The Physics of the Internet means there are a relatively finite number of types of user actions that can be captured. Sitecatalyst has set up a paradigm that requires so much client-side configuration/planning/customization/maintenance/incantations/prayer that the majority of implementations are doomed to take longer than expected (much longer than promised by the sales team) and then further doomed to be inadequately maintained.

The signals that Adobe is slowly taking steps to merge the distinction between eVars and sProps is an indication that they realize that there are cases where the backend architecture needlessly drives implementation complexity. But, just as the iPhone shattered the expectations we had for smartphones, and the iPad ushered in an era of tablet computing that will garner mass adoption, Adobe has a very real risk of Sitecatalyst becoming the Blackberry of web analytics. Sitecatalyst 15, for all of the excitement Adobe has tried to gin up, is a laundry list of incremental fixes to functional shortcomings that the industry has simply complained about for years (or, in the case of the the introduction of segmentation, a diluted attempt to provide “me, too” functionality based on what a competitor provides).

The vendors have to take some responsibility for simplifying things. The fact that I can pull Visits for an eVar and Visits for an sProp and get two completely different numbers (or do the same thing for instances and page views) is a shortcoming of the tool. We’ve got to get out of the mode of simply accepting that this will happen, that a deep and nuanced understanding of the platform is required to understand the difference, and then gnashing our teeth when more marketers don’t have the interest and/or time to develop that deep understanding of the minutia of the tool.

<pause>

Although I’ve focused on Sitecatalyst here, that doesn’t mean other platforms are beyond reproach:

  • Webtrends — Why do I have to employ black magic to get my analysis and report limits set such that I don’t miss data? Why do I have to employ Gestapo-like processes to prevent profile explosion (and confusion)? Why do I have to fall back on weeks-long reprocessing of the logs when someone comes up with a clever hypothesis that needs to be tested?
  • Google Analytics — Why can’t I do any sort of real pathing? Why do I start bumping up against sampled data that makes me leery…just when I’m about to get to something really cool I want to hang my hat on? Why is cross-domain and cross-subdomain tracking such a nightmare to really get to perform as I want it to?

My point here is that the first platform that gets a Jobs-like visionary in place who is prepared to totally destroy the current paradigm is going to have a real shot at dominating over the long haul. There are scads of upstarts in the space, but most of them are focused on excelling at one functional niche or another. Is there the possibility of a tool (or one of the current big players) really dramatically lowering the implementation/maintenance complexity bar (while also, of course, handling the proliferation of digital channels well beyond the traditional web site) so that the skills we need to develop can be the ones required to use the data rather than capture it?

Such a paradigm shift is sorely needed.

Update: Eric Peterson started a thread on Google+ spawned by this post, and the lengthy discussion that ensued is worth checking out.

Reporting, Social Media

Gamification — One Angle to Consider w/ Social Media Campaigns

At least once a month, something comes up that reminds me of the power of applying the lens of gamification to campaign planning. While slightly off topic for this blog (I’ll touch on measurement towards the end), it’s something that continues to rattle around in my skull, so I might as well work those thoughts out in a post.

The Basics

A fairly succinct explanation of what game mechanics is can be found in a paper published last October by Bunchball, a gamification platform provider:

At its root, gamification applies the mechanics of gaming to nongame activities to change people’s behavior. When used in a business context, gamification is the process of integrating game dynamics (and game mechanics) into a website, business service, online community, content portal, or marketing campaign in order to drive participation and engagement.
:
The overall goal of gamification is to engage with consumers and get them to participate, share and interact in some activity or community. A particularly compelling, dynamic, and sustained gamification experience can be used to accomplish a variety of business goals.

The key here — and this is actually the biggest detriment of the term itself — is that “gamification” is not simply “playing games.” All too often, I have conversations with people who immediately think XBox, Playstation, Kinect, Farmville, or any number of other “traditional games” when the topic of gamification comes up. That’s an entirely appropriate starting point, but it’s by no means the whole story.

Gamification is about using human nature’s inherent interest in being engaged with others, being rewarded, achieving goals, and, yes, having some fun in the process.

A Recent (and Simple) Example

During a #measureX trip to New Orleans, one of the other people on the trip mentioned that she had been doing a lot of travelling lately, and she tries to fly American Airlines, because they have good flights to most of the places she goes, and she is close to reaching the Gold Level of their Frequent Flier Program. Frequent flier programs are an example of gamification applied for the direct benefit of the brand, allowing travelers to earn points towards different levels, at which they are awarded with different perks. These programs don’t directly drive engagement with other consumers, but that’s another key to gamification — it’s not a one-size-fits-all deal.

And an Even More Recent #measure Example

Even the elusive @AnalyticsFTW has indulged in some gamification of late, with an infographic-creating contest to win a pass toeMetrics in NYC, <shamelessplug>where I will be speaking on Twitter analytics </shamelessplug>. It’s simply a matter of offering a prize (a valuable one, in this case), and then letting the #measure community spread the word, with entrants being challenged to come up with something original and amusing. On the one hand, it’s a “simple contest,” but it’s a simple contest that:

  • Forces all potential entrants to actually stop and think about the value of eMetrics
  • Requires an investment of time and energy to illustrate that value in a clever way (which causes them to thinkmore about the value of eMetrics)
  • Generates marketing collateral for the event that others will come and look at (user-generate content, baby! Not a single designer finger on the paid eMetrics team was lifted to generate the material)

It’s brilliant, really.

An Entirely Different (and More Involved) Example

I was tapped/volunteered to teach a “Microsoft Excel Tips & Tricks” brown bag lunch at work a couple of months ago. It was content that I knew attendees would get value out of…but with a title that didn’t exactly have a “Cowboys & Aliens”-type mystique that would be a natural attendance draw (and, while personable enough, I’m not exactly the office equivalent of Daniel Craig or Harrison Ford).

I applied some game mechanics to promote the event by distributing a series of cards around our various offices (physical cards as well as digital versions to our remote locations):

The cards led to a video (PowerPoint with low-fi voiceover) with details as to the “game,” which required participants to do a little searching and a little collaboration with another office before posting “the answer” on the wall of a Facebook group.

Here’s what I hoped to achieve:

  • Engage as many employees as possible just enough with the type of content that I would be presenting that they would have an opportunity to pause and think, “Hmmm… I might actually get something useful out of this”
  • Extend that engagement beyond our main office in Columbus to our satellite offices and remote workers
  • Find out if I could apply game mechanics without consulting a gamification expert and achieve good results

My KPI for the effort was pretty simple: a “healthy turnout” at the brown bag. I had a handful of additional measures in place:

  • Whether or not anyone actually managed to complete the challenge and, if so, how long it took for that to happen
  • The number of clicks on the goo.gl link/QR code link driving to the YouTube video
  • The number of views of the video
  • The number of people who walked by my desk and either chuckled or shook their head

In the end, we had a full room for the brown bag. KPI achieved!

We’re over 300 employees now, and my other measures played out as follows: 159 clicks on the link, 131 video views, and a half-dozen people who chuckled and shook their heads as they walked by my desk. Not bad.

Most surprisingly, though, was how quickly and to what extent people got into the activity. I launched on a Wednesday evening after almost everyone was gone for the day. At 8:29 AM on Thursday morning…9 seconds apart…two people (from two different offices, and they’d both colluded with the same person in a third office) posted the winning answer on the Facebook group’s wall. Considering that I was a little concerned that the whole thing would be a total dud, I certainly didn’t expect to have winners before 8:30 AM on the first day!

Different from “Games”

So, gamification is not simply “playing games.” It’s using the aspects of human nature that make playing games fun and engaging…and then leveraging those to drive interest and engagement around a brand, a product, an event, or something else. It’s an utterly intriguing concept, and it’s not hard to spot examples of marketers putting these ideas to good use.

Another paper/presentation on the subject published late last year by Resource Interactive has some additional good nuggets on the topic:

Game On: Gaming Mechanics

View more presentations from Resource Interactive

 

Measuring the Results

Any marketing initiative should be measured. Campaigns that rely heavily on game mechanics are easier to measure than a lot of always-on social media activities (a Twitter feed, a Facebook page, etc.). That is, they’re easier to measure if there is a clear objective for the effort, and if that objective is something that gamification is good at supporting: driving engagement and/or driving awareness (and education) through word-of-mouth. Meaningful KPIs may include:

  • The number of people exposed to the campaign
  • The number of people who participated in the game mechanics aspects of the game
  • The number of people who reached a certain level of engagement with the campaign

Now, this sets me up for the criticism, “Well, yeah, but did it drive business results.” In some cases, CTAs can be embedded in the game that can lead to conversions that can be measured as results. But, there is, admittedly, some requirement that the entire campaign has been designed with a logical link to business value. For instance, for a low-awareness brand targeted at a niche audience, then a campaign that grows awareness across a community that represents that niche, and that does so at a relatively low cost will often be a no-brainer when compared with low-engagement paid media.

Benchmarks will seldom be available for these types of campaigns. Get over it! If you’re developing a compelling campaign, it’s going to need some degree of originality, which means there won’t be a sea of comparable campaigns at your fingertips for benchmarking. That makes establishing targets a bit scary. Set a target anyway. Think through what would be acceptable and what would be clearly awesome based on other, more traditional ways you could have chosen to invest those same dollars. More often than not, if it’s a well-designed game-mechanics-applied campaign, you will know whether you are on to a good idea early in the planning, and you will be very pleasantly surprised by the results.

Social Media

You’re Using the Wrong Social Media Metrics!

This content originally posted on the ClickZ Marketing News & Expert Advice website on July 14, 2011.

In my experience, I’ve found that the vast majority of practitioners measuring social media currently rely on the wrong metrics. Metrics such as fans, followers, +1’s, shares, likes, and dislikes are easily captured and readily delivered by social networks, but they represent merely the low-hanging fruit of social analytics. These are the “counting metrics” of social media because using them typically equates to counting up digital trivia. Effective measurers of social media go beyond counting metrics to create outcome-based metrics and ultimately report on business value metrics to senior stakeholders across the enterprise. In this column, I’ll elaborate on the minutia of counting metrics and where they can add value to your social media operations, as well as how to take the next step of creating outcome and business value metrics to ratchet up your social analytics game to the next level.

Testing the Social Media Waters

The temptation for businesses to experiment with social media is practically irresistible. And in fact, you’d be foolish not to venture into new and emerging channels if your target audience leads you there. But experimentation and ongoing participation in social media must continually prove out the potential for business value. Often times, this potential is demonstrated in metrics that are indicative of volume and activity. Counting metrics do just that because they are measures that tell you how deep the social media pool really is. These counting metrics are typically the freebies offered by social media networks that quantify the basic observational statistics of participation. The stats include: number of users, number of fans, number of followers, number of posts, number of comments per post, number of check-ins, number of ratings, number of reviews…and so on. You quickly see that there’s numbers on top of numbers.

Yet, stopping at this point and using only counting metrics to measure and manage social media is not only just plain lazy, but also detrimental to your business. These metrics are important for gauging the health and activity of your social media operations, but they fail to tell you if you’re achieving your business goals. Counting metrics can offer insights into how many people are swimming and if the water is too cold, or just right. They can also tell you how many people you are reaching with your social media messages and if your content is worthy of passing on to their friends and followers. But, what counting metrics cannot tell you is who the lifeguards should be watching, and where management needs to focus their efforts. Thus, it’s imperative that you go beyond the counting metrics offered by social media platforms to formulate outcome metrics that constitute real measures of success.

Identifying Outcome Metrics for Social Media Measurement

Stepping away from the pool for a moment, I ask you to consider why you’re participating in social media in the first place. Are you working to build awareness for your new products or services? Do you want to initiate a dialogue with your customers to solicit their input on what you could be doing more effectively? Are you building goodwill with consumers by giving back through social media and encouraging philanthropy? Or, can you increase your profits by selling directly through social media platforms? The answers to these questions reveal the business outcomes that you should be working towards when participating in social media. It’s only when you have a clear understanding of what you’re trying to accomplish with your social media efforts that you can develop truly effective measures of success. If you can’t pinpoint why you’re participating in social media today, or if your answers are flimsy and won’t stand up to the scrutiny of executive leadership, I strongly advise that you stop everything and rethink your efforts.

However, if you have a strategic vision of what you’re trying to accomplish with social media, then developing your outcome metrics will become a much easier task. For example, if gaining exposure is the outcome that you are after, then metrics like reach, velocity, and share of voice will be extremely helpful in determining your progress toward this outcome. Similarly, if you’re working to foster a dialogue with customers, focus on metrics like audience engagement, key influencers, and trending topics. Or if cold hard cash is what you’re after, then metrics like social referral source, cost per acquisition, conversion rates, and average order value will illuminate progress toward your stated social media outcomes. Each of these metrics tells you how well you’re doing according to plan and reveals valuable business information.

Demonstrating Social Media Business Value

Now that you’re straight on using counting metrics for sizing up opportunities and outcome metrics for quantifying purpose, the next step is tying all this together to communicate your fabulous progress. To do this, you need to detach yourself from the metrics that you use everyday to manage your social operations and translate these granular metrics into more generalized business language. Think carefully about the things that matter to your organization and the stakeholders that oversee the business and communicate in ways that resonate with them. In most cases, this means aligning your business objectives with corporate goals. Demonstrate which social media channels are contributing to new customer acquisition, which are adding dollars to the corporate coffers, or which are elevating customer satisfaction. This takes some skill and corporate savvy to indoctrinate non-believers into the world of social media metrics, but it’s an entirely worthwhile endeavor that will pay dividends for your organization in the long run.

I’ve found that the most effective way to present a strategic plan and communicate your successes using metrics is to leverage a framework for social media measurement. The one I use includes an inside-out strategy that begins with corporate goals, then aligns business objectives, maps these to measures of success, and then extends out to operational tactics. Using this framework allows me to solicit feedback from stakeholders by actually including them in the planning process of developing social media programs. This encourages participation and gives everyone involved a vested interest in the success of social media endeavors. Ultimately, your social media metrics should build from the basic counting metrics to outcome-based objectives that wholly support your corporate goals. Once you have a solid plan and a strategic roadmap for how you’ll stitch this all together, then you’re ready to dive into the deep end of the social media pool.

Analytics Strategy, Social Media

Monish Datta: "Justin Kistner KNOWS Facebook Measurement!"

We had a fantastic Web Analytics Wednesday last night, sponsored by Webtrends with social media measurement guru Justin Kistner providing a wealth of information about Facebook measurement (and Facebook marketing in general).

With almost 50 attendees, we were, as best as I can tell, tied with the largest turnout we’ve ever had. Is “number of attendees” an appropriate success measure? Well, yeah, it is. Even better that the group was super-engaged, and I’ve never had so many people track me down to laud the content (including multiple follow-up requests as to whether I had the deck yet!

Justin was gracious enough to share his presentation, and it’s posted below (click through to Slideshare to download the source PowerPoint):

A handful of pictures from the event:

Mingling/Eating/Drinking

Food, Drink, and Chatting

Justin Launches His  Presentation

Justin Gets Things Rolling

The Late Night Lingerers
(that’s Monish Datta in the middle — a wholly gratuitous reference in pursuit of SEO silliness)

The Late Lingerers

Adobe Analytics, Technical/Implementation

SiteCatalyst Implementation Pet Peeves – Follow-up [SiteCatalyst]

I recently blogged a list of my top Omniture SiteCatalyst implementation “Pet Peeves.” While the response to my post was very positive, one reader agreed with most of what I said, but disagreed with a few of my assertions or felt I had made some omissions. First, let me state that I always encourage feedback and comments to my blog posts since that helps everyone in the community learn. In general, the reader was making the point that my post only took into account an implementer’s perspective vs. the perspective of the web analyst. Personally, I don’t like to divide the world into implementers and analysts, since some of the best implementers I know also have a deep understanding of web analysis and vice-versa. Having been a web analytics practitioner using SiteCatalyst at two different organizations, I feel that I am in a good position to know if items I suggest (or discourage) will lead to fruitful analysis. I always try to write my blog from the perspective of the in-house web analyst who has to deal with things that I dealt with in the past, such as adoption, enterprise scalability, training, variable documentation, etc… In fact, I attribute much of my consulting success to the fact that I have been in the shoes of my clients and that they appreciate that my recommendations are based upon actual pains that I have experienced.

Since my original post was a very quick “Top-10” list and didn’t provide an enormous amount detail, and given the interest that it generated, I thought it would be worthwhile to write this follow-up post to address the concerns raised related to my post and to elaborate on the rationale behind some of my original assertions. In the process, it will become clear that I don’t necessarily agree with the concerns raised to my original post, but I am always cognizant of the fact that every client situation is different and every SiteCatalyst implementer has experiences that color their own implementation preferences. I don’t see it as my place to say which techniques are right and which are wrong, but rather to do my best to state what I think is/is not “best practice” and why based upon what I have seen and experienced over the past ten years and let my readers decide how to proceed from there…

Tracking Every eVar as an sProp

The first pet peeve I mentioned is when I find clients that have duplicated every eVar with a similar sProp. I stated that there are only specific cases in which an sProp should be used including a need for unique visitor counts, Pathing, Correlations and to store data that exceeds unique limits for accessing in DataWarehouse. The reader seemed to think I was being hard on the poor sProp and listed a few other cases where they felt duplicating an eVar with an identical sProp or adding additional sProps was justified including:

  1. Using List sProps – The reader suggested that I had made an omission, by not mentioning List sProps as another reason to consider using an sProp in an implementation. I maintain that the use of List sProps was justifiably covered in my statement of other sProp uses that are “few and far between.” I don’t use List sProps very often because I feel that there are better ways to achieve the same goals. As the reader stated, List sProps have severe limitations and there is a reason that they are rarely used (maybe 2% of the implementations I have seen use them). I have found that you can achieve almost any goal you want to use List sProps for by re-using the Products variable and its multi-value capabilities instead. By using the Products variable, you can associate list items to KPI’s (Success Events) rather than just Traffic metrics. Using the reader’s own example of tracking impressions, illustrates the differences perfectly. You can store impressions and clicks of internal ads and calculate a CTR using the Products variable and two success events. This also gives you charts for impressions, clicks and the ratio of the two which can be easily added to SiteCatalyst dashboards. I have found that doing this with a List sProp is difficult, if not impossible and reporting on it is tedious. For more information on my approach, please check out my blog post on the subject.
  2. Page-Based Containers & Segmentation – Here the reader suggested that the need to isolate specific pages using a Page View-based container is important to the life of the web analyst. Ben Gaines from Omniture also commented about this on my original post and I do agree that this can be useful for some advanced segmentation cases. I did not include this in my original list because I find it to be a much more advanced topic than I intended to cover for this quick “Top 10” post. While there may be cases in which you want to set an sProp to filter out specific items using a Page View-based segment container, I find that I often do this using the Page Name sProp which is already present. I do not see too many cases where a client is storing an eVar (let’s say Zip Code) and will say, “I am going to duplicate it as an sProp for the sole purpose of building a Page-Based container segment to include or exclude page views where a page is seen where a Zip Code equaled 123456.” Maybe that happens sometimes, but I still think it falls out of the scope of the primary things you should be considering when deciding whether to duplicate an eVar and I think it is a stretch to say that this functionality establishes the line between those who care about implementation and those who care about web analysis.
  3. Correlations – With respect to Correlations, the reader suggested that users correlate as often as they can since cross-tabulation is so essential to the web analyst. This is exactly why I included Correlations in my list! I also mentioned that this justification for using an sProp may go away in SiteCatalyst v15 where all eVars have Full Subrelations. Also, one of the reasons I prefer Subrelations to Correlations is that Correlations only show intersections (Page Views) and do not show any cross-tabulation of KPI’s (Success Events). Personally, I would disagree with the reader about over-doing Correlations, since in my experience, implementing too many Correlations (especially 5-item or 20-item Correlations), with too many unique values, can cost a lot of $$$, lead to corruption and latency.
  4. Pathing – In the area of Pathing, I think the reader and I are on the same page about its importance which is why I have published so many posts related to Pathing such as KPI (Success Event) Pathing, Product Pathing, Page Type Pathing, etc… Again, I might differ with the reader in that I don’t think enabling Pathing on too many sProps is a good idea since it can cost $$$ and produce report suite latency, which is why I prefer to use Pathing only when it adds value.

At the end of the sProp duplication section, the reader stated that there was no downside to duplicating every eVar as an sProp since it has no additional cost. To this, I would reiterate that my post was not advocating abandoning the use of sProps, but instead, attempting to help readers determine when they might want to use sProps so as to avoid over-using them when they will not add additional value. Even after years of education, I still find that many clients get confused as to whether they should use an eVar or an sProp in various situation, and most people I speak to welcome advice on how to decide if each is necessary.

However, I disagree with the reader’s assertion that duplicating every eVar as an sProp has no costs. Maybe it is due to the fact that I have “been in the trenches,” but in my experience I have seen the following potential negative ramifications:

  • Over-implementing variables and enabling features unnecessarily can cause report suite latency
  • Over-implementing variables can increase page load time, which can negatively impact conversion
  • Over-implementing variables and features can cost additional $$$ as described above (e.g. Pathing, Correlations)
  • When you implement SiteCatalyst on a global scale, you often need to conserve variables for different departments or countries to track their own unique data points. This means that variables (even 75 of them!) are at a premium. Therefore, duplicating variables has, at times, caused issues in which clients run out of usable variables.
  • Most importantly, however, is the impact on adoption. Again, I may be biased due to my in-house experience, but here is a real-life example: Let’s say that you have duplicated all eVars as sProps. Now you get a phone call from a new SiteCatalyst user (who you have begged/pleaded for six months to get to login!). The end-user says they are trying to see Form Completions broken down by City. They opened the City report, but were only able to see Page Views or Visits as metrics. Why can’t they find the Form Completions metric? Is SiteCatalyst broken? Of course not! The issue is that they have chosen to view the sProp version of the report instead of the eVar version. That makes sense to a SiteCatalyst expert, but I have seen the puzzled look on the faces of people who don’t have any desire to understand the difference between an sProp and an eVar! In fact, if you try to explain it to them, you will win the battle, but lose the war. In their minds, you just implemented something that is way too complicated. You’ve just lost one advocate for your web analytics program – all so that you can track City in an sProp when you may not have needed to in the first place. In my experience, adoption is a huge problem for web analytics and is a valid reason to think twice about whether duplicating an sProp is worthwhile. While I’ll admit that duplicating all variables certainly helps “cover your butt,” I worry about the people who are at the client, left to navigate a bloated, confusing implementation…

Therefore, for the reasons listed above, I remain steadfast in my assertion that there are cases where sProps add value and cases where they just create noise. While there will always be edge cases, I think that the justifications I laid out in my original post are the big ones that the majority of SiteCatalyst clients should think about when deciding if they want to duplicate an eVar as an sProp or use an sProp in general.

As an aside, while we are revisiting my original post, I thought of a few more items I wish I would have included so I will list them here:

  1. One other justification for setting an sProp I should have mentioned is Participation. There are some fun uses of Participation that can improve analysis and I find that sProp Participation is easier to understand for most people than eVar Participation so I would add that to my original list.
  2. If you do find a need to duplicate an eVar as an sProp, but it is only for “power users,” keep in mind that you can hide the sProp variable from your novice end-users through the security settings under Groups.
  3. Finally, I see Omniture ultimately moving to a world where there will only be one variable so if you want to be part of that world, please vote for my suggestion of doing this in the Ideas Exchange here.

VISTA Rules

Another pet peeve I mentioned is that I often find clients who are using VISTA rules too often or as band-aids. The reader stated that VISTA rules are a good alternative to JavaScript tagging since they can speed up page load times. I think this is another situation where my time working at Omniture and in-house managing SiteCatalyst implementations may bias my recommendations. While I agree that page load time is important, most Omniture clients I saw never mentioned using VISTA rules as a way to decrease page load time, but rather as a way to avoid working with IT! Usually, when I find a client that has many VISTA rules, it is because they have a bad relationship with IT, who doesn’t want to do additional tagging, rather than to save page load time. However, if I were to address the reader’s point of page load speed, I would agree that there are cases where using VISTA rules over JavaScript can decrease page load time, but I certainly do not think this should be the primary deciding factor. Great strides have been made in tagging including things like dynamic variable tagging and tag management tools which have greatly reduced page load speeds. I suggest readers check out Ben Robison’s excellent post on VISTA vs. JavaScript which discusses not only page load speed, but also the many other important factors to consider before jumping into VISTA rules.

Another point I’d like to make about VISTA rules is that, in my experience, they have a high likelihood of breaking and leading to periods of bad data. VISTA rules are like Excel macros. They do what you tell them to do, but if something changes, it can easily throw off a VISTA rule and cause incomplete or inaccurate data to be reported in SiteCatalyst. In this point, perhaps I am a bit jaded because I have seen so many different VISTA implementations go awry while I was at Omniture. In fact, it is rare that I find clients that have a VISTA rule that has worked for several years without ever having an issue. And if you do encounter an issue, you will have to pay Omniture around $2,000 to update it – every time. Want to make an update to the VISTA rule? $2,000. Want to turn off the VISTA rule or move it to a different report suite? $2,000! Consultants don’t have to write these checks, but guess who does – the in-house people do! This is why people are so excited about the new V15 processing rules and emerging tag management vendors. It is this tendency to break and the risk of bad data that makes me a bit gun-shy about using VISTA rules simply as a replacement for JavaScript tagging. Moreover, since the reader’s overall premise was that one must keep the web analyst in mind during implementation, I would be cautious about being overly-reliant on a solution like VISTA that is so prone to causing data issues which could thwart the analyst’s ability to do web analysis. I have seen companies that have 20+ VISTA rules and I promise you that they are not huge fans of VISTA right now (though they should really blame themselves not the tool!)! If you do pursue VISTA rules, my advice is that you consider using DB VISTA over VISTA. DB VISTA rules cost a bit more, but do offer more flexibility since you can at least make updates to the data portion of your rules without having to pay Omniture additional $$$.

One additional point to think about when it comes to VISTA rules is the impact they can have on report suite latency. Having too many VISTA rules can slow down your ability to get timely data in SiteCatalyst and I have seen many large organizations have severe (several days) report suite latency due to multiple VISTA rules acting on each server call. This impacts the web analyst’s ability to get the data they need and should be factored into decisions about VISTA rules.

As I stated in my original post, I have nothing against VISTA rules, but do find the overuse of them to be a potential red flag when I look at a new implementation. I often find that excessive use of VISTA Rules can be a symptom of bigger problems which merit investigation. Just like I don’t advocate duplicating sProps or enabling Pathing when not necessary, I don’t advocate the use of too many VISTA rules since it can be great in the short term, but bad in the long term. Now that I am a consultant again, it would be easy for me to recommend VISTA rules left and right, but since I like to have long-term relationships with my clients, I don’t do this since I know what it is like to be around later if/when they have issues!

Final Thoughts
I hope this post provides some good food for thought and more in-depth information about some of the items I listed in my original post. If you would like to discuss any of the above topics in more detail, feel free to leave comments here or e-mail me. Thanks!

Analytics Strategy

An Explanation of Sitecatalyst Events for the Google Analytics Power User

This post is one half of a 2-post series of which, most likely, you are looking for only one of the two posts!

Here’s the guide:

  • If you are well-versed in Google Analytics and are trying to wrap your head around Adobe Omniture (Adobiture) Sitecatalyst “events,” read on! This is the post for you!
  • “If you are well-versed in Sitecatalyst and are trying to wrap your head around Google Analytics “events,” then this sister post is probably a better read.

If you’re looking for information about Coremetrics or Webtrends…well, you’re SOL. If you’re looking for a great flour tortilla recipe, then my limited SEO work on this blog has run so far amok that I’ll just thank my lucky stars that I’m an analytics guy rather than a search guy (but, hey, here’s a great recipe, anyway).

Why “Events” Seem Similar in Google Analytics and Sitecatalyst

At a basic/surface/misleading level, events in Google Analytics and Sitecatalyst are similar. In both cases, they’re something that are triggered by a user action on the site that then sends a special type of call to the web analytics tool:

Alas! The similarity ends there! But, since no one learns multiple tools simultaneously, this surface similarity causes confusion when crossing between tools. Hopefully, these posts will help a person or two overcome that messiness.

Google Analytics Events — Conceptually

Let’s just start by making sure we’re on the same page when it comes to Google Analytics events. In a nutshell, they’re just a handy way to record user actions that don’t get picked up by the base page tag and that don’t get recorded as page views, right? They’re useful for recording outbound link clicks, activities within Flash or DHTML content that don’t warrant the full “page view” treatment (and, hopefully, you have a standard and consistent approach for determining when to use an “event” and when to use a “virtual page view”). Those are Google Analytics events at their most basic level, right? Okay. Cool. Let’s continue.

Sitecatalyst Events — Different in Concept from Google Analytics Events

In Sitecatalyst, events are much more conceptually similar to Google Analytics goals than they are to Google Analytics events (except, of course, when it comes to their name!). In olden times (web analytics olden times, that is), so I hear, Sitecatalyst events were actually called “KPIs” — a nod to the fact that many of the best KPIs are about what visitors do on a site, rather than simply being related to the fact that they arrived on the site in the first place.

So, a key point:

Sitecatalyst “events” really are more like Google Analytics “goals” than they are like Google Analytics “events.”

Sitecatalyst Events — Different in Implementation from Google Analytics Events

If we can agree that Sitecatalyst events are really more like Google Analytics goals than they are like Google Analytics events, then it’s worth pointing out that Sitecatalyst events are primarily set/captured/identified client-side (on a page), while Google Analytics goals are primarily configured/set on the back end by a Google Analytics admin user.

Google Analytics goals are typically established by specifying a specific visitor activity or behavior (or set of behaviors) using already-being-captured data (page title, page URL, time on site, number of pages viewed, or, as of v5, triggering of a specific event category/action/label/value) as a “goal” within a profile. Once the goal is defined, you can track the number of visitors who complete that goal, the conversion rate, and a conversion funnel. And, you can do this (funnels excepted…unless you’re prepared to get fancy pants about it) by visitor segment. In short:

Google Analytics goals are primarily configured on the back end.

Now, you may occasionally do some tweaking on the client (page) side of things in order to enable you to set up the goals, but I’m not going to digress on that point (leave a comment if you’d like me to elaborate and I will).

With Sitecatalyst events, the main work occurs on the client side of things. You establish what activities/occurrences warrant “event” status, and then you make sure your site is configured so that the appropriate event(s) gets recorded any time that activity occurs. Typically,the event occurs at the same time — and in the same Sitecatalyst page tag call — that a view of a page (the pageName) and various other data gets passed to Sitecatalyst. For instance, when a visitor completes a site registration, the Sitecatalyst page tag will likely need to record the traffic to the confirmation page (via pageName and additional sProps) as well as the “event” of a registration being completed. This event (or “completion of a goal”) will be recorded as “event=eventx” in the page tag call. To make use of “eventx” (event1, event2, etc.) requires some backend configuration (including whether the event is serialized or not…but that’s another digression I will avoid). But, for the most part:

Unlike Google Analytics goals, Sitecatalyst events are primarily set up client-side — via values in the “event” variable recorded by the Sitecatalyst page tag.

eVars (aka “conversion variables”) and Sitecatalyst events –> Google Analytics Segments

With Google Analytics goals, it is handy to use segments — standard or advanced — to explore how different subsets of visitors behave. For instance, how do visitors who viewed product details tend to convert to an order as opposed to visitors that do not? Do visitors that entered the site via organic search reach product details pages at a higher rate than visitors who arrived as direct traffic? You get the idea.

Excepting the segmentation capabilities of the lugubriously rolling out v15, as well as the capabilities of Discover, Sitecatalyst relies largely on eVars to do this sort of exploration. With scads of available eVars to work with, and with the appropriate use of subrelations (allowing the cross-tabulation of eVars), it’s largely a matter of solid up-front thinking and planning, combined with a willingness (and ability) to make site-side adjustments over time, to get at “segmented conversions” using Sitecatalyst.

eVars are both a blessing and a curse when viewed through a Google Analytics lens. They’re a blessing because they allow much more detailed and sophisticated capture and analysis of conversion data. They’re a curse because they require much more site-side planning and implementation work!

Does This Help?

Describing  this distinction in a clarifying manner is tricky — it’s quite confusing and frustrating…until it makes sense, at which point it’s hard to identify exactly what made it so confusing in the first place!

If you’ve gone through (or are going through) the process of adding Sitecatalyst to your toolset after being deeply immersed in Google Analytics, please leave a comment as to how you overcame the “events” hurdle. With luck, others will benefit!

Analytics Strategy

An Explanation of Google Analytics Events for the Sitecatalyst Power User

This post is one half of a 2-post series of which, most likely, you are looking for only one of the two posts!

Here’s the guide:

  • If you are well-versed in Adobe Omniture (Adobiture) Sitecatalyst and are trying to wrap your head around Google Analytics “events,” read on! This is the post for you!
  • If you are well-versed in Google Analytics and are trying to wrap your head around Sitecatalyst “events,” then this sister post is probably a better read.

If you’re looking for information about Coremetrics or Webtrends…well, you’re SOL. If you’re looking for a great refried beans recipe, then my limited SEO work on this blog has run so far amok that I’ll just thank my lucky stars that I’m an analytics guy rather than a search guy (but, hey, here’s a great recipe, anyway).

Why “Events” Seem Similar in Google Analytics and Sitecatalyst

At a basic/surface/misleading level, events in Google Analytics and Sitecatalyst are similar. In both cases, they’re something that are triggered by a user action on the site that then sends a special type of call to the web analytics tool:

Alas! The similarity ends there! But, since no one learns multiple tools simultaneously, this surface similarity causes confusion when crossing between tools. Hopefully, these posts will help a person or two overcome that messiness.

Sitecatalyst Events…Conceptually (Just to Make Sure We’re on the Same Page)

With Sitecatalyst, an event is a success event — a conversion to an on-site activity you care about. I was told by a reliable source that, in early versions of Sitecatalyst, events were actually  called KPIs — a nod to the fact that many of the best KPIs are about what visitors do on a site, rather than simply being related to the fact that they arrived on the site in the first place. So, are we cool with that high-level definition of a Sitecatalyst event? Good. Let’s continue…

Google Analytics Events — Conceptually, a Completely Different Animal

A Google Analytics event is very different in both concept and in application from a Sitecatalyst event. It’s much more akin to Sitecatalyst link tracking or a “non-standard” Sitecatalyst call triggered for the sake of counting some activity other than viewing of a basic HTML page (viewing of content in Flash or DHTML…although these also use virtual page views — more on that in a bit) either via pageName or an sProp.

Events are, simply put, a way to record a user action that warrants being recorded, but that does not warrant being counted as a page view.

While events in Sitecatalyst, almost by definition, are “significant” actions — a product added to a car, an order completed, a product details page viewed, a site registration — Google Analytics events are often of much less on-going importance. For instance, they may include:

  • The use of minor navigational buttons or elements (in Flash, in DHTML, or elsewhere)
  • The reaching of a certain point (half viewed, 3/4 viewed, 95% viewed) in a streaming video
  • The exit from the site on an outbound link (virtually identical to Sitecatalyst “exit links,” but requiring explicit coding/customization to track using Google Analytics events)

Up until the most recent release of Google Analytics, and much to the chagrin of analysts the world over, Google Analytics events could not be set as “goals,” and goals in Google Analytics — configured by a Google Analytics admin user rather than anywhere in the page tag — are the closest that Google Analytics comes to the concept of a Sitecatalyst “event.”

Did you catch that? If you’re looking to implement something akin to a “Sitecatalyst event” in Google Analytics, read up on Google Analytics goals.

“eventx” (Sitecatalyst) vs. Category/Action/Label/Value (Google Analytics)

Another, albeit lesser and secondary, difference is that Google Analytics events are named and categorized at the point when the event is fired by a user action. As such, you won’t see a Google Analytics event called event1, event2, etc. that subsequently needs to be described and named on the back end. Google Analytics events have the requisite meta data built into the page-side recording of the action through the inclusion of a “category,” an “action,” and an (optional) “label” and “value” that will then appear in Google Analytics event reporting just as the values were called from the page. Plentiferous detail on the mechanics and syntax are available in the Google Analytics documentation on event tracking.

This is similar to how Google Analytics handles campaign tracking — all of the meta data about a campaign is included in multiple parameters in the target URL for the campaign, whereas, with Sitecatalyst, you have the opportunity to simply use SAINT classifications to map an alphanumeric campaign tracking ID to a range of different classifications.

Google Analytics Events vs. Virtual Page Views

One final area of confusion is the popular “when to use an event versus when to use a virtual page view in Google Analytics” conundrum. Sitecatalyst power users transitioning to Google Analytics can get all sorts of twisted in the head on this, as the basic question just doesn’t make sense…if they’re thinking about “events” in Sitecatalyst terms. If the question doesn’t make sense to you, er…re-read the first half of this post (or leave a scathing comment as to how non-elucidating the first part of the post is!).

When to use one or the other is both situational and a judgment call. The general rule our analysts apply is to consider whether the activity meets either of the following conditions:

  • Activity that is already being recorded as a standard page view elsewhere (e.g., clicks on home page promo areas where the target URL is on the same site and will soon be recorded as a page view…but where you want to be able to easily report or analyze individual promo or promo location clickthroughs)
  • Activity that the visitor would not consider as “viewing a new ‘page’ of content” (e.g., reaching the halfway point in a streaming video)

If either of these criteria is met, our bias is to record the activity as an event rather than as a virtual page view.

(Until recently, the exception to this criteria was if the user action was a “goal” for the site. Since events could not be set as goals, we would be required to a virtual page view. But, Google has, happily, added the ability to use events as goals in v5!)

Does This Help?

Describing  this distinction in a clarifying manner is tricky — it’s quite confusing and frustrating…until it makes sense, at which point it’s hard to identify exactly what made it so confusing in the first place!

If you’ve gone through (or are going through) the process of adding Google Analytics to your toolset after being deeply immersed in Sitecatalyst, please leave a comment as to how you overcame the “events” hurdle. With luck, others will benefit!

Adobe Analytics, Conferences/Community, General

Great jobs and a great gathering in Atlanta next week

Just a quick note from my vacation getwaway to call reader’s attention to two great jobs at The Home Depot and to let Atlanta-area readers know that I will be in town next week for a special “Web Analytics Wednesday on Tuesday” put together by Keystone Solutions Rudi Shumpert and HD’s own Wesley “Big Wes” Hall. The event will be at the Gordon Birsch in Buckhead and I’m hoping that Rudi and Wes will allow an informal Q&A session about some of the great things that are happening in our industry lately.

>>> Register to join us at Web Analytics Wednesday, Atlanta, on Tuesday, July 19th

Regarding the jobs, our client at Home Depot is aggressively putting together a team of digital measurement specialists to help lead the company’s digital efforts forward. We have been helping the company with their digital measurement strategy now for about six months and the effort is really beginning to pay off in terms of their use of technology, the talent they are getting in the door, and the value web analytics brings to the company both online and off.

Have a look at the Senior Analyst and Manager, Web Analytics jobs on our web site and come see me next week at Web Analytics Wednesday if you’d like a personal introduction or have any questions:

>>> Job description, Senior Web Business Analyst at The Home Depot

>>> Job description, Web Analytics Manager at The Home Depot

I hope you are all having a great, relaxing summer and look forward to seeing you at a conference, event, or Web Analytics Wednesday sometime in the near future.

Analytics Strategy

Quick Tip: Track Your Omniture JS File Version [SiteCatalyst]

Have you ever had someone run a report in SiteCatalyst and come running to you saying something like this?

This report doesn’t make sense…There is obviously a tagging issue and you need to fix it ASAP!

If I had a dollar for every time this happened to me, I’d be a rich man! The truth is, that after many wild goose chases, the problem is not usually tagging related (note you can use tools like ObservePoint and DigitalPulse to verify). But if it ever was tagging that caused the issue, it was usually related to the release of a new JavaScript file. That has been the culprit many times for me over the years. Therefore, in this post, I will share a trick you can use to easily find out if data issues you might be experiencing might be related to a new JavaScript file release.

Tracking Your JavaScript File

So how can you use SiteCatalyst to determine if a new JavaScript file you released is wreaking havoc on your data? For example, let’s imagine a scenario where the morning of May 18th, you started seeing some strange data irregularities (possibly by checking Data Quality as described here!). Here is what you need to do:

  1. Each time you create a new version of your JS file, assign it a version number (i.e. 0.5, 0.8, 1.2)
  2. Pass this version number into a tool that can store it and let you know when it sees the version number change
  3. Look at a report that shows you when the version number value has changed (what date it changed and at what time)

Sounds easy right? If only we knew of a tool into which we could pass data, have it be time-stamped and report upon changes in version number values…Hmmm….Where would we find such a tool??

Obviously, we already have that tool and it is SiteCatalyst! We can use the tool we know and love to track each version of the JavaScript file by simply passing in the version number of the file into an sProp on every page (and yes, I get the irony that we are using a JavaScript file which sets a beacon to enable tracking to track itself!). By doing this, we will have a historical record of when each JavaScript file was released. After you pass in the JavaScript File version you will see a report like this:

Here we can see the distribution of page views related to each JavaScript file version. In this case, we have been busy and have had four JavaScript file changes in one month! However, this report isn’t super-useful in answering our initial question: Were the issues we saw on the morning of May 18th related to a new JavaScript file release?

To answer this question, all we have to do is to switch to the “trended” view of this report and we will see a report like this:

Now we can start to see the flow of JavaScript file updates. Looking at this report, we can see that we moved from version 0.5 to version 0.7 (poor version 0.6!) on May 18th… This might support our hypothesis, but to be sure, we can look at this report by hour on May 17th & 18th and see this:

 

Now we can narrow things down to an hour and it looks like the JavaScript file did, in fact, change around 9:00 am on May 18th. As you can see, the simple action of taking the administrative step of keeping your JavaScript file in an sProp can provide an easy way for you to do some sleuthing when you are in a pinch!

Additionally, if you want to further test your hypothesis, you can isolate data that is related to a specific JavaScript file to see if it represents the issue you are seeing. To do this, simply use DataWarehouse to create a segment that only pulls pages that had data collected using a specific JavaScript file version as shown here:

 


Adobe Analytics

Some SiteCatalyst Implementation Pet Peeves [SiteCatalyst]

Over the years, when I have consulted clients who use the SiteCatalyst product, I have encountered some strange implementation items that made me scratch my head. In the beginning, when I saw these odd implementation quirks, I was mildly entertained, but as I saw them more and more, they were soon elevated to “pet peeve” status. Therefore, I thought I’d share some of these items with you to make sure that you are not doing any of them, and also because I am curious to see what other “pet peeves” you may have. Please check out my list (which is by no means exhaustive!) and if you have items that you have seen that bug you, please leave them here as a comment!

Tracking Every eVar as an sProp

I would say that my biggest pet peeve is when clients have an sProp for every eVar they have set (or vice versa). When I see this, it is an early warning sign that the client doesn’t fully understand the fundamentals of SiteCatalyst. While there are definitely cases where you would capture the same data in both an eVar and an sProp, they are usually few and far between. As a rule of thumb, I only set an sProp if:

  • There is a need to see Unique Visitor counts for the values stored in the sProp
  • There is a need for Pathing
  • You have run out of eVar Subrelations and need to break one variable down by another through the use of a Correlation (which will go away in SiteCatalyst v15)
  • There will be many values (exceeding the unique limits and you just want data stored so I can get to it in DataWarehouse or Adobe Insight

For the most part, that is it… Beyond that, I tend to use eVars and Success Events for most of my implementation items.

This is why I shudder when I see 40 eVars set and the same 40 sProps set. I find that this only confuses users since most don’t really understand the difference between the two variable types to begin with! Therefore, my advice is to make sure you understand the difference between eVars and sProps and make sure you use the right variable for the right purpose.

Pathing Enabled Unnecessarily

Another item I have seen a lot is when a customer will have Pathing enabled on an sProp that doesn’t change in a session. For example, let’s say you have people log into your website and you store the Customer ID in an sProp. That Customer ID is designed to be the same for each visitor during the entire visit. However, I often see clients who enable Pathing on this Customer ID sProp. My hunch is that they think this will show them the paths of that Customer ID, but the truth is that it will show no paths for each Customer ID so it is a complete waste of time. Keep in mind that pathing is only useful if values change in the same session. If you pass the same value in on every page of the session, SiteCatalyst will see that as a Bounce of 100% for every Customer ID! Since Adobe (Omniture) will only let you have so many variables with Pathing enabled, you need to make sure you are using them wisely!

No Friendly Page Names

The next pet peeve is when clients don’t pass any values to the Page Name variable and use the default of the URL. This really makes my blood boil! There are so many downsides to doing this when it comes to the Pages report since it impacts Page Views, Unique Visitors and Pathing. For better or worse, the Pages report tends to be a very popular one and I feel that, even if just for the perception of the integrity of your web analytics implementation, you need to take the time to make sure this report is accurate and understandable. For more information on this topic, please refer to my Page Naming Best Practices post by clicking here.

Passing Query Strings to Page Name Variable

On a related note, I have another gripe related to the Page Name variable and it has to do with query string parameters. Many times I find that companies are including query string parameters in the Page Name variable. This is a really bad idea. Here are two common things I see:

  • When a visitor arrives to the website from a campaign, the URL will have a campaign code in the query string parameter and pass this to the Page Name variable (i.e. zyz corp:home:homepage:cid-12345)
  • A company will have a search results page and include the keyword/phrase that the user searched upon to get to that page in the Page Name (i.e. zyz corp:search:searchresults:user manual)

Both of these examples involve the company having one page name essentially split out into hundreds (or thousands) of versions of the Page Name due to the query string parameter. Creating many versions of the same page has the effect of losing Visits, Unique Visitors and Pathing for the true Page Name. Most of the time this situation can be solved by using one Page Name and passing the query string parameter to another variable and using a Correlation. If you really need to have these extra query string parameters associated with pages, I recommend using another sProp instead of the Page Name variable…

Reports with No Data

Another thing I see quite often are implementations that have tons of variables labeled, but that have no data. As a rule of thumb, I recommend you disable any variables that have no data or at a minimum hide them from the menus using the Admin Console. There is nothing more frustrating to an end-user than opening up a report, getting excited to see the data and then realize that there is none! Besides being annoying, it hurts the credibility of your web analytics program. When I am in the midst of a new implementation and things are in flux, one thing I do is to put all reports that are coming, but have no data in ALL CAPS or I add the phrase “(COMING SOON)” after the variable name. This helps me see which variables are left to do and which ones I can begin to QA. However, once the implementation is semi-stable, I urge you to hide variables that are not coming for a while so you don’t annoy people unnecessarily!

No Menu Customization

On a related note, how many SiteCatalyst implementations have you seen where they use the default menu structure? Why would you want to tell users to look in “Customer Conversion 1-10” to find the report they are looking for? Not very helpful is it?

Instead, you should customize your menus so they make sense for your users. This will help in your adoption and make training much easier. For some great tips on how to customize your menus, check out Brent Dykes’ post by clicking here.

No Variable Standardization

The next one is when you have a situation where you have multiple report suites that are really the same website, just for different business units and/or locations and none of them are set-up consistently. I see many clients who are tracking some things in the US, but not in the UK or Japan, even though the websites are identical. When this happens and you select multiple report suites in the Admin Console, here is what you see in the variable screen:

I call the “Multiple Madness” due to what you see in the Admin Console and it is not a good thing! You should make sure that as many of your report suites are as consistent as possible so you can minimize your development time and roll-up data into higher-level report suites.

Wasting of Variables

This next one is a minor one but it is related to wasting variables. Even though there are more variables available now, it doesn’t mean that you should track everything or that every piece of data requires its own variable. For example, I recently ran into a client that was tracking Salutation (Mr., Mrs., Dr., etc…) in an eVar. This makes very little sense. How are you going to do cutting-edge analysis on that? Gender maybe, but I don’t think Salutation is worthwhile. Just because you know it, doesn’t mean you need to track it.

This leads to the other type of waste I see – not using SAINT Classifications to save variables. There are many cases where you can accomplish the same analysis objectives by using SAINT Classifications and save variables along the way. Using the prior example, instead of storing Salutation as an eVar, if you really need it, why not store a Customer ID value and then add Salutation as a classification value of that Customer ID? That saves you one eVar and if you happen to have Full Subrelations on that eVar, you get them on the classification of that eVar as well (which will be less of an advantage when using SiteCatalyst v15 since all eVars will have Full Subrelations).

But here is my favorite example since I see this all of the time! One of Omniture’s common JavaScript Plug-ins is the Time Parting plug-in. This allows you to see data segmented by Day of Week and Hour of Day. However, many clients also store an sProp and/or eVar for Weekday/Weekend through this plug-in. It makes sense that you might want to segment data by Weekday/Weekend, but why use an entirely new variable just to track the binary values of Weekday vs. Weekend? You can easily do a one-time classification of Day of Week and lump Mon-Fri into “Weekday” and Sat-Sun into “Weekend.” That will allow you to achieve the same goal, but saves a variable. Again, this is a minor annoyance, but it is the principle that counts. You can extrapolate this concept by thinking back to the Customer ID example I mentioned above. What if there were ten data points related to a customer that you chose to store in ten separate eVars? You might be able to make these classifications and save ten eVars!

My advice here is to just be thoughtful when assigning variables and if you have cases where there is a direct relationship between two variables that won’t change very often, consider using a SAINT classification and also think about whether you will ever use that data point for an analysis before tracking it in the first place.

VISTA Rule Chaos

The final pet peeve I will mention is related to VISTA Rules. Let me start by saying that VISTA and DB Vista rules are not bad. They can be very powerful, but it is also true that they can be easily misused and wreak havoc on a SiteCatalyst implementation. When using VISTA rules, it is critical that you and your entire team understand WHEN the rules are being used and WHAT they do in terms of setting variables. I have seen many cases where a developer will change a variable not knowing that there are VISTA rules impacting it. You need to make sure VISTA rules are heavily documented and as you change your site or implementation, they need to be factored into the equation. One suggestion I have is to add the phrase (SET VIA VISTA) in the name of any variable that is set via a VISTA rule in your documentation so there is no missing it!

The other pet peeve I have related to VISTA rules is when they are used as a “band-aid” to avoid doing real tagging. In the long-run, this always comes back to haunt you. I see many clients creating band-aids on top of band-aids until things fall apart. I am ok with companies using Vista rules to get things done quickly, but I recommend that, over time, you phase out as many VISTA rules as you can and move their logic to your regular tagging so you have all of your logic in one place.

Final Thoughts
Well, there you have it. Not all of my implementation pet peeves, but a bunch of them that popped into my head. I am sure you have seen some fun ones out there and I’d love to hear about them…Please leave them as comments here!

NOTE: For more details on these points, check out my follow-up post here.

Analytics Strategy

U.S. Privacy and Data Security Legislation Summary/Recap

Andy Kennemer, VP of Social Marketing & Media at Resource Interactive, recently attended the NRF Washington Leadership Conference, which included a meeting of the Shop.org Policy Advisory Group (PAG) meeting, of which he is a member. A major focus of the PAG meeting was the increased legislative focus on privacy and data security. Andy agreed to summarize some of the highlights for me to share here.

Legislation is cyclical, and we’re in a hot period right now.

The focus of our meeting was discussing in detail the various legislative actions in Congress regarding both online privacy and data security. These two issues are separate, but related, and the more they are mixed together in legislation, the more complicated and ambiguous it will make things for retailers and brands.

Last year we saw 3 main efforts:

  1. The FTC’s attempt to establish rule-making authority through a new US Privacy Framework proposal;
  2. Rep Boucher’s attempt to introduce an online privacy bill, which primarily would support the notion of consumer opt-IN to 3rd party tracking; and
  3. Sen. Pryor introduced a bill addressing Commerce Data Security.

The FTC is likely to release a final “staff report” on this matter sometime this year. The Boucher and Pryor bills never made it to the floor for debate.

This year, mainly in the last 3 months, we have seen a flurry of activity like never before.

Key privacy bills introduced this year:

  • Sens. Kerry & McCain introduce broad privacy bill (4/12/11)
  • Reps. Stearns & Matheson introduce broad privacy bill (4/13/11)
  • Sen. Rockefeller introduces “Do Not Track Online” bill (5/9/11)
  • Reps. Markey & Barton introduce “Do Not Track Kids” bill (5/13/11)
  • Sens. Franken & Blumenthal introduce Location Privacy bill (6/15/11)
  • Sen. Wyden & Rep. Chaffetz introduce “GPS” Privacy bill (6/15/11)

Key data security proposals:

  • White House releases Cyber Security proposal (5/25/11)
  • Sen. Leahy re-introduces Judiciary bill from 111th Congress (6/8/11)
  • Dept of Commerce “Green Paper” on Cyber Security framework (6/8/11)
  • Rep Bono Mack revises House-passed data security bill from 111th Congress (6/10/11)
  • Sen. Pryor re-introduces Commerce Bill from 111th Congress (6/15/11)

With so much activity, it’s challenging to even keep track of everything, and which bills and proposals matter the most. There are a few of these that are gaining momentum that, as an industry, we need to watch. The Kerry-McCain bill, White House Cyber Security proposal, and potential final report from the FTC on the privacy framework will have the broadest impact to brands and retailers.

For now, the feedback from a congressional staffer who attended the meeting was:

  • There is a fear of modernity within the government. What needs to be better articulated is how data collection is used to actually help consumers, to have a more relevant and enjoyable online experience.
  • We need the voice of actual consumers. Right now consumer advocacy groups have influence, but it isn’t clear if they really represent the concerns of average consumers.
  • Retailers have not adequately addressed the consequences of legislation, in terms of actual economic harm, or hindering innovation. Some sort of cost / benefit / risks analysis could be helpful (e.g., What does our online experience look like if advertising is not as effective? Are consumers ready to pay for services that are currently free and ad-supported?

This continues to be a complex and rapidly evolving area, and brands cannot afford to simply put their heads in the sand and hope it goes away. Legislation will get passed, but the extent and impact of that legislation is far from clear.

Social Media

TakeFive with TweetReach

TweetReach has started up a new interview series on their blog called TakeFive with TweetReach. The goal of the series is to “provide insight and commentary from notable members of the social media analytics and measurement community, with the goal of facilitating an ongoing conversation around all things measurement.” I’m not going to quibble with their loose interpretation of the term “notable.” I was happy to participate!

Some of my favorite quotes (from me…how egocentric is that) from the interview:

We use our measurement planning process to ensure we have alignment across the stakeholders involved in the campaign. For each tactic or channel, we try to make sure we’re all in agreement on the answers to two questions (we actually call these “the two magic questions”): 1) what is the tactic supposed to do? (these are the objectives for the tactic) and 2) how will we know if it did that? (these are the key performance indicators).

And:

Marketers tend to operate with a hefty level of cognitive dissonance: on the one hand, touting the importance of multi-channel marketing that has congruent and complementary messaging…and then asking, “What’s the value of a fan of my Facebook page?”

And, finally:

When I’m presented with a, “You must prove the ROI of our social media investment!” decree, I tend to redirect slightly and ask the requestor if what they really want to know is, “Did I efficiently and effectively invest in this effort and garner meaningful, quantifiable results from that investment?” If I can get agreement on that…

Brilliant stuff, I say! Check out the full interview on the TweetReach blog!

Adobe Analytics, General

SiteCatalyst Advanced Search Filters [SiteCatalyst]

One of the features that I find deceptively difficult at times in SiteCatalyst is the use of the Search feature. I feel like there are many times I use this and end up messing it up. Therefore, I decided to do my best to share what I have learned about what works and doesn’t work in the hopes that it will save you aggravation and time! I also hope that many you can add a comment to this post with your tips and tricks so we can all learn something…

The Basics

First, let’s start out with the basics. Hopefully if you are a SiteCatalyst user you know that the search function is used to filter results in eVar and sProp reports. You simply enter a value and SiteCatalyst will look for those values in the active report and return those rows. This is handy because you can bookmark reports, make custom reports or add reports to dashboards after you have created the filter so that you never have to apply it again.

For example, let’s start with a Pages report like this:

Obviously we have pages from all sorts of countries, but if we only wanted to look at pages from England, all we would have to do is enter “SFDC:uk:” in the search box (top-right) and we would then see a report like this:

But what if we wanted to see pages from England or France? At this point we have two options. You can either enter “SFDC:uk: OR SFDC:fr” in the search box or use the advanced search editor. Here is what it would look like with the OR statement in the regular search box (look at the top-right portion):

However, believe it or not, if you change the “OR” to be a lower case “or” you will get no results! I kid you not! I call that an “Omniture-ism” and you just have to remember it…

The other way to get to the same report is to use the Advanced Search tool. You get there by clicking on the Advanced link to the right of the search box. Once there, you would enter the appropriate phrase in the first box, click the “+” sign to add another search criteria and then enter the second phrase so it looks like this:

However, it is important that you change the top drop-down box from the default of “if all criteria are met” to “if any criteria are met” or you will get no results.

If you wanted to look for cases where there were pages on the UK website that had the phrase “form” in the pagename, that would be a case where you would use the “if all criteria are met” option and your query should look like this:

This would result in a report like this:

Finally, we can come full-circle and get more advanced and use an “AND” statement in the standard box to get the same result. Here is what the search box would look like:

Again, keep in mind that the “AND” is case-sensitive…

More Difficult Searches

So now that we have covered the basics, let’s get a bit more advanced. First, let’s keep going with our example and say that we need to find all pages in the UK or France that have the word “form” in them. This gets a bit tricky because we are mixing OR and AND statements. Using the Advanced Search query builder, here is how you would enter it:

Conversely, if for some reason we wanted to see any UK Pages that had the phrase “form” in them and all France pages (not sure why, but this is just an example), we would enter this:

Which would result in a report like this:

Note that in this case we had to change the drop-down box back to the “any criteria” option since we did the AND statement within one of the criteria (hey…I told you this was the difficult part!).

The trick here is to combine any OR and AND statements into each row since each of the individual search criteria have to be either an “AND” or “OR” clause.

On a separate note, in the advanced search area, you can change the drop-down which defaults to “Contains” to “Does Not Contain” so if, for example, you wanted to see all UK pages, but exclude those that had “login” in the name you would enter the following criteria:

Note that for this instance, we need the “all criteria are met” option…

Finally, just for fun I entered the following phrase in the “simple” search box…

…and miraculously it produced the same results!! I decided to stop here before I broke anything, but you can feel free to see how far you can push this!!

But wait…There’s more! I have been amazed by how few people I meet know this next one… Imagine that you are looking at an eVar report and you have broken it down by another eVar via Subrelations. Here is an example where I have taken the Site Locale eVar and broken it down by Internal Search Term:

Now, let’s say that you wanted to do a search filter to only see items that mention “Outlook.” The easy way to do this is to just enter the phrase “Outlook” in the search box and SiteCatalyst will show any rows that have that phrase. But what if you wanted to see the phrase “Outlook” in just United States or Japan? No matter what you put in the search box, you will not get the results you are looking for (i.e. outlook AND “united states” OR japan). Would you know how to do this? Most people I meet don’t. Here is how…

When you are using a Subrelation report, you have to keep in mind that SiteCatalyst is running two reports and it doesn’t know which report you want to filter on. Therefore, we need to tell SiteCatalyst which report we want the search term to be associated with. You can do this in the Advanced Search area. When you have a Subrelation report, and you click on the Advanced Search area, you will see a new option that allows you to select one of the two reports being subrelated like this:

Most people haven’t ever noticed this new option so now that we know it is there, all we have to do is select the right report and then enter the search term in the right report and we can get our results. For the example above, we would enter “Outlook” in the search box next to Internal Search Term and “United States OR Japan” in the search box next to Site Locale like this:

Now, since we have been a bit more specific, we can get a nice, clean report like this:

Just keep this handy feature in mind the next time you are trying to search in a Subrelations report and pulling your hair out because you can’t get the results you think you should!

Even More Difficult Stuff

Phew! If you’ve made it this far, you are really devoted to your craft. We’re almost there so hang on…

The next thing that is important to know is that you can use wildcards in your searches. To do this, you use the “*” symbol in the search query. For example, if we wanted to find any pages in the UK that has the phrase “landing” somewhere in the name, we could simply do a search like this:

The next thing to know is that Omniture can be a bit quirky when it comes to the [SPACE] separator in the search box. Let me illustrate. If I enter the phrase “home page” in the search box, here are the results I get:

This seems strange to me since none of these pages have a space in them. That would make you think that a [SPACE] is a valid separator and that this query is the same as “home OR page” right? But if I use that logic and enter this phrase “SFDC:uk: SFDC:fr:” which is really just two phrases separated by a space (just with a colon in the phrase), I get no results. I am sure there is a logical reason for this, but I am not sure what it is. Maybe if SiteCatalyst sees a “:” or a “|” it acts differently (maybe Jorgen can enlighten us on this)?

To be safe, I use the next feature – using quotes – whenever possible. My advice is that if you ever have phrases with spaces in them that you enclose them in quotes and stick to using OR statements. In the preceding example, if I change my “home page” query to be “home page” in quotes, I get the expected result which is no results. Another lesson to be learned here is that you should, whenever possible, avoid putting spaces in values that you think you will search upon. I do my best to remove all spaces from page names since that is the variable I search on the most!

Finally, you can use the “-” sign to remove things from search results. This produces the same effect as using the “Does Not Contain” feature in the advanced search area. As in the previous example, if I want to see all UK pages, but not ones that have the phrase “login”, I can enter the following in the search box:

To see UK pages that do have login in the name, you can also enter this phrase:

But when the results come back, it will mysteriously remove the “+” sign and just uses space as the separator producing the same results.

Final Thoughts…
So there you have it! Pretty much everything I know about using search and advanced search in SiteCatalyst. Do you have any additional tips or tricks? If so, leave a comment here…Thanks!

Adobe Analytics

My Latest SiteCatalyst Wishlist Items [SiteCatalyst]

A few weeks ago I was at the European Adobe (Omniture) Summit in the UK and had the pleasure of being in another one of Brett Error’s “what features are we missing” sessions. I find these sessions to be good and bad at the same time. The good part is that people are expressing what they need and others can validate or invalidate ideas in real-time. The bad part is that I often feel that the features that get voted up are the ones that are easy to understand (like Bounce Rate as a standard metric!), but that there are many features that people SHOULD want, but don’t know it yet. I don’t mean that to come out as sounding pretentious, but the fact is that many people have been using the product for only a few years and it is natural that the needs of those who have been using the product for many more years will have some more advanced feature requests. Unfortunately, many of these advanced features, no matter how important, will be trumped by more basic, globally understood feature requests.

The creation of the Ideas Exchange has been a great help in getting ideas big and small into the product and I am so pleased to see that many of the ideas in there have been added to the product and for that I commend Adobe (Omniture). I think the positive feedback around SiteCatalyst v15 is a direct result of people seeing their ideas manifested in the release.

In this post, I wanted to highlight a few ideas that are in the exchange that might not get as much “play” as they should and why I think they should be undertaken. If you agree and have a Login ID to SiteCatalyst, please feel free to login and vote for them!

SAINT Auto-Classifications
One of the ideas that came up in the UK session I mentioned earlier (and received the most votes!) was the notion of SAINT Auto-Classifications. This idea was submitted by Ben Gaines (probably as an initial test of the Idea Exchange!) the day the exchange came online. As most users know, SAINT Classifications are a way to add meta-data to values you have already captured in SiteCatalyst. It is similar to a pivot table in Microsoft Excel. However, SAINT Classifications have to be uploaded manually and it becomes very tedious over time. The feature request is to provide a way where administrators could set-up rules to auto-classify items or classify them on the fly (as reports open up). For example, if I have a report of campaign tracking codes and a bunch of them start with “seo|,” I could set something up where these would all be automatically classified as “SEO” in the Marketing Channel classification I have set-up. This is just one example, and the possibilities are endless.

The great news is that this idea has recently been changed to “Under Review” and geniuses like Sean Gubler have started playing around with tools to do this so I feel like it is only a matter of time before we see this. Keep your fingers crossed and vote for the idea by clicking here.

Multi-Session Attribution (Allocation)
The next idea is related to eVar attribution. Currently, you can attribute success to eVar values for First Touch, Last Touch. There is an option for Linear allocation, but that only works within one session so it is rarely used. The closest thing available for multi-session attribution is the Cross-Visit Participation plug-in which is really just a “hack” that concatenates eVar values into one string. This plug-in can be useful at times, but has some serious drawbacks.

In today’s world of people bouncing between websites and social media, you cannot count on the visit that people convert being the same one in which they came from a marketing campaign. Therefore, you often have cases where a visitor comes from an SEO keyword, does some product research, leaves the site, comes back the next day from a paid search ad, leaves the site and then comes back a third time just typing in the URL and then converts. This string of traffic sources is difficult to track and analyze using the eVar allocation feature set available today. What I feel is needed is a way to simply have SiteCatalyst extend its Linear Allocation feature to include multiple visits and make that a legitimate setting in the Admin Console. I’d even pay more for it if needed, since not everyone will need that level of sophistication. I personally think that attribution will become a bigger issue in the future as the current browser model fractures so I think this will be an important feature for all web analytics vendors going forward. You can read some of my partner Eric Peterson’s thoughts on appropriate attribution in this white paper. If you’d like to see SiteCatalyst go deeper with attribution, please vote for this idea by clicking here.

Multi-Session Pathing
Along the same lines, the next idea I’d like to suggest is the notion of multi-session Pathing. I suggested this to the Ideas Exchange over a year ago and was surprised to see that it only has 7 votes! Currently, pathing reports are limited to one session. However, it is often the case that visitors come to your website multiple times before they convert. Wouldn’t you want to see paths that span multiple visits for the same person? I realize that this can be data intensive, but even if it is for a subset of data, I think it would be interesting to pick a subset of visitors and see what they do over multiple visits. Currently, you can’t even do this in Discover. While I am not sure of the exact way the feature should be implemented, I feel that having some insight into multi-session pathing is important and should be somewhere on the roadmap. If you agree, you can vote for this idea by clicking here.

Expire eVars Based Upon Event or Time
The last feature request I’ll mention has to do with expiring eVars. Currently, you can expire an eVar based upon a time period (like Visit or 30 Days) or a Success Event but not both. So why is this important? Imagine that you have a situation where you have an eVar set to expire at the Purchase event. A person could come to your website from a specific campaign code and then not return for an entire year and then convert. In that scenario, the campaign code they came from a year ago would get credit for the conversion. However, there are cases in which you would not want that to happen so it would be great if it were possible to have SiteCatalyst expire the eVar at the Purchase event or after 30 days – whichever comes first. That would offer much more flexibility and tighten up eVar attribution across the board. Someone also added a comment to this idea with the idea of allowing an eVar to expire at Success Event X or Success Event Y. That would also be helpful. If you’d like to see this implemented, please click here to vote for it.

Final Thoughts
As I mentioned at the beginning of this post, there are some features that could have a big impact if added to SiteCatalyst, but they are ones that only those who have been through some big battles would know are needed. My hope is that you will think about these features and support them with your votes so we can all benefit. Thanks!

If you have any questions or want to learn more, feel free to contact me for more information.

Analytics Strategy, General

Three Great Jobs at Best Buy

Now that summer is upon us I suspect that some of my personal blogging activity will slow down but I wanted to call my reader’s attention to three great jobs that our good friends at Best Buy just posted:

  • Senior Analyst, Digital Analytics
  • Associate Manager, Digital Analytics
  • Manager, Digital Analytics

Those of you who were at Emetrics in San Francisco this Spring heard some of the story about the work we’ve been fortunate to help with at Best Buy. Those of you coming to Internet Retailer in San Diego on June 16th will get to hear a shortened version of the same story. If you can’t/didn’t make either event I am happy to put interested parties directly in touch with the hiring manager at Best Buy, email me directly for details.

If you are coming to Internet Retailer, come and hear Lynn Lanphier (Best Buy) and I tell their amazing story.

Adobe Analytics

5 Social Media Secrets – M.Tech 2011

The folks over at Thoughtlead have put together what they’re calling a Digital Influence Collaborative. It’s innovative and exciting and a new way to consume content in microbursts. If you haven’t gotten wind of these events yet, you’re missing out.

They typically feature 60 influencers on 60 topics in 60 seconds. Topics vary from Social Media to Enterprise Marketing Management.

Here’s a mashed version of the one I delivered for Mtech 2011:

5 Secrets for LEARNING from Social Media

Analytics Strategy

Web Analytics (How It Works) Explained in 4 Minutes

I was tinkering around a few weeks ago trying to figure out the best way to communicate an idea out to a group of people and hit on using Snagit to record me talking my way through a few PowerPoint slides that had some basic diagrams on them and then uploading the resulting video to YouTube (in that case, as a private video). It worked great — perfectly okay audio quality (I used a USB headset) and perfectly okay graphics. Lo-fi, but using the tools I already had at hand.

Below is an audio slideshow that uses the same approach to provide a very basic overview of how page tag-based web analytics tools work. If you’re a web analyst, I sincerely hope there is nothing new to you here. But, if you’re a web analyst who has repeatedly beaten your head against a brick wall when trying to explain to some marketers you work with that they need to put campaign tracking parameters on the links they use…maybe it’s a video you can send their way! It’s right at 4 minutes long, with a subtle-but-shameless suck-up to my favorite Irish web analyst at the 1:30 mark (it really never hurts to suck up to an Irish(wo)man, now, does it?).

The video is a much simplified overview of what I went into in greater detail in an earlier blog post.

If you’d like to download the slides (.pptx) for your own use (attribution appreciated but not required, and edit at will), you can do so here.

I’d love to hear what you think (of the format and/or of the content)!

Analytics Strategy

Webtrends Table Limits — Simply Explained

A co-worker ran into a classic Webtrends speed bump a couple of weeks ago. A new area of a client’s web site had rolled out a few days earlier…and Webtrends wasn’t showing any data for it on the Pages report. More perplexingly, there was traffic to the new content group that had been created along with the launch showing up in the Content Groups report. What was going on? I happened to walk by, and, although I haven’t done heavy Webtrends work in a few years, the miracle of cranial synapses meant that the issue jumped out pretty quickly (I can’t figure out how to say that without sounding egotistical; oh, well — it is what it is).

Heavy Webtrends users will recognize this as a classic symptom of “table limits reached.” There’s quite a bit written on the subject online…if you know where to look. The best post I found was You need to read this post about Table Limits by Rocky of the Webtrends Outsiders. The last sentence (well, sentence fragment, really) in the post is, “End of rant.” In other words, the post starts AND finishes strong, and the content in between is damn good, too.

What I found, though, was that it took a couple of conversations and a couple of whiteboard rounds to really explain to my colleague what was going on under the hood that was causing the issue in a way that he could really understand. That’s not a knock against him. Rather, it’s one of those things that makes perfect sense…once it makes sense. It’s like reading an analog clock or riding a bicycle (or, presumably, riding a RipStik…I wouldn’t know!).  So, I decided I’d take a crack at laying out a simplistic example in semi-graphical form as a supplement to the post above.

The Webtrends Report-Table Paradigm

First, it’s important to understand that every report in Webtrends has two tables associated with it:

  • Report Table — this is the table of data that gets displayed when you view a report
  • Analysis Table — the analysis table is identical in structure to the report table, but it has more rows, and it’s where the data really gets stored as it comes into the system

Webtrends aggregates data, meaning that it doesn’t store raw visitor-level, click-by-click data and then try to mine through a massive data volume any time someone runs a simple report. Rather, it simply increments counters in the analysis tables. That makes sense from a performance perspective, but can easily lead to a “hit the limits” issue.

Key: neither of these tables simply expands (adds rows) as needed. Both have their maximum row count configured in the admin console. Those limits can be adjusted…but that comes at a storage and a processing load price.

(Now, actually, there are multiple analysis tables for any single report — copies of the underlying table structure populated with data for a specific day, week, or month…but it’s beyond the scope of this post to go into detail there. Just tuck it away as another wrinkle to learn.)

In the rest of this post, I’m going to walk through an overly simplistic scenario of a series of visits to a fictitious site with unrealistically low table limits to illustrate what happens.

The Scenario

Let’s say we have a web site with a series of pages that we’ll call Page A, Page B, Page C,…Page Z. And, let’s say we have our Report Table limit for the Pages report set to “4” (in practice, it’s probably more like 5,000) and our Analysis Table limit set to “8” (in practice, it would be more like 20,000). That gives us a couple of empty tables that look something like this:

Now, we’re going to walk through a series of visits to the site and look at what gets put into the tables.

Visit 1

The first visitor to our site visits three pages in the following order: Page A –> Page B –> Page C  –> <Exit>.

The analysis table gets its first three rows loaded up in the order that the pages were visited, and each page gets a Visits value of 1. If we looked at the Pages report at that point, the Report Table would pull those top 3 values, and everything would look fine:

Visit 2

The next visitor comes to the site and visits 5 pages in the following order: Page B –> Page C –> Page D –> Page E –> Page F –> <Exit>

We’ve now had more unique pages visited than can be displayed in the report (because the report table limit is set to 4). But, that’s okay. After two visits to the site, our Analysis Table would still have a row or two to spare, and the Report Table could pull the top 4 pages from the Analysis Table and do a quick sort to display correctly, using the All Others row to lump in everything that didn’t make the top 4:

If you searched or queried for “Page F” at this point, you wouldn’t see it. It’s there in the Analysis Table, but you’re searching/querying off of the Report Table. That doesn’t mean Page F is lost, though. It just means it has less traffic (or is tied for last) with the last item that fit in the Report Table.

Visit 3

Sequence of pages: Page F –> Page G –> Page H –> Page B –> <Exit>

Following the same steps above and incrementing the values in our Analysis Table, and again looking at a report for the entire period, we see (bolded numbers in the Analysis Table are the ones that got created or incremented with this visit):

Look! Page F is now showing up in the Report Table! Can you see why? Because the Analysis Table has greater row limits, the Report Table can adjust and pick the top-visited pages.

Visit 4

Sequence of pages: Page F –> Page I –> Page J –> Page B –> <Exit>

Here’s where we really start to lose page-level granularity. Our Analysis Table is full, so there are no rows to store Page I and Page J. So, that will add 2 visits to the All Others row in the Analysis Table (while this is a single visit, this is the pages report, and each of those pages received a visit). Our tables now look like this:

Until the Analysis Table gets reset, no pages after Page H will ever appear in a report.

Even if Page I Becomes the Most Popular Page on My Site?

It’s time for a direct quote from the Webtrends Outsider post referenced at the beginning of this post:

Ugly example #1: Your end users contact you wanting to know about traffic to their expensive new microsite.  You know you’ve been collecting the data correctly because you triple-checked the tagging before and after launch.  So you open the Pages report and WebTrends tells you those pages don’t exist.  Those  expensive pages got no traffic at all, apparently.  Knowing how the CEO’s been obsessed with the new microsite, you call in sick indefinitely.

It doesn’t matter if Page I becomes the only page on your site. Until the tables reset, you won’t see the page in your Pages report — it will continue to be lumped into All Others.

And That Is Why…

If you started out on Google Analytics and then switched over to Webtrends you might have noticed something odd about the URLs being captured (I learned it going in the opposite direction): in Google Analytics, the full URLs for each page, including any query string parameters (campaign tracking parameters excluded) are reported by default. In Webtrends, query string parameters are dropped by default. In the case of Google Analytics, you can configure a profile to drop specific parameters, while, in Webtrends, you can configure the tool to include specific parameters.

Why does Webtrends exclude all parameters by default? The table limits is one of the reasons. If, for instance, your site search functionality passes the terms searched for and other details to the search engine using query parameters, the Analysis Table for the Pages report would fill up very quickly…with long tail searches that only received 1 or a small handful of requests.

What to Do?

The most important thing to do is to keep an eye on your table sizes and see which ones are getting close to hitting their limits. If they’re getting close, then consider adjusting your configuration to reduce “fluff” values going in. If that’s not an issue, then you need to bump up your table limits. That may slow down the time it takes for Webtrends to process your profiles, but it will keep you from unpleasant conversations with the business users you support!

Analytics Strategy, Conferences/Community

Amazing news from Analysis Exchange

UPDATED: We got great quotes from the Vice President of Human Resources who hired Jan Alden Cornish that clarify how Analysis Exchange is making a difference when it comes to hiring web analysts.  See below!

If you’ve worked in web analytics and digital measurement for long, or if you’ve ever tried to hire an experienced web analyst, you know that there are not enough qualified, experienced, and well-trained web analysts in the world. What’s more, for the majority of our sector’s development there was literally nowhere someone new could go to get the kind of hands-on education and experience that most hiring managers are looking for. Considered together the web analytics industry has been stuck in a “lose/lose” situation.

The training gap was the central problem we set out to solve in 2009 when we launched the Analysis Exchange. Our goal was to bring “student learners” together with experienced mentors to provide guided education and work to ensure that entry-level analysts were familiar with both the theory and practice of web analytics. Analysis Exchange was designed as a logical “next step” for people who had read books, followed blogs, or taken online training from great groups like the WAA via their University of British Columbia coursework.

What’s more, so that our students would learn to “tell a story with data and analysis” we opted to work with nonprofits from around the globe — a traditionally under-served group when it came to site analysis and insight generation. This turned out to be a great idea, and we are honored every week by a handful of organizations who are willing to help us create valuable training opportunities for our community.

I set a lofty goal for Analysis Exchange when I first announced the effort was open to everyone at the Emetrics Summit in San Jose last May — I wanted to help 1,000 nonprofits and create training opportunities for 500 students. Unfortunately we didn’t meet that goal … but we have made amazing strides, a few of which I’d like to share with you today:

  1. We have grown to over 1,250 members around the world, including 205 nonprofit groups and nearly 650 students. Following the Web Analytics Association we believe Analysis Exchange to be the single largest group of individuals interested in the subject of web analytics in the world — and we’re pretty excited about that!
  2. Our members have completed over 100 projects in the past year. What’s more, our students and mentors have earned awesome scores with an average “likelihood to recommend this mentor/student” score of 9.5 and an average rating for each member’s work of 9.4 (both out of 10.0)
  3. We won a prestigious award from the Web Analytics Association. Analysis Exchange was recognized as the “Most Influential Agency or Vendor” by the WAA at this year’s awards event.
  4. IQ Workforce has just agreed to help us grow and expand our efforts. Given our commitment to incubating new talent within the web analytics community this sponsorship makes great sense (read more about it here) and we’re delighted to have Corry Prohens and his team helping our mentors and students expand their horizons.
  5. We recently had our first student get a full-time job working in web analytics. This more than anything excites me … the fact that Analysis Exchange is working “as designed” for the web analytics community, helping individuals get the experience they need to bridge the gap between “knowledgable” and “employed.”

On this last point I wanted to share a little more detail. We have some pretty motivated mentors and students in the Analysis Exchange. One of our students is Jan Alden Cornish from Carmel, California. Jan has done three projects with us and in one case stepped in and helped out at the very last minute. He’s bright, articulate, and one of the nicest guys you’ll ever meet … so when he called and asked me to provide a reference for him on a job interview I was more than happy to help.

According to Jan:

“Completing three projects with the Analytics Exchange afforded me a rare opportunity to work side by side with seasoned practitioners. Each project had it’s own unique set of challenges. Nothing can replace hands on experience with real data and a need to solve real problems. Digital marketing doesn’t take in an organizational vacuum. These projects also provided me an understanding of organizational context in web analytics takes place.”

We also heard from the Vice President of Human Resources who hired Jan, Cynthia Nelson Holmsky:

“As a major e-commerce website we were recruiting for an E-Commerce Analyst and found an alumni of Analysis Exchange.  While the candidate had many years of business and software analytics, his only web experience was through Analysis Exchange.  However that Exchange experience provided just enough applied web analytics to win him the interview.  During this recruitment I met other candidates with strong business analysis backgrounds who lacked any web experience, and I referred all of them to Analysis Exchange as a great place to learn web analytics and expand their career potential.”

Cynthia clearly understands the challenges facing recruiters and HR specialists looking for web analytics talent (emphasis mine):

“Web analytics is still a young discipline.  Many individuals and businesses want to develop competencies in web analytics, but wonder “Where do you go to develop expertise?”  Many colleges and universities have yet to integrate web analytics into their curricula, or what they cover is not hands-on, so Analysis Exchange is meeting a key need in the marketplace for individuals who want real world experience, while at the same time building supply to meet the demand for web analysis talent in the tech job market.  Plus, the Exchange is meeting the needs of non-profit organizations that normally could not tap into this type of expertise.  Analysis Exchange is a  great idea, and a win-win-win model.

Hopefully Jan will continue to support the Analysis Exchange — as a mentor, now that he is working professionally in the field. I also hope those of you reading this post will consider joining Jan in the Analysis Exchange. Signing up takes less than a minute and there are plenty of projects looking for mentors and students available right now.

Excel Tips

Excel Dropdowns Done Right: Data Validation and Named Ranges

NOTE: There is an updated version of this post posted here. I recommend reading that one rather than this one.

Every once in a very rare while, I find myself not motivated to expound upon deep and meaningful subjects. So, this post is not about the latest turn in world of privacy legislation, it’s not about my deepening fascination with two-tiered segmentation, it’s not about the perplexing and depressing indefinite postponement of Demystified Days, and it’s not even about pondering when Team Evil Forces will have a web site.

Nope. Not today. This is just a good ol’, “Hey, let’s look at a handy capability of Excel…and how to use it to the best of its ability.”

This came up last week when a co-worker asked me: “How do I get dropdowns working in cells in Excel?” She knew she had done it before, but she couldn’t remember how. In the course of showing her, I realized that, therein, was one of those handy little tips worth sharing. I’m going to walk through three different ways to accomplish this:

  • The totally common, mundane way — straightforward, but it has limitations
  • The way I always do it — almost no more effort to implement than the first way…but with fewer limitations
  • The way I may start doing it (sometimes), which would make the approach just that much slicker

Bounce around as you see fit!

The Scenario

You’re using Excel to enter a table of data, where one or more of the columns have a standard set of possible values. For instance, let’s say you’ve made a list of household chores, and you use that list to both assign a priority to each task as well as to note the status of the work:

For both the Priority and the Status column, you’d like to enter the values using a dropdown menu, rather than needing to retype a value in each cell:

The wrinkle is that you expect this list to live for a while, and there’s a good chance that you may want to have other values available for either the Priority or the Status columns (or both). We’ll get to that.

The Standard Excel Way — Data Validation

The quickest way to set this up is with basic data validation:

  1. Highlight all of the cells that will use the same dropdown values
  2. Select Data » Data Tools » Data Validation
  3. Change the Allow dropdown to List
  4. Enter the values in the Source box (separating different values using commas)
  5. Click OK
  6. Repeat for each set of cells that has a unique set of dropdown value options.

That’s all there is to it, and it works.

The Limitation: Suppose that you decided you wanted to add a new value to the list of options, and that, rather than four cells right next to each other, this same data validation rule was used across numerous non-contiguous cells, even cells across multiple worksheets. Going in and updating the available list of values is a real pain. That brings us to…

My Standard Way — Data Validation with a Named Range

I regularly use dropdowns to make Excel-based reports more dynamic — enabling the user to choose whether he wants to see a weekly or a monthly version of the report, as well as to select the specific date range (this isn’t so much for the user’s benefit as it is for mine — it means I don’t make a “new report” each week or month, but, rather, update the data in the same workbook and then update the dropdown to get the current report; read more about my approach for that in this post).

I have a standard way of generating dropdowns that gets around the limitation described earlier: rather than entering the list of values directly in the data validation dialog box, I reference a named range. Using the same household chores scenario, I would accomplish the same end result, sans limitation, as follows:

  1. Add a new worksheet (I usually name it something like “Lookups” and then hide the worksheet once everything is set up so it’s never something that the user sees)
  2. Enter the lists of values at the top of that sheet — one list per column
  3. Select all of the values for one set of dropdown options and enter a name for that range (in this case, “List_Priority”)
  4. Repeat this  for the other list of values (I named it “List_Status” — I like to prepend the names of similar types of named ranges so that they group easily in the Named Ranges dialog box)
  5. Now, it’s the same basic process as described earlier, except, rather than entering the specific values in the data validation Source field, you enter a named range (note the “=” before the named range!):
  6. Click OK, and you’re good to go again!

Now, if you ever want to update values in the list, you can edit the values on the Lookups sheet. This won’t update the cell values that have already been populated — just the available values in the dropdown anywhere that named range is used.

The Limitation: even this approach has a limitation, but it has a couple of workarounds. Let’s say you decide to add a value to one of your lists — say you want to add “Unknown” as an option for Priority. If you simply type it at the bottom of the list, it falls outside of the named range and won’t be reflected in your dropdowns. Two different ways to work around this:

  • After adding the value, edit the named range (Formulas » Defined Names » Name Manager) to include the additional cell
  • Before adding the value, select the bottom value in the current list, right-click, and select Insert » Shift cells down » OK.This will have effectively expanded the named range by a cell. You can then either add the new value in the blank cell or copy and paste the “bottom” value (“Low” in this case) into the blank cell and then enter the new value into the bottom cell

Both of these approaches are a little bit clunky, so let’s add a twist to make the named ranges a bit more elegant…

Data Validation with Named Ranges with a Clever Twist

[Update: See the first comment below — from Julien. As he notes, the formula described here is a little messy, and he proposes a cleaner solution. I’m leaving my original approach here to provide a “multiple ways to skin a cat” demonstration…but I expect I’ll be using the approach described in the comment.]

This is simply a couple of additional steps beyond the steps described in the previous section to make the named ranges a little smarter:

  1. Select Formulas » Defined Names » Name Manager
  2. Select List_Priority and click Edit to see the current definition
  3. Replace the Refers to: formula with the following formula:

=OFFSET(Lookups!$A$2,0,0,COUNTA(Lookups!$A:$A)-1)

And, voila! You can now go nuts with adding and removing values from the Priority list and the dropdowns will have updated values with no additional effort!

To do the same for the List_Status named range, the formula you would use for the named range would be:

=OFFSET(Lookups!$B$2,0,0,COUNTA(Lookups!$B:$B)-1)

To break down the OFFSET formula usage (using List_Priority as the example):

  • Lookups!$A$2: start at cell $A$2, which is the first value in the list
  • 0: stay in that same row (so still at $A$2)
  • 0: stay in that same column (so, again, still at $A$2)
  • COUNTA(Lookups$A:$A)-1: count the number of cells in column A that have values and then subtract 1 (the heading cell: “Priority”); grab an area that is that tall, starting with the cell currently “selected” ($A$2)

By checking Excel’s documentation on the OFFSET function and fiddling around a little bit with the formula, you can see how it’s working pretty easily.

Is It Worth the Effort?

I always use the second option described in this post. You just never know when a hastily hacked together spreadsheet will get “legs” and start growing and expanding its footprint. Better to spend an extra 10 seconds to add flexibility and maintainability.

Will I use the third option? I might. We’ll see. It didn’t occur to me that I should even try until I showed my co-worker the second option…and then watched her immediately get tripped up trying to add a new value to the list. If I’m handing off a document where flexibility in the dropdown values is needed, I might just Google my way back to this post to see how it’s done!

 

Conferences/Community

Demystified Days has been postponed

Unfortunately Analytics Demystified has been threatened with costly litigation over our Demystified Days event series. Out of respect for our current partners, our sponsors, and the entire community we have decided to postpone theses events for the time being.  We are certainly disappointed by this situation, but we remain committed to:

Now that Analysis Exchange has real momentum, our hope was to take this effort to the next level and begin to make real investments in the nonprofits that honor our efforts to train future web analysts through their participation. Our goal for this coming Fall was to donate $10,000 each to six different nonprofit participants in the Analysis Exchange in San Francisco, Atlanta, and Boston. Sadly we have been prevented from making those donations.

Such is life.

At Analytics Demystified we truly do believe in the community — be it the Analysis Exchange, the free web analytics documentation and content we all produce, John’s participation in the Web Analytics Association, my founding of the Web Analytics Forums in 2004, or Adam’s contribution to the award-winning Beyond Web Analytics podcast series. While we regroup and refocus our efforts expect to see us supporting Emetrics, the Web Analytics Association, Analysis Exchange, Web Analytics Wednesday, Beyond Web Analytics, and any other event or organization that is sincere in their investment in the web analytics community.

General, Social Media

The Crowd Has Spoken: Gilligan It Is

(I’ll return to serious posts shortly!)

A couple of weeks ago, I asked for input as to my new profile picture on this blog and elsewhere across the socialmediaverse. The crowd has spoken, a $74 donation has been made to the Appalachian Trail Conservancy, and it looks like I’m now due to have photographic alignment with the blog name:

Part of the inspiration for this exercise was that I’ve had the experience before of knowing what someone looks like as I’ve gotten to know them digitally solely based on a single picture…and then been surprised in some way by their appearance when I actually meet them in person. This came up a couple of times at eMetrics in San Francisco. So, in addition to changing my standard profile picture, I’ve also added a collage of photos to my About page. The challenge there is that I’m an amateur photographer, so am more often behind the camera than in front of it. That made for slim pickin’s on the photo front, but there’s enough there that you can get a better sense o’ me, should you care to have that!

Analytics Strategy

Would you pay $100 per year for Google Analytics?

Back in February our newest partner Adam Greco waxed philosophical about Google offering a paid version of Google Analytics. He got a bunch of feedback and all-in-all the post raised some interesting questions about Google’s place in the web analytics marketplace. Now there is a new rumor — one far less substantiated than the so called “Enterprise” offering Adam discussed — but one with potentially more far reaching implications.

On a call today, we heard that Google may be considering charging everyone for the use of Google Analytics.

Everyone? Yep. Everyone.

The details were sparse and wholly unsubstantiated, but come from a source that we generally trust as reliable. And while we normally don’t deal with rumors here at Analytics Demystified, given Google’s footprint — conservatively estimated to be around 30,000 business sites around the world with perhaps an order (or two) more non-business sites being tracked today — the implications of this rumor are interesting for two reasons:

  1. If Google were to charge a fee similar to other of their offerings, say $100 per year, to use Google Analytics, these fees may produce millions of dollars in annual revenue (and profits) for Google and their investors. Outside projections for Google Analytics installations range into the millions, which, given a reasonable retention rate (say, 10%) would produce substantial revenue. Given the changes Google is going through right now on their management team it’s hard to say how important “revenue” is, especially when the bandwidth and data storage costs for Google Analytics are likely to be significant given estimated volumes and at a time when Google is being criticized over their increasing operating costs. Given the constant criticism over the years of Google’s inability to generate profits outside of their advertising business perhaps this sort of obvious revenue is suddenly appealing.
  2. If Google were to start forcing folks to pay, this might be a huge boon to the emerging “secondary” market of web and digital analytics vendors including Woopra, Chartbeat, Performable, Clicky, Kissmetrics, and a rapidly expanding set of web and mobile analytics vendors who largely charge tens to hundreds of dollars per month. Despite the odds given the footprint “traditional” web analytics vendors have within the Enterprise, combined with the hegemony Google has over entry-level businesses and small companies, in the last three years we have seen a surprising “second coming” of web analytics vendors gaining traction in a variety of niches. Be it real-time analytics for blogs (Chartbeat, Woopra), funnel analysis (Kissmetrics), heat-mapping and session recording (Robot Replay, ClickTale, Reinvigorate), or customer-focused analytics (Performable), these companies are, by-and-large small, agile, and somehow managing to gain adoption despite the presence of Google and “the bigs”.

This second implication is very interesting to me … the fact that against well-established, already deployed, and in Google’s case completely free competition, these start ups are able to grow and, at least in a few cases, thrive (see Clicktale, Robot Replay who was acquired by Foresee Results, Performable, etc.) Imagine the glee that founders and investors in these companies would experience if Google Analytics were to put up a real (albeit potentially small) barrier to entry. It likely wouldn’t be enough to stop companies, but it might be enough to make them think “Hmm, I wonder what else is out there?”

Given that most of these start-ups are focusing on ease-of-use and specific use cases, and in many instances are doing a pretty darn good job (my opinion), this pause might be exactly what these start-ups need. Heck, it might even help some of the bigs, given the trouble they have had selling against Google Analytics juxtaposed against the dramatic interface changes that some are poised to unleash. Don’t get me wrong — I’m not saying that Google charging $100 for analytics magically re-opens the door for traditional vendors with annual contracts in the tens of thousands of dollars. But every substantial change in the marketplace is an opportunity for great management teams, and Google suddenly charging anything would surely be a substantial change.

But I digress.

At the end of the day I personally consider it highly unlikely that Google would start to charge everyone just because they can — it just seems like an unnecessary and “evil” thing to do (despite the fact that we did the same thing earlier this year at Twitalyzer without any negative impact on our business.) Still, Google is held to a pretty high standard, and I suspect that the (relatively) small amount of revenue they would ultimately generate is hardly worth the negative press they would likely receive.

But I’m interested in what you folks think. Would you pay $100 per year for Google Analytics as it exists today? What if they offered more features or functionality? If the latter, what would they need to add to get you to pony up? Or would you immediately pull the code off your site if Google required any kind of payment? If so, why?

I welcome your comments and conversation.

Social Media

Tweeting on Schedule

I’ve been playing around a bit with scheduling my Tweets and thought that I’d share some of my findings with you. But first, I’ll riff a bit on the fragility of this nascent channel and Twitter’s amazing rise to prominence as the 3rd largest social network in this universe. The figure I’m using for scale is 145 million registered users, which came straight from the Twitter CEO, Evan Williams back in November, 2010. But, it wouldn’t surprise me one bit if another 55 million users joined in the past 5 months. That’s the number that’s being bandied about today.

With ad revenues estimated at $45 million and projections escalating at a 3x clip this year, Twitter is rocketing unequivocally skyward. The only problem with attaining massive growth with user populations rivaling the number of people residing in Brazil, is that Tweets are extremely perishable. If you aren’t watching, listening or searching for a Tweet, it’s highly likely that it will slip right past an entire country of users without ever being noticed. That’s a problem. It’s bad because it seriously erodes any value proposition of time or dollars invested in the channel. Thus, the argument for scheduling Tweets.

Researching Tweets

The best research I’m reading about Twitter is coming from Sysmos, where they continue to crank out valuable insights. Back in September, 2010, they found that the average lifespan of a Tweet is about an hour. Sysomos discovered that 92.4% of Retweets happen within one hour after publication and 96.9% of @replies occur within the first hour. This means if your Tweet isn’t circulated after 60 minutes, it’s likely a goner. Of course there are numerous tools that allow you to automate this process. And that’s what I’ve been exploring. Even the most pedestrian Twitter clients now allow you to schedule your 140 character missives for posting at a later time.

What are the drawbacks of scheduling Tweets?

Scheduling Tweets is a tenuous business. For the most part, you should be Tweeting to deliver good content, but also to initiate a dialogue with your followers. If you’re out on the golf course and your Tweets are generating a firestorm of activity, who’s going to respond? Be cognizant of this fact when scheduling Tweets, because if your Tweet gains velocity and lots of people hear it, you better be at the ready to engage. If not, you’ll quickly lose credence as a friendly human and instead come off looking like a bit of a bot yourself. For this reason alone, if you’re planning to schedule Tweets, do so with considered caution and release news or informative Tweets purely to gain exposure. You don’t want to provoke a dialogue when you’re not ready to interact.

Who offers Tweet scheduling?

This isn’t meant to be a full and comprehensive review of Tweet scheduling tools. These are just a few that I’ve used personally, and my observations of each. I look forward to hearing what you think about Tweet scheduling and which tools if any you use. I’ll commit to updating my list as you offer more…

Tweetdeck – Ahh…my first real Twitter client and a darn good one at that. It’s iconic black interface offers de facto functionality and does so with a fine polish. (I’ve tried to use the “light” interface but just can’t make the switch). Tweetdeck is lightning fast with Tweets posted in real-time. But more to the point, they allow users to schedule Tweets in the future by simply selecting the date and time of your desired launch.

Hootsuite – This little freemium gem is quickly becoming my go-to Twitter client. Despite their recent service outage (which wasn’t really their fault), It’s winning me over with the multi-tabbed interface, multi-user efficiency and slick stream views. Hootsuite allows users to pre-schedule Tweets as well, with the option to select the date and time and receive an email when your 140 character missive flies.

Crowdbooster – I gained access to this product only recently and have been intrigued since my first login. This beauty not only allows you to schedule Tweets, but also recommends the best times to give a shout out. I really like that they deliver an explanation of why specific times are best for Tweeting based on when my followers are active and when I’ve gained the greatest reach. Crowdbooster also has the best charting I’ve seen yet from a Tweet scheduling interface that reveals which Tweets attained reach…and RT’s and @replies as well. I’m having fun with this freemium tool and may even upgrade.

Timely.is – Here’s an interesting new app, that I learned about recently. It uses an algorithm to Tweet when your message is likely to reach the largest audience. Currently, they don’t provide any visibility into how they make this determination, but you can override it by forcing the Tweet to send within the next 30minutes. While they do offer a few cheesy “suggested” tweets, this tool is a product of Flowtown and I’ve been waiting to see what these guys bring out of their private beta. This is definitely one to watch.

Buffer – Buffer offers a slick user interface allowing users to schedule Tweets across a number of recommended times. It has links to the Bit.ly API, but requires premium access to utilize this function. Yet, the free version delivers solid capabilities and collaboration functions for adding additional team members. Perhaps the easiest function is the Chome browser extension that enables you to schedule a Tweet directly from a webpage. This makes scheduling convenient and will be helpful in getting to word out on those juicy bits you discover during non-peak times.

LaterBro – Yo, bro…I haven’t actually tried this one yet, but its interface is simple and clean. I trust it works just fine for planning ahead.

Since drafting this blog post has taken beyond my optimal Tweeting window, I’m signing off now. But before I do, here’s a few more Tweet schedulers that I haven’t tried yet. I’m sure there’s a whole lot more too.

What do you use for scheduling Tweets and what do you like about it? Curious minds want to know.

Analytics Strategy, Social Media

Privacy: It's a 2.5-Dimensional Issue

I’m keeping the voting open for another week or so on my “choose a new profile picture” poll, so if you haven’t voted yet, please click over and do so. There’s a charitable donation (by me!) involved!

“Privacy” is a hot topic in the world of marketing analytics, driven primarily by shifting consumer (and, in turn, regulatory) sentiment on the subject. That shifting sentiment, I think, is largely being driven by the increasing integration of social media into our lives and our online behavior.

The WAA stepped up and put together a Code of Ethics a few months ago, and privacy is going to be a recurring topic at eMetrics and other conferences for the foreseeable future. Following the San Francisco eMetrics conference, Stéphane Hamel put together three scenarios and asked the #measure community to vote as to the ethics and allowability of each situation. He then revealed the results and added his own thoughts. Towards the end of that second post, Stéphane noted that he was disappointed by the lack of interest in the exercise, given the generally accepted importance of the topic.

Emer Kirrane responded in the comments:

It’s interesting that there seems to be a correlation between legality and ethics in the minds of your respondents. To me, the Code of Ethics is there as a flag against practices that are deemed unethical by the community, rather than deemed unethical by law.

Stéphane’s concern and Emer’s response have been bouncing around in my brain for several weeks. My conclusion: “ethics vs. legality” is going to continue to give us fits.

I realize this isn’t the first time that “ethics” and “the law” haven’t perfectly aligned (they almost never do, actually, even though that, from a purist point of view, is the goal), but bear with me — it’s worth using that lens to explore the issue and outline the challenges we’re going to have to deal with. These are two very different dimensions of the privacy debate, and one of them is in flux on several fronts.

Why 2.5 Dimensions?

Obviously, there is a legal/regulatory dimension, and there is an ethical dimension. But, really, the legal/regulatory dimension is heavily driven and influenced by consumer perceptions and fears. I actually wrote some thoughts on that a couple of years ago. With high-profile Facebook snafus and high-profile media outlets reporting on cookies and cross-site tracking, politicians have found an issue that their constituents care about (or can be prodded to care about). So, in a sense, the legal/regulatory dimension has some added “oomph” of consumer concerns behind it; I’m calling that “consumer perspective” another half a dimension.

It’s possible that “consumer perception” should be a third dimension in and of itself. But, oh boy, that would make for some hairy sketching in the remainder of this post. I’m pretty sure I’m not just punting, though — the will of the consumer when it comes to something like privacy does generally get manifested through some form of government regulation.

Start with the Basics

Two dimensions: legal and ethical. We can look at them like this:

Various practices raise privacy questions. In theory, we can plot each of them on this (conceptual) grid — there are more than shown here, but I’m just laying out the basic idea of the framework:

In Theory, We’d Have Harmonious Dimensions

If life was simple, we would have perfect clarity for each dimension, and perfect alignment between dimensions:

Notice the shaded quadrants at top left and bottom right — there would be no practices that were ethical but not legal, nor would there be any practices that were legal but unethical.

Alas! Privacy is Rife with Gray Areas!

Reality is more like this — gray areas rather than hard lines along both dimensions:

Ugh. Things get messy. There are more activities that are questionable — they may or may not be legal and/or they may or may not be ethical! Argh!

But Wait! There’s More!

Ever since the web went mainstream, it’s been a more global medium than anything that came before. And, we’ve all run into cases and concerns that our standard web analytics implementation runs afoul of the law in some country somewhere. This grid illustrates that wrinkle, too — the legal/regulatory gray areas live in different places depending on the country (only the U.S. and the E.U. are shown here — it’s an illustrative diagram, people! Not a comprehensive one!):

And the big blue arrow shows where pressure is being applied (back to that half-dimension of consumer fears mentioned at the beginning of this post). It’s a little counterintuitive that the arrow is pointing upward, isn’t it? How could it be that things are trending towards “allowed?” They’re not. Rather, the “interpretation zone” is moving upward — practices that used to be “clearly allowed” aren’t inherently changing what they are, but those practices are moving from “in the clear” towards the gray area.

Helpful?

This was definitely one of those situations where, when I initially had a rough picture in my mind that would represent these two dimensions, it was simple and clear. It was only as I put pen to paper to sketch it out that it turned out to be tricky. Shortly after I finished writing this post (but, obviously, before I published it…as I’m adding this comment at the end), Jason Thompson made a really good case as to what is (misguidedly) driving the legal dimension out of alignment with the ethical perspective. That reminded me that I keep meaning to go back and re-read the last chapter (chapter 9?) of Jim Sterne’s Social Media Metrics book, as I recall that it was an intriguing non-sequitur that considered turning the entire “tracking” model on its head. Food for thought for another post, that.

What do you think? Is this an effective representation of the shifting privacy landscape we’re dealing with? What does it miss?

Social Media

U-Slurping Influence

I’ve noticed something recently that appears to be a burgeoning trend, and I don’t like it. Startups dangling the promise of exclusivity and early admission to their private beta parties in exchange for wielding your influence to “Spread the Word”. Pssst…”The more friends you invite, the sooner you’ll get access!” C’mon! If your product is good, people are going to use it and talk about it. Don’t patronize me with your bad Charlie Sheen references and generic html. This is lazy social media marketing in my opinion. And its a tactic that I won’t pander to.

However, it’s not nearly as bad as hitting submit on a digital form only to realize that the teeny-tiny checkbox in the bottom left hand corner, yeah…the one you didn’t UN-check?? Well, they opted you right into Tweeting to your entire following that you just signed up for the latest whatever on Twitter. These sneaky little broadcast methods are cheap trix and I say you marketers should be ashamed of yourselves.

I’ll keep this rant short, but influence is and will be a contributing factor in the success of many social marketing activities. Yet, as with all things social, leveraging influence must be genuine. Blatant solicitation of influence is only adding to the derision of influencer metrics and the narcissists who work to game the system. The real value of your influencers will pay dividends when they choose to talk about your products and services unprovoked. Doing it otherwise is a surefire way to usurp the power of the influencers you’re trying to enlist.

Social Media

It's Time for a Change, and I Need Your Help

If you have any regular interaction with me on this blog, Facebook, Twitter, LinkedIn, or scads of other social media sites, then you’re used to seeing my visage as such:

It’s time for a change, and I’d like a little wisdom of the crowds to drive it. Two quick background notes.

Why the Jester Hat in the First Place?

I started using social media heavily when I started working at Bulldog Solutions. I didn’t have much in the way of digital/digitized pictures of myself. The one above was handy (my wife and our three kids made these hats for all of us for a Bulldog social event), and, before I knew it, I’d signed up for a half-dozen services and dropped the image in as my profile picture. Since then, I’ve met numerous people for the first time who have asked, “Where’s the jester hat?” (including being asked by Rudi Shumpert on the Live at eMetrics edition of the Beyond Web Analytics podcast). Who knew? I had apparently done a moderately successful job of personal branding!

Where’d ‘Gilligan’ Come From, Anyway?

In 1993, I hiked the Appalachian Trail from Georgia to Maine. It’s a tradition on the trail to adopt a “trail name” for the duration of the hike. Partly due to my lanky frame, partly due to the fact that I had a tendency to bang my head on the low beams in many of the shelters along the trail, and largely because I wore a Tilley hat, I was dubbed “Gilligan” one evening by several other hikers who were staying at the same shelter that night.

I started this blog on something of a lark, and I knew that “Tim Wilson” was entirely too common of a name to base the blog on, as I’ve written about before.

Cast Your Vote to Contribute to a Worthy Cause

I’ve come up with four options for my new standard profile picture, and I want you to help me choose the one I go with. As a moderate incentive, for every vote cast, my wife and I will contribute $1 to the Appalachian Trail Conservancy (up to $250). And, I’ll wear the chosen hat in public at the next major geek conference I attend (so spread the #measure word, people!).

A – The Hat That Started It All
It’s the same hat I wore for a 2,100-mile hike — and it still sees occasional use.
 
B – A New Jester Hat
Sticking with the jester theme, but with a new hat and a new picture
 
C – As Gilligan As I Can Be
 
D – If You’re Simply Opposed to Change

The voting is wrapped up! You can find the results in this post.

Thanks!

Analytics Strategy, Conferences/Community

WAA Elections: I Support the Slate

While the voting period is mostly over I wanted to drop a quick note and offer up some thoughts on the candidates and process for the current Web Analytics Association elections. This year is clearly different thanks to a new process, one that has the membership voting on both a “slate” of candidates and two “at large” positions. While initially I didn’t understand the need to change the process, upon further explanation and a little reflection, I believe the new process makes sense and has the best interests of the Association and it’s membership at heart.

Before you go and Tweet “he’s lost his mind …” hear me out.

As the Web Analytics Association has grown the few board positions have become less of an obligation and more of an opportunity for people. In that, in recent years, we have seen an almost staggering number of people nominated into the election process. This, in my opinion, has created a problem in that A) most of the candidates, despite qualification, are relatively unknown to the web analytics community and B) because of the relatively low number of voters, a “popular vote” has become relatively easily gamed. I have certainly thrown my weight behind individual candidates in the past and, because my blog has tens of thousands of readers worldwide (many of whom do vote in WAA elections), I believe I have been able to help folks get elected.

Yeah for us and our friends, but boo for the process in general.

The popular vote has led to some truly great people participating in the WAA — folks (and my bias here) like John Lovett, June Dershewitz, Matt Langie, Dennis Mortensen, Ed Wu, and Peter Sanborn. But the popular vote has also led to some less-than-stellar participants in my humble opinion — people who either quit the board mid-stream or who served more as obstructionists than participants.

This new process, with what I believe to be a pretty well vetted board “slate” and list of “at large” candidates, has tremendous potential to do one very important thing: allow the Association to maintain the momentum they have today. From where I sit, in the past year the Association has:

  • Hired a very qualified Executive Director in Mike Levin
  • Started a very successful “local” event in the Symposium
  • Launched a very important community initiative with the Code of Ethics
  • Held a wonderful recognition event in the Emetrics/WAA Gala

and more. Plus, while I am not privy to any greater level of detail than anyone else, my general sense is that the current board is more productive and more collegial than many (or any) past boards and that bodes well for all of us.

So when it comes to the current election cycle, the “slate” has three returning Board members in Peter Sanborn (currently the Board President), Ed Wu, and Alex Yoder plus two new members who are, in my opinion, tremendously qualified to serve in Jodi McDermott and Shari Cleary. I have faith in Peter, Ed, and Alex based on their past work, Jodi has been a passionate contributor to WAA Standards and a number of other initiatives, and Shari is one of the most intelligent, level-headed people I know in life, much less web analytics.

The “at large” positions do create some problems, to be sure. The proposed group was whittled down from a larger group of folks, several of whom were qualified, passionate, and motivated, but my understanding is that the “secret selection committee” (which I offered to help with but asked too late) made decisions based on demonstrated commitment, involvement, and a willingness to work within the processes the Association has already established for the benefit of the membership. This strategy ends up recognizing folks like Chris Berry, a huge supporter of Research and Standards, Eric Feinberg and Lee Isensee, the “Laurel and Hardy” of the WAA and critical members of the membership committee, and Bob Page and Joe Megibow, two individuals who represent the level of leadership in web analytics that many (if not all) of us aspire to. In short, a brilliant group.

This list leaves off some pretty nice people as well, and this I think is what is creating some of the recent consternation in Twitter, but from where I sit the opportunity is clear: Participate in the WAA at the level that Chris, Eric, and Lee have, or build the reputation that Bob and Joe have, and you’re a shoe-in for the “at large” slots in the future.

For the record I am voting for Joe Megibow and Bob Page for the “at large” positions. Both are brilliant, both are passionate about measurement, and both serve as an excellent example of the kind of work we should all be doing. The Association needs more practitioners to represent the real needs of our industry and I cannot  think of two better people to fill that role.

Anyway, for what it’s worth, I too was confused about the “slate” process and this election, but hopefully like me you are willing to give the process a chance.

I welcome your comments.

 

Analysis, Analytics Strategy, Reporting

In Defense of "Web Reporting"

Avinash’s last post attempted to describe The Difference Between Web Reporting and Web Analysis. While I have some quibbles with the core content of the post — the difference between reporting and analysis — I take real issue with the general tone that “reporting = non-value-add data puking.”

I’ve always felt that “web analytics” is a poor label for what most of us who spend a significant amount of our time with web behavioral data do day in and day out. I see three different types of information-providing:

  • Reporting — recurring delivery of the same set of metrics as a critical tool for performance monitoring and performance management
  • Analysis —  hypothesis-driven ad hoc assessment geared towards answering a business question or solving a business problem (testing and optimization falls into this bucket as well)
  • Analytics — the development and application of predictive models in the support of forecasting and planning

My dander gets raised when anyone claims or implies that our goal should be to spend all of our time and effort in only one of these areas.

Reporting <> (Necessarily) Data Puking

I’ll be the first person to decry reporting squirrel-age. I expect to go to my grave in a world where there is still all too much pulling and puking of reams of data. But (or, really, BUT, as this is a biggie), a wise and extremely good-looking man once wrote:

If you don’t have a useful performance measurement report, you have stacked the deck against yourself when it comes to delivering useful analyses.

It bears repeating, and it bears repeating that dashboards are one of the most effective means of reporting. Dashboards done well (and none of the web analytics vendors provide dashboards well enough to use their tools as the dashboarding tool) meet a handful of dos and don’ts:

  • They DO provide an at-a-glance view of the status and trending of key indicators of performance (the so-called “Oh, shit!” metrics)
  • They DO provide that information in the context of overarching business objectives
  • They DO provide some minimal level of contextual data/information as warranted
  • They DON’T exceed a single page (single eyescan) of information
  • They DON’T require the person looking at them to “think” in order to interpret them (no mental math required, no difficult assessment of the areas of circles)
  • They DON’T try to provide “insight” with every updated instance of the dashboard

The last item in this list uses the “i” word (“insight”) and can launch a heated debate. But, it’s true: if you’re looking for your daily, weekly, monthly, or real-time-on-demand dashboard to deliver deep and meaningful insights every time someone looks at it, then either:

  • You’re not clear on the purpose of a dashboard, OR
  • You count, “everything is working as expected” to be a deep insight

Below is a perfectly fine (I’ll pick one nit after the picture) dashboard example. It’s for a microsite whose primary purpose is to drive registrations to an annual user conference for a major manufacturer. It is produced weekly, and it is produced in Excel, using data from Sitecatalyst, Twitalyzer, and Facebook. Is this a case of, as Avinash put it, us being paid “an extra $15 an hour to dump the data into Excel and add a color to the table header?” Well, maybe. But, by using a clunky Sitecatalyst dashboard and a quick glance at Twitalyzer and Facebook, the weekly effort to compile this is: 15 minutes. Is it worth $3.75 per week to get this? The client has said, “Absolutely!”

I said I would pick one nit, and I will. The example above does not do a good job of really calling out the key performance indicators (KPIs). It does, however, focus on the information that matters — how much traffic is coming to the site, how many registrations for the event are occurring, and what the fallout looks like in the registration process. Okay…one more nit — there is no segmentation of the traffic going on here. I’ll accept a slap on the wrist from Avinash or Gary Angel for that — at a minimum, segmenting by new vs. returning visitors would make sense, but that data wasn’t available from the tools and implementation at hand.

An Aside About On-Dashboard Text

I find myself engaged in regular debates as to whether our dashboards should include descriptive text. The “for” argument goes much like Avinash’s implication that “no text” = “limited value.” The main beef I have with any sort of standardized report or dashboard including a text block is that, when baked into a design, it assumes that there is the same basic word count of content to say each time the report is delivered. That isn’t my experience. In some cases, there may be quite a bit of key callouts for a given report…and the text area isn’t large enough to fit it all in. In other cases, in a performance monitoring context, there might not be much to say at all, other than, “All systems are functioning fine.” Invariably, when the latter occurs, in an attempt to fill the space, the analyst is forced to simply describe the information already effectively presented graphically. This doesn’t add value.

If a text-based description is warranted, it can be included as companion material. <forinstance> “Below is this week’s dashboard. If you take a look at it, you will, as I did, say, ‘Oh, shit! we have a problem!’ I am looking into the [apparent calamitous drop] in [KPI] and will provide an update within the next few hours. If you have any hypotheses as to what might be the root cause of [apparent calamitous drop], please let me know” </forinstance> This does two things:

  1. Enables the report to be delivered on a consistent schedule
  2. Engages the recipients in any potential trouble spots the (well-formed) dashboard highlights, and leverages their expertise in understanding the root cause

Which…gets us to…

Analysis

Analysis, by [my] definition, cannot be something that is scheduled/recurring/repeating. Analysis is hypothesis-driven:

  • The dashboard showed an unexpected change in KPIs. “Oh, shit!” occurred, and some root cause work is in order
  • A business question is asked: “How can we drive more Y?” Hypotheses ensue

If you are repeating the same analysis…you’re doing something wrong. By its very nature, analysis is ad hoc and varied from one analysis to another.

When it comes to the delivery of analysis results, the medium and format can vary. But, I try to stick with two key concepts — both of which are violated multiple times over in every example included in Avinash’s post:

  • The principles of effective data visualization (maximize the data-pixel ratio, minimize the use of a rainbow palette, use the best visualization to support the information you’re trying to convey, ensure “the point” really pops, avoid pie charts at all costs, …) still need to be applied
  • Guy Kawasaki’s 10-20-30 rule is widely referenced for a reason — violate it if needed, but do so with extreme bias (aka, slideuments are evil)

While I am extremely wordy on this blog, and my emails sometimes tend in a similar direction, my analyses are not. When it comes to presenting analyses, analysts are well-served to learn from the likes of Garr Reynolds and Nancy Duarte when it comes to how to communicate effectively. It’s sooooo easy to get caught up in our own brilliant writing that we believe that every word we write is being consumed with equal care (you’re on your third reading of this brilliant blog post, are you not? No doubt trying to figure which paragraph most deserves to be immortalized as a tattoo on your forearm, right? You’re not? What?!!!). “Dumb it down” sounds like an insult to the audience, and it’s not. Whittle, hone, remove, repeat. We’re not talking hours and hours of iterations. We’re talking about simplifying the message and breaking it up into bite-sized, consumable, repeatable (to others)  chunks of actionable information.

Analysis Isn’t Reporting

Analysis and reporting are unquestionably two very differing things, but I don’t know that I agree with assertions that analysis requires an entirely different skillset from reporting. Meaningful reporting requires a different mindset and skillset from data puking, for sure. And, reporting and analysis are two different things, but you can’t be successful with the latter without being successful with the former.

Effective reporting requires a laser focus on business needs and business context, and the ability to crisply and effectively determine how to measure and monitor progress towards business objectives. In and of itself, that requires some creativity — there are seldom available metrics that are perfectly and directly aligned with a business objective.

Effective analysis requires creativity as well — developing reasonable hypotheses and approaches for testing them.

Both reporting and analysis require business knowledge, a clear understanding of the objectives for the site/project/campaign/initiative, a better-than-solid understanding of the underlying data being used (and its myriad caveats), and effective presentation of information. These skills make up the core of a good analyst…who will do some reporting and some analysis.

What About Analytics?

I’m a fan of analytics…but see it as pretty far along the data maturity continuum. It’s easy to poo-poo reporting by pointing out that it is “all about looking backwards” or “looking at where you’ve been.” But, hey, those who don’t learn from the past are condemned to repeat it, no? And, “How did that work?” or “How is that working?” are totally normal, human, helpful questions. For instance, say we did a project for a client that, when it came to the results of the campaign from the client’s perspective, was a fantastic success! But, when it came to what it cost us to deliver the campaign, the results were abysmal. Without an appropriate look backwards, we very well might do another project the same way — good for the client, perhaps, but not for us.

In general, I avoid using the term “analytics” in my day-to-day communication. The reason is pretty simple — it’s not something I do in my daily job, and I don’t want to put on airs by applying a fancy word to good, solid reporting and analysis. At a WAW once, I actually heard someone say that they did predictive modeling. When pressed (not by me), it turned out that, to this person, that meant, “putting a trendline on historical data.” That’s not exactly congruent with my use of the term analytics.

Your Thoughts?

Is this a fair breakdown of the work? I scanned through the comments on Avinash’s post as of this writing, and I’m feeling as though I am a bit more contrarian than I would have expected.

Social Media

One Digital Analyst’s Guide to Using Twitter

Did you come to this post via a #measure link on Twitter? If so, then fair warning: your arrival probably has more opportunity to benefit me than it does to benefit you – I’d love to get some tips in the comments section that help me evolve my own process!

Guy Kawasaki spoke at the San Francisco eMetrics this year, and one of his early statements that most people in the room seemed to agree with was, “If the first time you saw or used Twitter, you didn’t think, ‘This is the dumbest thing I’ve ever seen,’ you probably aren’t that bright.”

Anyone who actively uses Twitter has struggled to articulate how and why it brings value to their lives when discussing it with a non-user. Consistently, those of us who are active users wind up falling back on, “You really have to get in and try it out and stick with it for a couple of weeks before it will start making sense.”

This post is my attempt to provide a guide/process specifically for analysts who fall into that “skeptical non-user” camp to try to make that “try it out for a couple of weeks” as smooth and worthwhile as possible.

A second Guy Kawasaki reference: he wrote a post once where he articulated another absolute truism:

There is no right and wrong with Twitter. There’s only what works for you and what doesn’t, so telling people how to use Twitter is as laughable as telling people what kind of websites were acceptable in 1980.

I’m walking a fine line with this post then, am I not? What I’m laying out here has two critical caveats:

  • It’s how I use Twitter – what I use it for and the tools I employ to use it effectively
  • It’s how I use Twitter as of April 2011 – as the medium continues to evolve and shift, and as I pick up tips and tools from others (I’m hoping to get some such tips from comments to this post), my process evolves

I (like many others) disagree with a lot that Kawasaki has to say about Twitter, but I agree that there is no single “right” way to use it, and this post shouldn’t be taken as such. It’s how I use it — if you find a thought or two that you think would be useful, use it. If you find a thought or two that you think is inane, then don’t (but I’d like a comment on this post so I can evolve my own approach).

Let’s dive in, shall we?

What I Get Out of Twitter

As an analyst, I get a range of benefits from my use of Twitter. Trying to organize them into a list makes it seem like they fall into discrete buckets, when, in reality, they’re a bunch of fuzzy overlapping circles, but here goes, anyway:

  • Breaking news in the industry – product launches, acquisitions, hot topics
  • Useful thinking from members of the industry – blog posts with tips/tools/explanations/philosophies
  • Relationship building – tweets can lead to emails, phone calls, and in-person meetings with both recognized industry leaders as well as analysts grappling with similar issues to me, and two minds are better than one almost all the time! I’ve also had relationships with vendors seeded through Twitter – several that have led to very real and very valuable partnerships
  • Technical support…from the community – I regularly tap into the Twitterverse to confirm oddities (the disappearance of Google Analytics Benchmarking, the fact that GA shows Safari as the top mobile browser for Android devices, etc.), which is often a quick way to confirm that I’m not missing something obvious
  • Technical support…from vendors – many vendors have a formal customer service presence (username) on Twitter or monitor references to their products and will respond promptly. I’ve gotten quick and helpful responses with very targeted queries to @OmnitureCare, @OmnitureFC, @Twitalyzer, and others

Even without all of these benefits, I would still value Twitter as a useful tool in my analyst workbelt. So, let’s get onto the actual process I use with Twitter.

But First! A Critical Understanding

If you’re new to Twitter, there is one key, key thing you absolutely must understand:

98%* of the content on Twitter that you could see and that might be of interest to you…you will miss…and that’s okay.

It’s easy to get sucked into your Twitter streams and find one useful link or reference after another. You then jump into some other work for a few hours (or days), and a little voice in the back of your head starts saying, “You’re missing valuable content!”

You are…and you aren’t. There is simply too much information out there to consume it all. Don’t try. Think about the information you get out of Twitter in terms of incremental bonus information on top of your existing resources rather than an entire body of information that you should be consuming as much of as possible, and you will be much more able to go to bed at night and rest peacefully. As it happens, my own Twitter usage has been pretty light for the past couple of weeks due to travel. It’s not keeping me up at night!

NOW…to the Brass Tacks of Using Twitter

There are three aspects (plus an optional bonus) to using Twitter:

  • Twitter tools (clients)
  • Filtering and categorizing content
  • Contributing and engaging
  • (Optional) Measurement and analysis

Twitter Tools

I use Hootsuite. It’s web-based, has all the functionality I need (the key one being the ability to show multiple “streams” of content at once), and has a mobile app that works fairly well. And, it’s got a nice bookmarklet (“hootlet”) that is a persistent button in my preferred browser, Google Chrome, so I can quickly tweet any link I find.

My setup at work and at home is to use an external monitor with my laptop. I use the monitor as my primary workspace, and my laptop off to the side with a browser maximized with Hootsuite running in it at all times. That way, I engage with the various streams I set up (discussed below), simply by glancing off to the side – where my laptop sits.

There are other clients, for sure. You can look at the tweets of people you follow (or ones you don’t follow) to see what clients they use.

‘nuf said. I’d be surprised if I was still using Hootsuite a year from now, but maybe not terribly surprised.

Filtering and Categorizing Content

One of the great things about the evolution of Twitter is that there is little harm in having a high count of people you are following. You may occasionally scan the timeline that intermingles all of their tweets, but, in practice, that’s going to be an unmanageable sea of information. I have the following “streams” that I set up to drastically filter and organize the content:

  • Replies – I have a stream for people who reference me in a tweet; this is one of the more important ones, because anyone who says something to me or about me can reasonably expect a timely response or acknowledgement
  • #measure hashtag – this is a search of “#measure,” basically, and it’s the widely adopted convention that web analysts (or, really, digital marketing analysts) use for tweets relevant to the field
  • Other searches – during eMetrics, I also followed the #emetrics hashtag; if I were working exclusively with a single web analytics platform, I might follow a search for that tool’s name or the hashtag…but I don’t do that currently (and there are a finite number of streams that I can reasonably follow at once)
  • Lists – I have several lists of people I follow; most notably, a “web analytics” list that I add people to when I see them tweet something of interest in #measure, a “Resource Interactive” list that contains current (and former) co-workers of the agency where I work, and even a private “client” list where I add employees of clients with whom I’ve had some interaction

I have a stream for Twitter direct messages…but I seldom look at it. I have DMs set up to send me a text message so that I’ll be more likely to get them even if I am away from my computer.

Contributing and Engaging

Some people use Twitter solely as a one-way communication vehicle – so-called “lurkers.” They use a combination of the techniques above, but they seldom actually tweet anything themselves. This really cuts down on the overall value that can be gained from the medium.

My personal strategy for contributing/engaging goes something like this:

  • Using the various streams described above, I retweet information that I genuinely find valuable and try to include a few words as to what struck me about the tweet or link it referenced
  • I get emails with interesting content – often people send me a note because they found content that they thought might be of interest to me, and, when I check it out, I feel like it’s actually content that might be of interest to the larger #measure community, so I tweet it (and credit the person who emailed it to me, if they’re on Twitter and I know their username).
  • I reply to people when I’ve got something to contribute – either a humorous response that might make them chuckle, a helpful link that I remember/can track down, or actual information that answers a question they’re asking or furthers a conversation
  • I have a list of “Measurement and Analytics” bloggers that I’ve built a feed for in Google Reader. This is my own version of something Stéphane Hamel (@immeria) set up years ago, and I used to use Yahoo! Pipes for, but which I recently cut over to Google Reader. I regularly add additional blogs to this feed, and I go beyond the pure “measurement” blogs that I find – pulling in both some data visualization and presentation tips blogs as well. This is an information resource for me in its own right, but, I start my day by scanning the new entries in that feed, and, if I see anything that might be of interest to the #measure community or to a specific person…I tweet it.

None of these are time-consuming activities for me. They’re either 5-10 seconds tacked on to whatever I’m already doing (to share the content), or they’re micro-interruptions throughout the day when I need to glance away from whatever work is currently at hand for a quick mental break.

(Optional) Measurement and Analysis

I wonder if it might be a bit controversial to say that measurement is optional. But, I don’t measure my e-mail use or my phone use, and, in many respects, all Twitter is is another channel along those lines.

Having said that, Twitter is also a key tool I use to build and evolve my personal brand. I use it to promote new blog posts I’ve written (this one, for instance, was auto-tweeted when it was published), as well as, I hope, to elevate awareness of who I am and what types of expertise I haven, and even some degree of my personality.

So, for me, it is important for me to measure whether my contributions and use of the platform are having a positive impact on @tgwilson as a Twitter presence. I use Twitalyzer for this…and you can read about how (in near-excruciating detail) in a post from a few months back.

Some Closing Thoughts

In the end, I try to hook into a few different dimensions of my social graph and engage with each of those dimensions. At times – fairly often, actually – I find content from one dimension of my social graph (say, my non-analyst co-workers) and port it over to share with the #measure community.

Both Twitter and various Twitter tools will continue to evolve. In a medium that is inherently micro and choppy, relatively small nuisances (being limited to 4 streams showing concurrently on my laptop screen, for instance) can really start to grate on my nerves over time. But, time and again, both Twitter and Twitter tool vendors continue to innovate and improve the user experience.

Looking back over the past 3 years, I realize that I’m now consuming and engaging much more, garnering more value, and spending the same or less actual time in the medium than I did when I started out. That’s partly from improvements in the tools, partly from the organic evolution of my personal process, and partly from…practice.

What Do You Think?

Chime in! What else do you do to efficiently leverage Twitter as an analyst? What frustrates you or is holding you back?

 

*Completely unsubstantiated, mostly defensible estimate.

Conferences/Community, General, Social Media

Announcing "Demystified Days"

UPDATE MAY 6, 2011: Under threat of litigation we have decided to postpone Demystified Days for the time being. You can read more about this decision here.

I am incredibly excited to let all of you know about something that Adam, John, and our friends at Keystone Solutions will be doing this coming September that builds on our long-standing commitment to local web analytics communities and our more recent efforts to support nonprofits around the world … something we are calling “Demystified Days!”

Check out the mini-site for Demystified Days right now!

For years we have been helping local web analytics communities around the globe connect with each other as part of Web Analytics Wednesday, and by every measure, Web Analytics Wednesday works. Thanks to current and past sponsors — great companies like I.Q. Workforce, Coremetrics (an IBM Company), SiteSpect, and hundreds of other companies who have hosted regional events — Analytics Demystified has brokered more personal introductions (and served more beers) than any other organization or group in our industry.

This past year we have been trying to leverage our connections in the industry to do something truly good and solve bigger problems. The result was, of course, the Analysis Exchange — the world’s only effort to provide free analytics support to nonprofits and nongovernmental organizations — which thanks to the efforts of great people like Wendy Greco, Emer Kirrane, Jason Thompson and our mentors and students has changed how people learn how to tells stories with data.

Now we are taking it to the next level, one city at a time.

Starting September 12th in San Francisco we will be bringing a day long educational and networking event to cities across the globe.  The format will be one you are all familiar with — great presentations in the morning and great conversations in the afternoon, of course followed by drinks and networking at Web Analytics Wednesdays in the evening.

We could easily do these events for free … but we aren’t going to. Instead we are going to find awesome sponsors to help us offset costs and ask everyone who participates to buy a $99 ticket to the event. Then, at the end of the day, we are going to add up all of the revenues, subtract out all of the costs, and donate every penny that is left to two local charities decided on by the event participants.

Our hope is to be able to donate a total of $50,000 to six charities in the United States. You can help us achieve that goal by doing three very easy things:

  1. Helping us spread the word about Demystified Days within your social network. We have created a short URL http://bit.ly/demystifieddays and you can tag tweets about these events with #demystifieddays.
  2. Joining us in San Francisco, Atlanta, and Boston. We are finalizing venues right now and will post ticket purchasing information in the next few weeks so watch for that!
  3. Email us and let us know you are interested in Demystified Days. The mini-site has a form at the bottom that will let you indicate your interest. Fill out the form and we will keep you in the loop!

On behalf of the teams at Analytics Demystified and Keystone Solutions we sincerely hope you are excited about what Demystified Days can become. We welcome your questions in comments or directly via email.

Help spread the word!

 

Adobe Analytics

Time Zone Trick [SiteCatalyst]

EDITOR’S NOTE:
Since joining Analytics Demystified, the most common email/comment I have received goes something like this:

“When are you going to get back to blogging about cool, advanced stuff you can do in SiteCatalyst?”

While in my new role, I am vendor-agnostic, I will do my best to keep sharing the SiteCatalyst tips & tricks I used to on my old blog. My hope is that as I work with clients using all web analytics vendors, I will branch out and share tips & tricks for all technologies. However, as I always tell people, the goal of my blog posts are to introduce concepts that can be applied to all web analytics tools…

Now on with a new tip/trick…

Dealing With Time of Day (Time Parting)

One of the analyses that I have done from time to time is Time Parting Analysis. Time Parting Analysis consists of looking at the time of the day (or day of week) that website success takes place in order to better understand its importance. While I don’t usually put a whole lot of stock into the time of day, there can be times where websites do much better/worse in the morning vs. evening. Knowing this can be used when planning advertising so you can “strike while the iron is hot,” so to speak.

If you think Time Parting might be important to your business, you should capture the time of day in some manner into variables in your web analytics tool. For example, if you use Omniture SiteCatalyst, you might use the Time Parting Plug-in to pass the time of day (in half-hour increments) to an eVar or sProp. Doing this allows you to look at a report that might resemble the one shown here:

As you can see, this report allows us to see what the action is taking place on our website down to the half-hour increment. If you are not already doing this type of analysis, it may be worthwhile since you can glean some new insights and use this data point for visitor segmentation.

Time Zone Hell!

However, inevitably you will run into a few problems with the above report. First, someone at your organization will ask you which time zone the above report is related to. Therefore, the first thing I recommend is that you clearly label your Time Parting reports with the time zone that the JavaScript file is using to capture the data. In the example above, the “Hour of Day” report was labeled do be PST (Pacific Standard Time) so it can be easily interpreted by everyone using it.

The next problem you will encounter is that of multiple time zones. If you work at a global organization and have people focusing on business in various locales, the above report is pretty much useless to many of your internal customers. If they happen to be good at math and can calculate time zone differences in their head, then you’ll be ok, but most people have trouble interpreting web analytics reports without the added labor of doing on-the-fly time zone translation!

Want to see this problem in action? Take a closer look at the report above. Do you notice anything strange? If you look closely, most of the Visits and Form activity took place in the evening. People might like your product(s), but not so much that they are willing to spend their evenings looking at them! The reason the above report looks strange is because it is for an Australian website, but the time zone is Pacific Standard Time. If you are a web analyst in Australia, seeing your website success events in the Pacific Time Zone is not super-helpful!

So how do we fix this? All it takes is a bit of creativity and meta-data. Keeping in mind that there is a direct relationship between time zones, you can take the above report and apply meta-data to it to adjust for alternative time zones. If you are using Omniture SiteCatalyst as in the example above, this means using SAINT Classifications. By applying a different SAINT Classification for each time zone you care about, you can create new reports for each time zone. Here is an example of what the SAINT file might look like for a few additional cities:

As you can see here, we took the data that was already being collected (the Key column, which in this case is PST) and added meta-data for four additional cities. You can add as many cities as you want and each column you add will create a new report for that city time zone. Once you have done this, you can see a new version of the report above adjusted for each time zone. Now if we look at the same report above, but use the Sydney Time Zone classification report, we see a report like this:

You will notice that now we are seeing the same exact data as the first report, but now the times of the website successes are adjusted for the Sydney time zone. This makes the report look a bit more normal for the Australian web analyst as the success events are now shown as taking place during more realistic business hours. The best part of this solution is that anyone using the standard Time Parting plug-in Omniture provides can use the same SAINT Classification file. It just needs to be adjusted so the “Key” column is the time zone for which you are collected the data. If you are using the PST time zone, you can download the file I showed above. If you are in a different time zone, you can still download the file and adjust it as necessary.

Caveats

As always, there are a few caveats with any “hack,” so here are mine:

  • I take no responsibility for daylight savings time which can wreak havoc on time zone translations, but even in that worst case, your data will be an hour off…
  • Time Parting reports can also be used to track Day of Week. This is harder to adjust for than is time zone unless you are time stamping using the actual date and are willing to have a massive, multi-year SAINT Classification file. This is not a bad approach, but is much more involved. Contact me if you’d like to explore this.
  • It is possible to collect time zone data using different time zones for each report suite. For example, it may be better for you in the long run to have your Sydney data collected in the Sydney time zone and your London data in the London time zone, but I have often seen clients have issues with this and if you don’t start doing this from the onset, you can have issues going to it later. Please consult your account manager for more details.

Final Thoughts

So there you have it, a few thoughts on Time Parting and a fun trick to make it more useful if you do business in multiple time zones. Give it a whirl and let me know what you think…

If you have any questions or want to learn more, feel free to contact me for more information.

Analytics Strategy, Social Media

eMetrics San Francisco 2011 — Recap by the Tweets

Note: There’s a lot of gushing I could do about how great it was to meet a lot of people in person whom I’d only known via Twitter prior, to see people I’ve met before, and to meet new people…but I’ll save some of that for a later post. This is the “content recap” post.

The last 3-4 conferences I’ve gone to, I’ve used Twitter in lieu of a notepad for my note-taking. What I realized after the first time I tried this was that it forced me to be succinct and to be selective as to what I noted. Now, for better or worse, my thumbs have gotten a bit more nimble at the same time that the input mechanisms on my mobile devices have improved. So, some might say I’m not as selective as I should be!

But, after eMetrics in D.C. last fall, I realized that there’s another benefit of tweet-based note-taking at conferences — it enables crowdsourcing the key takeaways. In theory, at least! Given that, I decided to organize this recap based on one thing: the top most retweeted tweets during the conference, as reported by a TweetReach tracker I set up. Scroll to the end of this post to download a CSV with the raw data, if you’re interested in crunching it yourself.

With that, here are my six summary takeaways:

The Data Isn’t Actionable without Storytelling

Hands, down, the most retweeted tweet (14 times) from the conference was this from Wendy Ertter:

Several presenters touched on the fact that one of the key challenges in our industry is communicating what the data means. As analysts, it can be easy to get absorbed in the data to the point that we intuitively can interpret our analyses. All too often, though, we forget that the business users we’re supporting are neither “wired for data” nor have they been as immersed in it as we have. So, rather than getting stamp-your-foot irritated that your brilliant insights have not led to action, take a look at how those insights are being communicated.

Now, not discussed was the fact that “tell a story with the data” can easily come across as “torture the data until it tells the story you want it to.” It’s a fine line, really, that means transparency has to come along with the storytelling. Storytelling must be merely a means of “effectively communicating the truth” — conveying what the data really is saying, but in a digestible manner.

Social Media, Social Media, Social Media

Ken Burbary’s tweet during Guy Kawasaki’s closing keynote (which garnered quite a bit of ire from the attendees, but that’s potential fodder for a future post) was retweeted 11 times:

Social media was a hot topic at the conference, with the sessions devoted to it concluding: “It’s tough to analyze.” In general, there was consensus that Performance Measurement 101 still applies — if you want to have any hope of measuring social media, you darn tootin’ better have clear objectives for your investment in the channel. Now, because social media isn’t the same as longer standing channels, there are different measures to work with.

One of the more intriguing sessions I attended was a panel, moderated by Michele Hinojosa, that featured Gary Angel of Semphonic and Michael Healy. The subject was sentiment analysis. Specifically, sentiment analysis of short-form text messages — Twitter and the like. Both by illustrating examples and talking through some of the advanced machine learning algorithms that have been applied to the challenge, they made a pretty strong case that trying to discretely quantify sentiment in a Twitter world is a fool’s errand.

Gary also made a distinction between “monitoring” and “measurement” and, later in the discussion, postulated that social media may be one case where you actually need to do analysis first and then set up your measurement. This makes sense, even in light of my “Performance Measurement 101” comment above. It does make sense to sift around in the conversation that is going on around a topic or a brand a bit to get a human and qualitative sense of the lay of the land before determining exactly what to measure and how.

[Update: I just realized that Gary wrote up a pretty detailed post about his key points in the session over on his blog last week — it’s worth a read.]

Attitude Is As Important As Behavior

This tweet from @SocialMedia2Day during Larry Freed’s opening day keynote was retweeted 10 times:

Foresee Results was the Diamond Sponsor for eMetrics, and the company continues to push the web analytics industry to recognize attitudinal data as being every bit as important as behavioral data. Interestingly, VOC vendors overall had a much more prominent presence than web analytics vendors (only Google Analytics and Yahoo! Web Analytics were exhibitors at the event — Webtrends, Adobe/Omniture, and Coremetrics were nowhere to be seen in the exhibit hall).

I have to credit Chris Dooley from Foresee Results for initially introducing me to (read: pestering me about) the rightful place of attitudinal data as a companion to behavioral data. He was right when he started preaching it, and he’s still right today. Another VOC vendor noted during his presentation that, when his company surveyed the top 500 retail sites and the top 500 overall trafficked sites, they found that only 15% were running on-site surveys. That is both surprising and alarming! OpinionLab also impressed a number of people with their presentations in the exhibit hall theater, and iPerceptions provided a bit more detail about their coming 4Q Premium product (which, seeing as how they announced it was coming back in October, is somewhat underwhelming given the price tag).

In short, lots of reinforcement that the voice of the customer matters and shouldn’t be ignored!

comScore’s Silver Bullet (A Bit Tarnished, IMHO)

Since I said I’d go with the most retweets, I have to include this one from John Lovett, which was retweeted 10 times:

The key here is that comScore announced all the problems they were solving. The main differentiator, as best as I can tell, is that comScore is combining web analytics capabilities with its rich demographic/audience-based data. That might be slick, although it seems that they’re overpromising a bit when it comes to the flexibility of the tool and the completeness of the demographic data. I trust John…a lot…so maybe I’m being unduly and prematurely cynical. We’ll see.

Consumers Are Cross-Channel — So Should Be Your Analysis

At the risk of inflating John’s ego (which I’m not all that worried about, but if, ages hence, he’s turned into a pompous ass, I’ll dig up this post and claim credit for starting a perfectly pleasant guy down that path!), the next tweet and the last one are all Lovett-related. Lovett. Love it! 🙂 This next one was Eric Peterson quoting John and was retweeted 10 times:

Data integration and cross-channel analytics were covered by a number of presenters. With the exception of one vendor (who shall remain nameless…but who announced a name change to his company at the conference), the overwhelming agreement was that cross-channel integration is hard, tedious, expensive…and necessary. That one vendor had a video that showed it as being simply a technology issue (and they had the technology!). I’ve dabbled in the customer data integration (CDI) world enough to know that doing this integration at the individual person level is a bear.

But, because customers are living in multiple channels — offline, digital, mobile, social — and are switching freely between them, it’s dangerous to narrow in on a single channel and draw too many conclusions. This challenge isn’t going to go away any time soon.

Several times, both in sessions and in hallway discussions, it came up that both “WAA” and “eMetrics” have quickly become misnomers. Most of the attendees at the conference have responsibilities well beyond simply “web site analytics,” and simply “digital metrics.” I put a plug in that we could start considering “eMetrics” to be “everywhereMetrics,” which is a shameless ripoff of Resource Interactive‘s stance that “eCommerce” has become “everywhereCommerce.”

Fun times to come!

Consumer Privacy — the Regulations, the Law, the Ethics of It

Covered briefly in several sessions, touched on in the WAA Member Meeting, and then covered in depth in a panel was the challenges our industry is facing with regards to consumer concerns about privacy:

John has been the face of a multi-person effort to craft a code of ethics that individuals can sign that lays out how we will treat customer data. What became evident at eMetrics is that there simply is no easy answer to “consumer privacy.” And, the fact that the FTC covers the U.S. and has taken differing stances from the EU, and the EU will get to “one policy…implemented and enforced by country,” just makes my head hurt.

The good news, it seems, is that there seems to be an emerging philosophical consensus as to what is “good/okay” and what is “bad” when it comes to user tracking. The kicker is that it’s really, really hard to write that down in an unambiguous, loophole-free way.

If anything, I took away a sense of empowerment when it comes to really living the Code of Ethics and speaking up if/when I see an initiative starting to get into a gray area — it’s not just a “do the right thing because it’s ethical” case at this point. It’s a “do the right thing…or it might come out that you didn’t, and your brand can get burned severely.”

The Tweets Themselves…

As promised at the beginning of this post, if you want to download the data file with all of the tweets from 14-Mar-2011 to 16-Mar-2011 (Eastern time) that came out of my Tweetreach tracker, you can do so here. If you do anything interesting with them, please leave a comment here as to what that was.

Social Media

eMetrics Day 1 — Let's Look at the Tweets!

Update: I misstated @johnlovett’s follower count in the initial post. This was a fatigue-driven user error on my end — not bad data coming from either tool employed in this analysis and has been corrected!

Picking up on Michele Hinojosa’s quick analysis of tweets from the first day of the Omniture Summit, I thought I’d take a quick crack at Day 1 of eMetrics. I used TweetReach and a “tracker” (query) I set up a couple of weeks ago for that.

Now, I was a bit short-sighted, in that I set up the tracker on Eastern time. But, we still cover the main bulk of the tweets by selecting March 14th for the analysis range, so I’m not going to lose any sleep over it. The high-level summary:

Let’s take a look at some of the more interesting tweets, as identified using a few different criteria.

Just looking at raw exposure of the tweets, @SocialMedia2Day really dominated with their tweets. Now, @SocialMedia2Day has over 59,000 followers, which means every tweet gets recorded as that many impressions — even before anyone retweets (and there are more followers who might retweet). According to Twitalyzer, @SocialMedia2Day has an effective reach of 175,226, which puts the account in the 98.2nd percentile. The top 3 tweets, just based on raw exposure:

Notice that the top tweet had 10 retweets — 10 people in @socialmedia2day’s network thought it worth repeating. And, it’s a pretty good point content-wise.

@comScore also has a high follower count — more than 24,000, and an effective reach from Twitalyzer of 46,474 (94.3rd percentile). So, after all of the @socialmedia2day tweets comes a list of all of the @comScore tweets. Jumping beyond those as anomalies, of sorts, we get the top tweets by “individual” contributors:

John’s Code of Ethics tweet was retweeted 9 times and garnered almost 30,000 impressions. Nice! We care about acting responsibly! John’s tweet generated its exposure through retweets, as he has around 2,500 people following him…which is a lot of people, but only 1/4 of Ken, who has 10,000 people following him (and he’s following 10,000 people), so his tweets generate ~10,000 impressions just from him tweeting them.

So, looking at raw retweet volume is an indication of how naturally interesting and repeatable a user’s followers (and any followers who retweeted) found the tweet to be. The top retweeted tweet was retweeted 11 times:

Again…a pretty sharp observation.

Shifting around to the top contributors, TweetReach again provides a list based on the exposure generated by each user. The top 35:

We covered that @SocialMedia2Day, @comScore, and @kenburbary have a very high follower count, so let’s take a look at the next two. First, @michelehinojosa, who has just under 1,000 followers, an effective reach in Twitalyzer of 18,852 (89.7th percentila), and tweeted about eMetrics 127 tweets over the course of the day (tweet detail sorted by highest to lowest exposure):

Note the top two tweets were retweeted multiple times…and they’re worth sharing!

And, finally, yours truly — a bit under 1,200 followers, and a Twitalyzer effective reach of ~3,000  (although it jumped up to north of 89,900 starting on March 9th, which is twice what @comScore’s effective reach is, and they have 20X the followers; I need to ping the Twitalyzer folk to help me understand how that happened). My top 5 highest exposure eMetrics tweets for the day:

The second tweet — which was just a humorous observation — was interpreted as a “reply” to @jimsterne…but it showed up as the second-highest exposure tweet. That’s not exactly high-value content — more of a chuckle for those in the room who were watching the #emetrics stream. And, interestingly, I got a direct message from a follower midway through the day that they were unfollowing me as I was clogging their stream. I’m somewhat sensitive to that, but, with tweets being, essentially, public note-taking for me at conferences (and the enticing opportunity to then analyze and summarize those tweets after the conference, so it’s actually shared public note-taking), I suppose I’m okay with that.

Overall, this (very quick) analysis seems to reveal that the most engaging (egad! scary word!) tweets were one that stated, succinctly and eloquently, truths about our profession. I also  I would’ve liked to generate a word cloud of all of the tweets (appropriately cleansed)…but that’s simply not as quick and easy as I wish it was!

What do you think?

 

 

Excel Tips

Excel Dynamic Named Ranges Redux — Multiple Series in One Chart

In one of the more consistently popular posts I’ve written, I went into detail about how to set up charts that would update based on a value selected from a couple of dropdown menus – specifically geared towards a dropdown menu that allows the selection of a date such that the chart(s) would update to reflect the data up to that date.

One of the commenters asked how to include multiple data series in a single chart using that same technique. I did a very quick example via email, but I mentally committed to documenting the specifics on the blog, so here we go (file download at the end of this post).

Add Some Data

I could, of course, just use the data I was already working with, but none of that fictitious data made sense as a stacked bar chart. So, the first step is to add a couple of data series that might reasonably belong in a stacked column chart – an easy one is to break out the web traffic into “New Visitors” versus “Returning Visitors.”

Following the same technique as described in the original post, I name the top cells NewVisitors_Current (Column E) and ReturningVisitors_Current (Column F) and copy the formula from the Web Traffic column into those two columns (it’s the same formula in all cells in row 1, and they can be copied without modification due to the use of “COLUMN()” in the formula).

Then, create NewVisitors_Range and ReturningVisitors_Range named ranges by going to Formulas » Name Manager, copying the formula for WebTraffic_Range, and then creating the two new named ranges using the same formula, except swapping out “WebTraffic” in the formula with “NewVisitors” and “ReturningVisitors.”

Note: This may seem like a complicated setup. It’s actually pretty quick and simple, and can even be achieved using a macro if there are a slew of metrics that need to be set up. One tip, though, is to establish a consistent naming convention for the different aspects of each metric.

So, enough with the seup. How do we put multiple series into a stacked bar chart?

Copy One of the Line Charts

The easiest way to get our base chart is to simply hold down <Ctrl>-<Shift> and click and drag one of the existing charts straight down on the worksheet. I’m a fan of copying charts rather than making new charts from scratch for two reasons: 1) It’s easier to keep them aligned and exactly the same size, and 2) It’s easier to keep the formatting the same (the formatting in this example is horrid, but that was for the sake of simplicity in the initial tutorial).

So, now we have two charts (I copied the date and “current total” cells as well, but we’re pretty much done there now – in this case, the current total uses the “Web Traffic” value, and it’s the sum of the New Visitors and Returning Visitors):

Change the Chart Type

Select the chart and then go to Chart Tools » Design » Change Chart Type and select the Stacked Column chart type:

You will now have a chart that looks like this:

But, this is still only one data series, and it’s the overall web traffic – not the breakout of new visitors and returning visitors. So…

Update the Data Series

Click on the columns in the chart, and a formula will appear in the formula bar (it’s not you…it’s a small image; image width constraints I apply to this blog, but you get the idea):

There are other ways to update the data, but this is the fastest when it’s a viable option. Simply change the first “Web Traffic,” which is the name of the data series, to “New Visitors.” Then, change “WebTraffic” later in the formula to “NewVisitors”. What you’re really doing with this second change is changing the data source from “WebTraffic_Range” to “NewVisitors_Range”.

The chart will update and will look like this:

Now, since we’re going to have two series on this chart, let’s go ahead and click on the column title and change it to “Web Traffic” manually (when you changed “Web Traffic” to “New Visitors” in the formula bar, you were changing the series name — Excel just noticed that you had only one series and no legend, so it decided to make that the chart title, too; you’ll still want the series name to be “New Visitors,” though; the reason should become apparent shortly…like…after the next sentence!). And, while we’re at it, let’s add a legend and make the chart a bit taller to make room for it!

Add the Second Series

Now, here’s where the fun happens. Right-click in the chart and select Select Data. Then, select New Visitors and click the Edit button. You’re not actually going to edit that data series, but it’s the fastest way to get the second series set up. In the Edit dialog box, select the entire contents of the Series values field and select <Ctrl>-<C> to copy the formula:

Click Cancel.

Click Add.

For the Series name enter “Returning Visitors” and then paste the formula (<Ctrl>-<V>) you just copied into the Series values field. Then, scroll to the end of that formula and replace “New” with “Returning”:

Click OK and then click OK again on the next screen.

Voila!

Still, as before, you can change the Report Period and the Report Range dropdowns to alter what data gets displayed on the chart.

You can download the spreadsheet with the full example if you want to fiddle around with it without starting from scratch.

Happy charting!

 

 

Social Media

Omniture Announces SocialAnalytics

Omniture’s SocialAnalytics offering won’t be publicly available until summer of this year, but the early glimpses show big promise for the burgeoning field of SocialAnalytics. What makes this tool different from the many capable tools already out on the market is the tight integration of web analytics data with social brand or keyword mentions. This means that you can collect and analyze data from major social media channels like Facebook, Twitter, YouTube (45 data social media sources in total) and perform web analytics style slicing and dicing on the results.

Yet, the beauty of this solution is that users can trend and analyze social metrics against any metric within the SiteCatalyst interface. Further, the SocialAnalytics offering allows users to correlate data from social media with SiteCatalyst metrics and even offers a percentage of statistical confidence. This exceeds what I’ve seen in any other social analytics offering currently on the market. To illustrate with a hypothetical example, the Omniture SocialAnalytics capabilities will allow you to imbed traditional SiteCatalyst campaign ID codes into a your social media marketing on Twitter, YouTube and Facebook, which could all be monitored for activity within the SiteCatalyst interface. You could then trend the social data from campaigns and mentions against any metrics that you currently use within SiteCatalyst such as visitors or conversions. Thus, you could monitor the impact of your social marketing as a driver for website traffic and determine what percentage of that traffic actually purchased online as a result of the social campaign. The tool does this by making a correlation (versus actually pinning causation), but the statistical confidence will deliver assurance as to the validity of the correlation. This is magical. It actually enables users to quantify ROI from social marketing activities with a degree of statistical confidence. No one else has this that I’m aware of today.

Further, one of my pet peeves with today’s social analytics tools is the inability to create custom metrics. In most cases, you have to deal with the formulas and calculations that vendors deliver. The exception here is firms like Radian6 that allow users to weight factors for calculated metrics like Influence, whereby users do have some controls over their metrics. Yet, Omniture’s SocialAnalytics allows users carte blanche ability to create custom metrics and report on them within SiteCatalyst and even leverage in report builder and other Omniture functions. This is a revolutionary step in controlling the way that social is currently measured because it introduces a level of customization that was formerly absent.

While, it’s still early days and this was only my first glimpse at the product, you can probably tell that I’m bullish already. It’s currently in private beta for a few lucky Omniture customers who will undoubtedly bang away at it and help to shape the future of this product. However, there’s still a long way to go before this new tool is street legal, so most Omniture users will have to wait until the general release this summer. I’ll also say that this tool currently does not offer a wholesale replacement for Radian6 or other enterprise social analytics vendors on the market. The primary reason for this is that there is no engagement capability from the interface (i.e., can’t send Tweets or respond to Facebook comments directly). Additionally, there is no workflow built into the SocialAnalytics solution either. Thus, while social is about the interaction between a brand and its customers, Omniture is still leaving its clients to work that out using other means. They do however deliver some of the most robust analysis and reporting capabilities of anyone out there. If you’re looking to make sense of social media and measure the impact it has on your business operations; I suggest you give Omniture’s new SocialAnalytics tool a good look.

Adobe Analytics

Welcome to SiteCatalyst v15

Among the many announcements Adobe made at the 2011 Omniture Summit (#omtrsummit), probably the most anticipated was the release of version 15 of the flagship SiteCatalyst product. Those of us who follow SiteCatalyst regularly know that this release has been a long time in the making. Unfortunately, Omniture didn’t provide much detail in the keynote about specific enhancements so in this post, I will try to highlight some of the key things that I have heard about this new release (but in the interest of sharing info in semi-real-time, forgive me if I am not 100% correct and keep in mind I am writing this between sessions!). Since version 15 isn’t scheduled to be released right away (April?), not all features listed here are set in stone and as more details emerge about the release, I will follow-up with additional information/corrections…

Instant Segmentation Segmentation
The ability to segment data has always been a two-step process in SiteCatalyst. You could segment your data by passing values into eVars and sProps, utilize DataWarehouse/ASI, but if you wanted real-time segmentation you had to pay additional $$ for Discover or Insight. Unfortunately, most of the available options required you to wait for your segmented data which is not ideal from a web analytics perspective. However, when Google’s free analytics product released the ability to segment data in real-time, it became apparent that SiteCatalyst’s segmentation capabilities needed to be improved. The masses asked why they were getting less functionality than a free product?

With version 15, Omniture will now provide the ability to segment data in real-time. This will go a long way to appeasing those who realize that segmenting data is almost as critical as collecting the data. Instant segmentation will allow casual users to slice and dice website data without having to go through power users and then wait for the data to process. As you might expect, when business users have questions, they usually want the answer NOW! Forcing them to wait causes you to lose momentum and prohibits adoption so I think this feature will really help create more data-driven cultures. While this feature will be a big hit with the SiteCatalyst community, I expect that other web analytics vendors will position this as Omniture gaining parity with what they have already had.

One outstanding question I have is what this release means for ASI? Does this product/feature go away? Do customers who have paid for it, get some $$$ back?

New Architecture
So why did it take so long to introduce instant segmentation? Well the answer lies in the next big item Omniture discussed – a next-generation architecture. While I am not privy to all of the details, Omniture has stated that they redesigned the entire back-end of SiteCatalyst so that it could scale better and provide additional functionality like instant segmentation. Unfortunately, since most end-users won’t ever see the “back-end” of SiteCatalyst it will be hard to appreciate what went into it, but if this new architecture is as described, it should allow for more features and faster product improvements in the coming months/years.

However, there is one important catch to this v15 architecture. End-users will not be able to upgrade to v15 be themselves, but instead will need to work with Omniture to upgrade. This is due to the fact that once you upgrade, there is no going back. This is because v15 processes data differently than its predecessor. In general, v15 will process data in a manner that is more similar to Discover so users of both products should find that their data between SiteCatalyst match much more closely going forward. However, this means that SiteCatalyst v15 will approach things in a slightly different manner which could result in key metrics like Visits being slightly different than they were in previous versions. This means that looking at YoY data could show some variances, but SiteCatalyst will have an alert that tells you when you are comparing pre v15 data to v15 data so you are at least aware of this potential anomaly.

While it will remain to be seen how the SiteCatalyst community reacts to this, my hunch tells me that most clients will bite the bullet and upgrade to v15 and deal with this one time data discrepancy and take the benefits that v15 provides with respect to functionality.

More eVar Subrelations
Power SiteCatalyst users will rejoice in the fact that hey can now have full subrelations on more (all?) eVars. This means that you can break down more eVars by other eVars. In the past, you could only select a few conversion variables for which you wanted to see breakdowns, but this limitation will be reduced (abolished?) in v15. This is huge news and is another example of why the new back-end architecture is so vital.

UPDATE: Brett’s closing session suggested that ALL eVars and sProps could be broken down by each other. I have heard conflicting things on this so stay tuned!

Trend Multiple Metrics!
While it may not sound super-sexy, one new feature of the v15 release is the ability to trend more than one metric at the same time. To date, you can view multiple metrics in a “Ranked” report, but as soon as you switch to the trended view, only the 1st metric is trended. This has been a real bottle-neck and forced people like me to create additional reports in ReportBuilder to get this functionality. Version 15 solves this and I am told that it will continue to improve over time.

Visits & Visitors in all Reports
Another sticking point for SiteCatalyst users was that you could not see Visit/Visitor metrics in all reports. There were numerous workarounds, but most had an additional cost associated with them, but in v15 you can see both metrics in most reports. This will be a welcome addition, especially in conversion reports where they are needed the most. I have not confirmed whether Visits and Visitors will have full subrelations or not.

Ad-Hoc Unique Visitors
Currently in SiteCatalyst you can see unique visitors for set time periods, such as day, week, month, but if you choose an ad-hoc date range, you cannot see an accurate unique visitor count. In version 15, Omniture has rectified this like it had done in the Discover product. This feature will bring SiteCatalyst to closer parity to other web analytics vendors who have been providing similar unique visitor counts for any timeframe.

UPDATE: Brett’s closing session mentioned that you can also see ad-hoc unique visitors for Pages as well. That might mean that arbitrary time frame ad-hoc unique visitors might be available for all sProps?

Bounce Rate
After many requests from customers, Bounce Rate will finally be a standard metric in SiteCatalyst. Initially it sounds like it will be limited to a few reports, but it looks like it will be more pervasive in the future. While it has been possible to create Bounce Rate work arounds to compensate for not having Bounce Rate as a default metric, I think the gerenal population will be happy to have this baked into the product.

Video Enhancements
Previously, video data was relegated to video-specific reports only. Those clever enough would add custom metrics and eVars to get around this, but now it appears that you no longer have to do this to see video data in all SiteCatalyst reports.

New iPad App
In v15, the SiteCatalyst iPad app is getting a huge overhaul and will allow for more advanced web analysis on-the-go:

General UI Enhancements
Mixed into this release are a bunch of UI enhancements that people will probably like. These include searchable menus, report-specific default metrics, some new dashboard stuff and some more hand-offs between multiple Omniture products (like sharing segments with Test&Target). I think most will notice that that Omniture spent some cycles thinking about how an analyst uses the tool on a daily basis.

Long Live the Idea Exchange!
Lastly, I wanted to take a moment to thank Omniture for listening to its customers via the Idea Exchange. Many of the items above were highly voted upon by the Omniture community through the Idea Exchange. Omniture has done a great job of listening to its clients throughout the year (in addition to Brett’s fun Summit session!) so that it can focus its development efforts on what the majority of people are saying they want. It takes real courage as an organization to open up and ask customers what they want, interact with them, let all customers see this and then deliver the top items. While it sounds like common sense, there are very few vendors doing this today and I applaud Omniture for being forward-thinking about it. A special shout-out goes to Bill, JD and Ben who worked hard to champion this effort and I hope that v15 and beyond are the better for it…

UPDATE – ADDITIONAL FEATURES MENTIONED AT BRETT’S CLOSING SESSION

Dashboard Segmentation
In v15 it will be possible to apply real-time segments to SiteCatalyst Dashboards which will change all reportlets on the dashboard

Default Metrics by Report
In current versions of SiteCatalyst, you can set default success event metrics for conversion reports, but it is an all or nothing proposition. In v15, it sounds like you will be able to assign different default metrics for different conversion reports.

Data Warehouse Improvements
It sounds like v15 will provide more information about pending DataWarehouse requests and possibly allow for re-running (or “Save As”) of DataWarehouse requests. The latter will be a huge time-saver since today, you have to re-create each from scratch to make any changes…

Adobe SocialAnalytics
In addition to the new version of SiteCatalyst, another related product release is the Adobe SocialAnalytics product. This new product will compliment SiteCatalyst and will allow companies to monitor all social media activity. This product will be positioned as a competitor to Radian 6 and others in the social media measurement space. Key parts of this product are measurement for Twitter, Facebook and YouTube. Personally, I am excited that Omniture has formalized some of the cool social media tracking things I spoke about a few years ago and delivering on past promised features like viral video measurement. This new product will allow you to see Social Media metrics side by side, but and filter on specific influential users, but doesn’t appear to show if it is the same people who are coming from Social Media sites and then converting (if that is even possible!). Unfortunately, I believe this new product won’t be available to everyone until Q3, but it is interesting to see this new product, especially in the context of what was announced by Webtrends last week around social dashboards.

Final Thoughts
While I will defer final judgement until I learn more about all of the new v15 features, at a high level, I think that version 15 is a big step forward for Omniture. While there are not hundreds of new features, they have hit some really big ones that will have a real impact for power users. I predict that Omniture’s competitors will discount this release by saying that SiteCatalyst is now providing functionality they have had for years. While I could see that argument (and don’t disagree with it), I will offer the following perspective. SiteCatalyst was a product that experienced tremendous growth over a very short time frame as they went from a vendor that no one had heard of ten years ago, to one of the most popular web analytics tools used in the enterprise. With that growth, it was likely hard for SiteCatalyst to change its back-end architecture during this growth spurt, whereas other tools have either been around longer (and had more time), or come around afterwards (and had the ability to start with a clean slate). It is in that context that I still believe that Omniture is taking a big step forward and that the move to a new architecture is probably the right move for the SiteCatalyst product. I will be curious to hear your thoughts as you start seeing more about the product this week and beyond…

So those are a few of my favorite new features of SiteCatalyst v15 and some thoughts on SocialAnalytics and the new platform. What are your favorites? Have you heard of others? Are there any I listed that are not true? Which features were you hoping for that didn’t make the release?

As always, if you have any questions about SiteCatalyst or migrating to v15, feel free to contact me to learn more. Thanks!

Analytics Strategy, Reporting, Social Media

A Framework for Social Media Measurement Tools

Fundamental marketing measurement best practices apply to social media as much as they apply to email marketing and web site analytics. It all begins with clear objectives and well-formed key performance indicators (KPIs). The metrics that are actually available are irrelevant when it comes to establishing clear objectives, but they do come into play when establishing KPIs and other measures.

In a discussion last week, I grabbed a dry erase marker and sketched out a quick diagram on an 8″x8″ square of nearby whiteboard to try to illustrate the landscape of social media measurement tools. A commute’s worth o’ pondering heading home that evening, followed by a similar commute back in the next morning, and I realized I might have actually gotten a reasonable-to-comprehend picture that showed how and wear the myriad social media measurement tools fit.

Here it is (yep — click on the image to view a larger version):

‘Splain Yourself, Lucy

The first key to this diagram is that it makes a distinction between “individual channel performance” and “overall brand results.” Think about the green box as being similar to a publicly traded company’s quarterly filing. It includes an income statement that shows total revenue, total expenses, and net income. Those are important measures, but they’re not directly actionable. If a company’s profitability tanks in any given quarter, the CEO can’t simply say, “We’re going to take action to increase profitability!”  Rather, she will have to articulate actions to be taken in each line of business, within specific product lines, regarding specific types of expenses, etc. to drive an increase profitability. At the same time, by publicly announcing that profitability is important (a key objective) and that it is suffering, line of business managers can assess their own domains (the blue boxes above) and look for ways to increase profitability. In practice, both approaches are needed, but the actions actually occur in the “blue box” area.

When it comes to marketing, and especially when it comes to the fragmented consumer world of social media, things are quite a bit murkier. This means performance measurement should occur at two levels — at the overall ecosystem (the green box above), which is akin to the quarterly financial reporting of a public company, and at the individual channel level, which is akin to the line of business manager evaluating his area’s finances. I use a Mississippi River analogy to try to explain that approach to marketers.

Okay. Got It. Now, What about These “Measurement Instruments?”

Long, long, LONG gone are the days when a “web analyst” simply lived an breathed a web analytics tool and looked within that tool for all answers to all questions. First, we realized that behavioral data needed to be considered along with attitudinal data and backend system data. Then, social media came along introduced a whole other set of wrinkles. Initially, social media was simply “people talking about your brand.” Online listening platforms came onto the scene to help us “listen” (but not necessarily “measure”). Soon, though, social media channels became a platform where brands could have a formally managed presence: a Facebook fan page, a Twitter account, a YouTube channel, etc. Once that happened, performance measurement of specific channels became as important as performance measurement of the brand’s web site.

When it comes to “managing social media,” brand actions occur within a specific channel, and each channel should be managed and measured to ensure it is as effective as possible. Unfortunately, each of the channels is unique when it comes to what can be measured and what should be measured. Facebook, for instance, is an inherently closed environment. No tool can simply “listen” to everything being said in Facebook, because much of users’ content is only available to members of their social graph within the environment, or interactions they have with a public fan page. Twitter, on the other hand, is largely public (with the exception of direct messages and users who have their profile set to “private”). The differing nature of these environments mean that they should be managed differently, that they should be measured differently, and that different measurement instruments are needed to effectively perform that measurement.

Online listening platforms are not a panacea, no matter how much they present themselves as such. Despite what may be implied in their demos and on their sites, both the Physics of Facebook and the Physics of Twitter apply — data access limited by privacy settings in the former and limited by API throttling in the latter. That doesn’t mean these tools don’t have their place, but they are generalist tools and should be seen primarily as generalist measurement platforms.

Your Diagram Is Missing…

I sketched the above diagram in under a minute and then drew it in a formal diagram in under 30 minutes the next morning. It’s not comprehensive by any means — neither with the three “social media channels” (the three channels listed are skewed heavily towards North America and towards consumer brands…because that’s where I spend the bulk of my measurement effort these days) nor with the specific measurement instruments. I’m aware of that. I wasn’t trying to make a totally comprehensive eye chart. Rather, I was trying to illustrate that there are multiple measurement instruments that need to be implemented depending on what and where measurement is occurring.

As one final point, you can actually wipe out the “measurement instrument” boxes and replace those with KPIs at each level. You can swap out the blue boxes with mobile channels (apps, mobile site, SMS/MMS, mobile advertising). I’m (clearly) somewhat tickled with the construct as a communication and planning tool. I’d love to field some critiques so I can evolve it!

Analytics Strategy

Web Analytics Tools Comparison — Columbus WAW Recap Part 2

[Update: After getting some feedback from a Coremetrics expert and kicking around the content with a few other people, I rounded out the presentation a bit.]

In my last post, I recapped and posted the content from Bryan Cristina’s 10-minute presentation and discussion of campaign measurement planning at February’s Columbus Web Analytics Wednesday. For my part of the event, I tackled a comparison of the major web analytics platforms: Google Analytics, Adobe/Omniture Sitecatalyst, Webtrends, and, to a certain extent, Coremetrics. I only had five minutes to present, so I focussed in on just the base tools — not the various “warehouse” add-ons, not the A/B and MVT testing tools, etc.

Which Tool Is Best?

This question gets asked all the time. And, anyone who has been in the industry for more than six nanoseconds knows the answer: “It depends.” That’s not a very satisfying answer, but it’s true. Unfortunately, it’s also an easy answer — someone who knows Google Analytics inside and out, has never seen the letters “DCS,” referenced the funkily-spelled “eluminate” tag, or bristled at Microsoft usurping the word “Vista” for use with a crappy OS, can still confidently answer the, “Which tool is best?” question with, “It depends.”

And You’re Different?

The challenge is that very, very few people are truly fluent in more than a couple of web analytics tools. I’ve heard that a sign of fluency in a language is that you actually think in the language. Most of us in web analytics, I suspect, are not able to immediately slip into translated thought when it comes to a tool. So, here’s my self-evaluation of my web analytics tool fluency (with regards to the base tools offered — excluding add-ons for this assessment; since the add-ons bring a lot of power, that’s an important limitation to note):

  • Basic page tag data capture mechanics — 95th percentile — this is actually something pretty important to have a good handle on when it comes to understanding one of the key differences between Sitecatalyst and other tools
  • Google Analytics — 95th percentile — I’m not Brian Clifton or  John Henson, but I’ve crafted some pretty slick implementations in some pretty tricky situations
  • Adobe-iture Sitecatalyst — 80th percentile — I’m more recent to the Sitecatalyst world, but I’ve now gotten some implementations under my belt that leverage props, evars, correlations, subrelations, classifications, and even a crafty usage of the products variable
  • Webtrends — 80th percentile — I cut my teeth on Webtrends and would have put myself in the 95th percentile five years ago, but my use of the tool has been limited of late; I’m actually surprised at how little some of the fundamentals change, but maybe I should
  • Coremetrics — 25th percentile — I can navigate the interface, I’ve dived into the mechanics of the different tags, and I’ve done some basic implementation work; it’s just the nature of the client work I’ve done — my agency has Coremetrics expertise, and I’m hoping to rely on that to refine the presentation over time

So, there’s my full disclosure. I consider myself to be pretty impartial when it comes to tools (I don’t have much patience for people who claim impartiality and then exhibit a clear bias towards “their” tool — the one tool they know really well), but, who knows? It’s a fine line between “lack of bias” and “waffler.”

Any More Caveats Before You Get to the Content?

My goal with this exercise was to sink my teeth in a bit and see what I could clearly capture and explain as the differences. Ideally, this would also get to the, “So what?” question. What I’ve found, though, is that answering that question gets circular in a hurry: “If <something one tool shines as> is important to you, then you really should go with <that tool>.” Two examples:

  • If enabling users to quickly segment traffic and view any number of reports by those segments is important, then you should consider Google Analytics (…or buying the “warehouse” add-on and plenty of seats for whatever other tool you go with)
  • If being able to view clickpaths through content aggregated different ways is important, then you should consider Sitecatalyst

These are more of a “features”-oriented assessment, and they rely on a level of expertise with web analytics in order to assess their importance in a given situation. That makes it tough.

Any tool is only as good as its implementation and the analysts using it (see Avinash’s 10/90 rule!). Some tools are much trickier to implement and maintain than others — that trickiness brings a lot of analytics flexibility, so the implementation challenges have an upside. In the end, I’ll take any tool properly implemented and maintained over a tool I get to choose that is going to be poorly implemented.

Finally! The Comparison

I expect to continue to revisit this subject, but the presentation below is the first cut. You might want to click through to view it on SlideShare and click the “Speaker Notes” tab under the main slide area — I added those in after I presented to try to catch the highlights of what I spoke to on each slide.

Do you see anything I missed or with which you violently disagree? Let me know!

http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6= http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6=

http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6=http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6=http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6= http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6=

http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6=
http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6=

http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6=
http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6=http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6=http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6=

http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6=

Reporting

Campaign Measurement Planning — Columbus WAW Recap Part 1

We tried a new format at last week’s Columbus Web Analytics Wednesday, in that we had three completely unrelated presentations, and we kept the entire presentation period to right at a half hour. Mathematically, that gave us 10 minutes per presentation, and we split the time between formal presenting and Q&A. The event was sponsored by Resource Interactive (population 320-ish and growin’; SA…LUTE! </heehaw>), and it was our first “presentation included” WAW since last November. Apparently, we had some pent-up WAW demand, as we had right around 45 attendees.

The three presentations of the evening were:

Dave’s presentation was the most informal and focussed on the various developments across Facebook/Bing and Google when it comes to incorporating social graph and social profile data into search results. The Google video he showed was pretty interesting, and he illustrated how rapidly the space is evolving. But, overall, I took lousy notes, so I don’t know that I’ll manage to get a full blog post up on the subject.

As for Bryan’s presentation, I had the benefit of previewing the material and, as such, getting to have a mini-Q&A with Bryan via e-mail.

Campaign Measurement Planning

Bryan and I are both pretty passionate about measurement planning. His presentation really nails some key points about the topic and has a fantastic list on slide 8 as to elements to consider including in a measurement plan:

In addition, Bryan provided a (click to download) measurement plan example Word document (it’s an auto insurance company example, so it’s obviously grounded in reality, but he worked it over pretty thoroughly on several fronts in preparation for the presentation, so it is an entirely fictional example).

I asked Bryan a couple of questions offline about his approach prior to the event:

Q: Under “Targets and Benchmarks,” you note, “Don’t be afraid to put ‘TBD’ or ‘No Data’ for some benchmarks.” If that is the case, do you support not setting a target, or should you still try to set a target (even noting that it is a bit of a swag) in the absence of a benchmark?

Bryan’s response: I try to set a target no matter what because it gets people at least thinking about it and TRYING to set up some kind of expectation.  It makes sure that people are at least estimating.  Maybe they don’t know the CPC for the search terms yet, aren’t sure on the demand, and aren’t sure on the completion rates, but it’s at least a start.  We were completely off for one of our last campaigns because we had no idea on all those factors.  It still gave the agency something to report on for their % of goal and it drove an informed discussion mid-campaign.

[I, of course, loved this answer…because I totally agreed with it]

Q: You’re at a company that uses agencies for much of the campaign execution, and, clearly, you have put in a process whereby you develop this sort of plan partly as a tool to drive clarity and alignment with the agencies and their work. As an agency analyst, we are increasingly including “measurement planning” as a non-optional part of the scope of our engagements. In those cases, we (the agency) actually do the discovery and documentation of the measurement plan (which clients provide input to, review, and approve). I actually would love to have clients coming to us with this level of forethought, but, in the absence of that, what are your thoughts on having the accountability for the creation of a measurement plan reside with an agency?

Bryan’s response: I think this varies on the relationship between the agency and company.  For us, we’re very capable, we all know how to do the campaign execution, but we just don’t have the time or bodies to do it.  I’m sure there are many companies that have no clue how to do it, so the agency does both the execution and the strategy, or the execution, strategy, tracking, plus reporting, or whatever else. It really depends on the analytics maturity of the client as to whether it makes more sense for the agency or the client to own the creation of the plan.  If it’s the agency, you’d have to be absolutely be sure to talk to the client in depth about all of it and make sure they’re on board with all the points.  In the end, the outcome should be the same, the only difference really being the author of the document would be the agency instead of the business, and I’m sure some of the reporting responsibilities would change based on that.

Bryan joked during his presentation about how “exciting” the topic of measurement planning is. Obviously, it can seem like a pretty dry topic, but, in both of our experiences, measurement planning can drive some tough and interesting discussions. More importantly, it’s a foundational element of marketing — without it, you wind up looking back after the fact and wondering if what you executed was successful, whether you captured the right data, and whether you learned anything that can be meaningfully applied to the next initiative.

Hey…I also cleaned up my “sharing” options on my blog this weekend. Go ahead. Give it a try! See how easy it is to Like or Tweet (or…er…whether I really got those implemented and functioning correctly). Who knows, maybe Facebook Insights will start giving me some interesting web site data!

General, Technical/Implementation

Need A Checkup? The Doctor Is In!

When it comes to your health, most doctors say that having a regular checkup is the easiest way to prevent major illness. By simply going to see your doctor once a year, you can get your vitals evaluated and see if your blood pressure is too high or low, check your cholesterol, etc… If you happen to be sick at the time you have your checkup, you can find out if it is serious or not and if you feel fine, the checkup is a way to confirm that you are in good shape.

However, when it comes to web analytics implementations, it isn’t always easy to know how “healthy” you are. You might wonder the following:

  • Is my organization capturing the right data to ensure it can do the analysis needed to improve conversion rates?
  • Do the configuration settings of our web analytics tool make sense?
  • Are we maximizing the use of our web analytics tool or are we only using 20% of its capabilities?
  • How does our web analytics implementation compare to that of my peers/competitors?

Over the past decade, I have been associated with hundreds of web analytics implementations, and the above questions were ones that often kept my clients awake at night. And, truth be told, based upon my experience, many of them had reason to be worried. More often than not, when I crack open a client’s web analytics implementation, I am shocked by what I see. Here are a few examples of problems I encounter repeatedly:

  • Unusable pathing reports due to inconsistent page naming practices
  • Unusable campaign reports due to inconsistent tracking code naming conventions
  • Web analytic variables/reports defined, but with no data
  • Cookie settings that don’t line up with business goals (i.e. Cookie using Last Touch when Marketing uses First Touch)
  • Data inconsistencies resulting in reports that are highly suspect or untrustworthy
  • Incomplete meta-data or look-up tables
  • Lack of critical KPI’s and best practices specific to the industry vertical the website serves
  • Lack of appropriate usage of key web analytics tool features that could improve overall analytic success

The remainder of this post will discuss a new service offering Analytics Demystified will be providing to address the preceding concerns. If you are interested in knowing the “health” of your organization’s web analytics implementation, please read on…

Introducing the Web Analytics Operational Audit

So how do you know if you are doing well or poorly? Like anything, the best way to know where you stand is to perform a checkup or audit. In this case, I am referring to an audit that reviews which web analytic tool features you are utilizing and what data your web analytics implementation is currently collecting.

Since there is no official “doctor” when it comes to web analytics, we at Analytics Demystified have created what we believe is the next best thing. Taking advantage of our depth of experience in the web analytics arena, we have created a Web Analytics Operational Audit scorecard that encompasses the best practices we have seen across all company sizes and industry verticals. This scorecard is vendor-agnostic and has over 100 specific items and categories that allow you to see where your current web analytics implementation excels and where it is lacking.

Over the years, I have done this type of scoring informally, but the Operational Audit framework we have created at Demystified takes this to a whole new level. Here is a snapshot of what the scorecard looks like so you can see the format:

Our goal in creating this Operational Audit project is to have a simple, yet powerful way to objectively score any web analytics implementation from a functionality point of view. Knowing where your organization stands with respect to its web analytics implementation is beneficial for the following reasons:

  • If you think you have a robust implementation, but it turns out that you do not, you may be making poor business decisions today based upon faulty data and/or incorrect assumptions
  • What if your implementation is worse than you thought? You can try and hide it, but I have found that in the long run, bad web analytics implementations are eventually found out…usually at the worst time when an executive needs something critical and you have to come back and say “sorry, we don’t have a way to know that…” Wouldn’t you like to know sooner, rather than later, what shape you are in so you can get your web analytics house in order?
  • Maybe you have an awesome web analytics implementation, but your boss doesn’t know it! What would it do to your job/career if your boss was told by an independent 3rd party that all of the time and money they invested in your web analytics implementation have paid off! What if your web analytics implementation was in the top 10% of the general web analytics population? Promotion anyone?
  • Your organization doesn’t have unlimited time and budget for web analytics implementation projects. When the stars align and you do get resources or budget, wouldn’t it be great to be armed and ready with the top things you should be doing so you don’t miss these golden opportunities?

These are just a few of the many reasons that auditing your implementation makes sense. One important note: this Operational Audit does not include a technical audit of JavaScript tagging (which can be equally as important!).

Go Forth and Audit!

As I stated earlier, the unfortunate truth is that there is more bad than good out there. People change roles, priorities change, people leave your company, companies merge. There can be any number of reasons contributing to the devolution of web analytics implementations, but regardless of how you got to where you are, if you want to be successful, you need to grab hold of the reins of your current web analytics implementation and take ownership of it.

For example, when I joined Salesforce.com, I could have spent my time blaming our implementation shortcomings on my predecessors, but that wouldn’t help me get to where I needed to go. Instead, I chose to audit our implementation and identify what was worth keeping and what had to go! In the end, our company was better for it, and the audit led to an implementation roadmap for the next year, allowing me to know how long it would take to turn things around and what type of resources I would need.

It is based upon this recent experience that I highly encourage you to consider this Operational Audit service for your organization. Long term, one of my hopes is that I can audit enough companies, across various company sizes and verticals to enable me to create a benchmark of web analytics implementations so I can let you know how your scores compare to others like you. This way, even if most companies score poorly, you can possibly claim to be the best of what is currently out there (can you tell I liked being graded on a curve in high school?). I am also looking forward to re-scoring companies next year so they can see how their implementation has improved year over year.

Intrigued? Interested? Scared?

If you’d like to learn more about having your web analytics implementation audited, please contact me and I’d be happy to answer any questions. Thanks!

 

Reporting

The Ugly Truth About Benchmarks

Why Do We Want Benchmarks in the First Place?

As Garrison Keillor says every week, in Lake Wobegon, “all the kids are above average.” If we can simply be “above average,” then we know we’re pulling away from mediocrity. And that’s what we want with benchmarks — we want to know what “average” is so that we know the exact height of the measurement bar that, if we clear it, we can claim success (if not necessarily supremacy). It’s something to aim for that must be attainable, because others have attained it.

We’re surrounded with benchmarks in our personal lives, too: doctors tell us how our weight, blood pressure, and cholesterol compare to benchmarks for healthy people of the same age, gender, and height; standardized testing in schools are compared to statewide benchmarks; salary surveys tell us (generally in a flawed way) benchmarks for pay for others in our field. We’re used to benchmarks, and we want to use them to set targets for the key performance indicators (KPIs) for our marketing initiatives.

Benchmark = Target…right?

All too often, I run up against someone who equates a benchmark with a target. That’s dangerous for two reasons:

  • Benchmarks are a reasonable sanity check, but targets should be driven by what success will really look like — where does a particular metric need to be in order to justify the investment required to get there?
  • If targets are solely driven by benchmarks, then it’s an easy (if faulty) deductive leap to believe that, in the absence of a benchmark, no target can be set

So, resolved: benchmarks are not targets.

The Benchmarks We Most Want Are the Ones We Can’t Realistically Have

The easiest, and, in most cases, most relevant and useful benchmarks generally come from your own historical data. If you’re considering an initiative that will improve a certain metric, then your track record with that metrics is a fantastic baseline input into target-setting. Since that data is usually readily available, it gets used. It’s when a totally new initiative is launching — a Facebook page, a mobile app, a community contest — that we get the most anxious about what a “reasonable target” is and, therefore, launch a quest to find benchmarks.

The problem is that these are most often the benchmarks that are least likely to be available. Or, if they are available, there is so much variability inside the data set that it’s hard to put much stock in the data.

Even with something as massively established as email marketing, getting a reasonable benchmark for something as common as open rate has a lot of underlying variables mucking up the data:

  • The type of e-mail — newsletter vs. general promotion vs. targeted promotion vs. something else
  • The target of the e-mail — internal house list vs. rented list, for instance
  • The specific industry and consumer type the emails target
  • The email platform in use and how it captures and calculates open rate
  • The basic deliverability of the emails included in the benchmark, as driven by content, email platform, and user type

If all of these factors are at work with something as established as e-mail, then what does that mean for a relatively knew and evolving medium like social media or mobile? Almost every time we launch a new Facebook page, we get asked what the “benchmark is for new fan growth.” In that case, the single biggest driver of fans — outside of brands that have a massive number of rabidly enthusiastic customers — is the promotion of the page, be it through Facebook advertising, through channels the brand already owns (email database, web site, TV advertising, etc.), or through paid promotion elsewhere. It’s an unsatisfactory reality…but it’s reality nevertheless.

Should We Just Abandon All Hope, Then?

There are some cases where relevant and appropriate benchmarks are available. For instance, Google Analytics provides benchmark data for common web metrics based on sites of “similar size” and in a user-selectable site category/industry. Twitalyzer can be used to gather benchmarks using all of the tracked users who fall into a given “community.” Email marketing platforms often do provide benchmark data by industry, but they can fall short on the critical “e-mail type” front. When benchmarks are available, by all means use them as an input!

In the absence of available benchmarks, meaningful targets can absolutely still be set. It’s just largely a matter of ferreting out stakeholder expectations. Expectations always exist, even if they are claimed to not:

Expectations almost always exist. In the (real) example illustrated above, I pointed out that, if there truly were no expectations, then there would have been no “shock.”

The expectations that exist may not be precise , but, with a little bit of probing, you can generally find a range, below which the initiative will undoubtedly be judged as disappointing, and above which the initiative will certainly be judged a success. Starting with that range and then narrowing down as best you can and getting agreement of this target range from all of the key stakeholders is just smart performance measurement.

Analytics Strategy, Conferences/Community

Guest Post: Success in The Analysis Exchange!

Since Analysis Exchange has been honored with a nomination in the Web Analytics Association Gala Awards, while our community is considering their votes I figured it was a good time to share some of the great email we get from Exchange participants.  This one is from David Schuette who started as a student and has already graduated to mentor!  You can follow David on Twitter @TheCakeScraps and thanks to David and everyone who has benefitted from Analysis Exchange!

If you are in the WAA please consider that a vote for Analysis Exchange is a vote for EVERYONE who contributes to the effort around the world.

A Tale of Two Projects

In the middle of 2010 – 2.5 years into my career as a web analyst – I made one of the better decisions on my journey through the field of web analytics.  A friend of mine, active in the community for some time, pointed me to a project called the Analysis Exchange; he encouraged me to check it out and to sign up as a student.  I did some research and it seemed like a great match.  I would get to help nonprofits and learn a lot in the process.

I’ll be honest; it took a while for me to secure my first project.  I wasn’t sure what the problem was until Eric pointed out (to all members) that a complete profile greatly contributed to the likelihood that a student or mentor would be selected for a project.  I filled it out and started applying again.  At the time there were only a few projects available, in contrast to the 5+ open right now thanks to the hard work of Wendy and team, so it took some time but I was picked to work with Kids Matter, Inc. – an organization supporting foster children.

The experience couldn’t have been better.  Megan, the partner at Kids Matter, was filled with excitement and ambition.  She had done some great work for her organization and wanted to learn more.  She wanted to let the data take away some of the guessing and let it do part of the work for her.

I dove right in and, before long, I had a great presentation that I was able to tweak based on the feedback from my mentor.  The presentation went smoothly and the people at Kids Matter were extremely appreciative of the work.  I even got a thank-you card that was hand-made by one of the kids.  It really made me stop and appreciate just how much good can come from a little time given.

While I was busy working on my first project, the Analysis Exchange kept improving.  The Google Group, a bit quiet recently, contributed in a huge way to make small but important improvements to the Exchange site.  It is cool to look at some of the discussions from just a few months ago and see the ideas already implemented into the site.  It made it all the easier to sign-up for my next project, at the Apalachicola Bay Chamber of Commerce.

The second project went as well as the first.  My mentor and I provided a high-level usability driven analysis to Anita, our partner at Apalachicola Bay.  The analysis focused on opportunities to draw visitors deeper into the site so they could really see what the Apalachicola Bay area had to offer.  Again, our partner was excited about the results and was genuinely appreciative of the work we put in.  It was our pleasure.

And now I have transitioned myself to a Mentor on The Exchange.  If my next experiences are half as good as the first two I would be thrilled.  I’m excited, even anxious, to have the chance to help another organization and provide some coaching to an upcoming analytics ninja.  But I also view this change to a mentor as a re-upping of my commitment to The Exchange; I have made it my goal to bring at least 1 local non-profit to The Exchange this year and hopefully more!

Everything about my experience has been wonderful.  If you have thought about joining, or perhaps have not participated in awhile, go check it out.  You won’t regret it.

Social Media

Twitter Influence — Still Searching for the Perfect Answer

[Updated on 2/17/2011 — added the last section with additional information about Twitalyzer’s Community measurement and a little additional nod to TweetReach.]

A pretty intriguing post from Michael Healy came across the Twitterverse yesterday: #Measuring in 2010 — Analyzing the #measure Data of the Twitterati. What Michael did was take all of the Tweets that used the #measure hashtag in 2010 and run them through an “influence” formula he developed. The tweets data was courtesy of Kevin Hillstrom, who had set up a Twapper Keeper archive of #measure tweets. I’ve set up a couple of Twapper Keeper archives (hopefully, the #emetrics one will continue to function through the upcoming eMetrics conference, as I smell another juicy data set for us to play around with)  in my day and have been a bit frustrated with the quality of the exports — they required quite a bit of cleanup, especially of the timestamps — and, I’ve been a little skeptical of the completeness of the data. But, maybe that’s just because I’ve been working with TweetReach of late, and it’s just so darn clean and robust that the free services really do start to pale in comparison.

I digress.</statementoftheobvious>

Michael’s stated goal was pretty simple:

I wanted to know who were the most influential members of the #measure Twitterverse.

This was an exercise he did as prep work for the Web Analytics Association Spring Awards Gala, which, if you’re going to be in the area, you should plan to attend, as it should be a really good time.

I read the post and had three immediate thoughts:

  • “Influence” is one of the Mid-Major Holy Grails of social media management
  • “#measure” could be replaced by any brand or topic in, and the ability to achieve Michael’s stated goal would come in damn handy in all sorts of situations
  • Michael is one of those Brains with a capital “B,” so it’s worth taking a close look at what he produces when he claims he’s “just having fun.”

The final result was a big ol’ diagram (click on the image to jump over to Michael’s original post and a link to the full-sized image):

Michael’s formula focussed on both the volume of tweets and the “original content” in the tweets (using an Entropy calculation and a slight dampening of the score based on the volume of retweets).

Like Michael, I was surprised to see Szymon Szymanski (@ulyssez) as the dominant circle in the diagram. I’d certainly seen his tweets in the #measure stream, but I would have been more likely to guess @aknecht or @KISSMetrics (which is the good-sized circle off to the right of the diagram, as it so happens) would have had the dominant slot based on volume/variety.

So…Influence, You Say?

Seeing as I’ve been spending a lot of time with Twitalyzer of late, the next thought that popped into my mind was, “I wonder how Michael’s analysis of the top influencers would line up with Twitalyzer’s?”

It doesn’t take much to push that thought further and immediately hit a wrinkle:

  • Michael’s definition of influence was oriented towards tweet volume and tweet content
  • Twitalyzer’s definition of influence is based on an assessment of how likely a user is to be referenced or retweeted

Now, logically, if you have a high tweet volume, and your tweets contain a lot of original content (and, presumably, it’s not navel-gazing content, as that would rarely warrant the inclusion of the #measure hashtag), you’re more likely to be referenced or retweeted. Okay, there’s a logical link there, so maybe that’s not a huge wrinkle.

“Oh, bother,” said Pooh almost immediately, “I think we have a second wrinkle.” That being:

  • Michael’s analysis was based solely on tweets that included the #measure hashtag and the users who tweeted those tweets
  • Twitalyzer’s definition is more “user-based,” and takes into account the user’s direct and indirect network

And, a third wrinkle, just to round out a nice list:

  • Michael’s analysis was based on all #measure tweets from 2010
  • Twitalyzer operates more on a last day, last 7 days, last 30 days mode (with historical data going back much further…but it’s all based on when the user got plugged in as a daily-updated account)

For this third wrinkle, it seems reasonable to assume that the most influential #measure tweeters in 2010 are likely still fairly influential as of the last month.

In the end…does it matter? There’s only one way to know! Let’s take a look!

A Semi-Random Comparison

I’m not going to go through every bubble in Michael’s diagram. But, I am going to hit the “big” bubbles, look up their Twitalyzer Influence scores for the last 30 days, and then do the same for a smattering of small bubbles. For the “small bubble” users I”ve only included users where it looks like Twitalyzer has been doing daily tracking for the last 30 days. And, these bubbles were also a random selection of users that I readily recognized (which, I realize, very likely introduced some sample bias).

Let’s see what we see:

Username Bubble Size Twitalyzer Influence
@ulyssez Ginormous 1.0%
@immeria Huge 1.0%
@analyticscanvas Huge Not available*
@kissmetrics Damn Big 24.0%
@mongoosemetrics Big 5.0%
@thebrandbuilder Big Not available*
@cjpberry Pretty Big Not available*
@usujason Pretty Big 2.0%
@corryprohens Pretty Big 0.0%
@jdersh Pretty Big 1.0%
@minethatdata Pretty Big 3.0%
@hkwebanalytics Pretty Big 1.0%
@johnlovett Pretty Big 2.0%
@jimsterne Small 2.0%
@analyticspierce Small 1.0%
@ericjhansen Small 1.0%
@aknecht Small 4.0%
@tgwilson Small 2.0%
@jojoba Small 1.0%

* These accounts had not yet been Twitalyzed. As such, while I Twitalyzed them, a reliable 30-day average was not available, so I have not included their reported scores here.

So, what does this tell us? Well…seems like we don’t have a perfect correlation (we never do, do we?). From my own use of Twitter, the Twitalyzer scores square pretty well with what I would expect, although, yowza!, I wouldn’t have expected @kissmetrics to be running away from the pack like that!

I don’t think either one of these is “right” in any absolute way. Both approaches were developed with different purposes. Michael’s exercise was, I think, a couple of idle thoughts taken to a logical conclusion. Twitalyzer’s score is one metric inside a measurement platform that offers a whole suite of metrics and that has been evolving and maturing a couple of years.

Does Any of This Really Matter?

I’m drawn to these sorts of exercises because I think they do matter. As web analysts, we got to a pretty consistent definition of a “page view” and a “visit” (“different tools calculate differently” be damned — the basic definition is the same), left things a little loose on “unique visitors,” and never really reached closure on “engagement” (philosophical debates as to whether it even matters notwithstanding).

As social media continues to gain traction with consumers and as social media platforms continue to evolve and mature, we absolutely need to be thinking about measurement within those platforms and we need to keep scrambling to keep up. And, hopefully, maybe we’ll be able to influence the evolution of those platforms so that they’re at least somewhat measurement-friendly. As long as we’ve got analysts pushing the tools and experimenting with new approaches (another example: @jojoba’s oxygenating alter ego posted her Social Media Masters Twitter Analytics presentation over the weekend — lot’s o’ tools out there!), we’ll get there!

So, yeah, it matters. The fact that it’s pretty interesting to watch (and maybe even help) some really, really sharp minds in our space try to crack some pretty hard nuts is just added gravy.

Update: Twitalyzer Community Scores

One of the few benefits of being based in the Eastern timezone with a lot of the heavy analytics work occurring on the west coast is that I got to get up this morning with an inbox and comments on a post that went up shortly before I retired for the evening!

Eric Peterson sent me some of his thoughts and pointed me to the Community area under Tweets and Tags in Twitalyzer:

So…now I need to go do some more Twitalyzer exploration and thinking — from the list above, I need to think through the relationship between Participation, Influence, and Attention, methinks. One of the real draws, for me, of Twitalyzer, is that it enables picking a set of appropriate metrics that, together, measure the effectiveness of any particular Twitter engagement approach. The kicker is nailing down which of those metrics are the right fit in any given situation.

Jenn Deering Davis of TweetReach also sent me some TweetReach data on #measure that covers 2011 to date. TweetReach focusses on reach and exposure of tweets (the difference being that reach is “unique people exposed” and exposure is more “raw impressions”). From that perspective:

TweetReach’s approach has a more direct tie to traditional advertising measurement when it comes to tracking “impressions.” But, it also has a lot of other features that can help sniff out influential people on a particular topic — a major differentiator is that its trackers can use boolean logic, which they showcased in the work they did around the Super Bowl ads. It doesn’t really show this off when we’re looking at a community that is defined as tightly as the #measure hashtag.

Again, I say… so many tool…!

 

Social Media

Is Social Media Encouraging Narcissism?

I’m a little worried about us. At first I was really psyched to see a tweet about my friend and business partner Eric appearing the the WSJ for his not-so-small side project, Twitalyzer. I eagerly clicked through from Tweetdeck to read all about the great strides that Twitalyzer was making in the marketplace only to be massively disappointed by the article, Wannabe Cool Kids Aim to Game the Web’s New Social Scorekeepers. This article is all about gaming the social system to increase influence scores from services like Klout and Twitalyzer and to personally benefit from doing so. Is this what we’re training kids to aspire towards today?

Have you Googled yourself lately…?

Okay, just admit it. At one point of another you’ve typed your own name into to Google just to see what shows up. Or perhaps, if you’re like me you’ve even created a proactive alert that informs you every time you or your business is mentioned in media outlets? It’s not that I’m vain, but I want to know when something or someone publishes about me or about our brand. Isn’t this the cost of putting yourself out there today? Social media has accelerated this exponentially.

I don’t fault people like the ones described in the WSJ article for working to improve their social influence scores as long as they’re genuine. It’s smart to understand how rankings are formulated and how you can improve your scores. That makes the difference between individuals who are building their personal brands with an entrepreneurial drive and those who simply aren’t tuned in enough to know how. Done right, that’s commendable. But understanding the system and rigging it to your favor is potentially where we’re headed in this age of social media. It’s an environment where your potential employer will check your Facebook page prior to extending that job offer; and they definitely will follow your Tweets after that offer is extended; and you can bet on the fact that they’ll be watching your social escapades after you’re hired to ensure that you don’t misconstrue ideas that are yours alone with those of your employer. Or heaven forbid you’re passed over for a consulting job because of a low Twitalyzer score, like the story Shel Israel foretells. But, this is business today, I just wonder if we’re encouraging an unhealthy level of narcissism?

What’s your Social Media Credit Score…?

One of the topics I’ve been researching lately is Social Media Profile Management. This started with the whitepaper that I authored for Unica called, True Profiles: A Contemporary Method for Managing Customer Data (download the paper next week) where I explored what it takes to integrate data streams from disparate sources. Yet, while that’s happening on the business side, consumers are in desperate need of managing their own social profiles. Services like Rapleaf, PeerIndex, Klout and Twitalyzer all reinforce the need to know how you’re portrayed as an individual in social circles and how much personal information about you is floating around out there.

Brian Solis talks about this as well in his compelling Lift presentation where he describes the sociology and psychology behind what we do in social media. He mentions that debt collectors are now visiting individual’s Facebook pages to track them down and sometimes publicly humiliate them into paying their debts. That’s absolutely frightening! But it’s a reality of the world we live in.

Managing your social credit score is important and undoubtedly we’ll see a burgeoning slew of services like Identity Mixer and others that allow you to manage what appears in the databases of companies like Spokeo.com and whitepages.com for all to see. You’re already being indexed, ranked and reported on whether you like it or not. I just can’t help from wondering if the way we (or at least some people) operate with the aid social profile management technologies is disingenuous?

What Should You Recommend To Your Business…?

Those of you who know anything about Analytics Demystified recognize that we’re not ones to take data and simply gaze at it in wonderment. We use data to make recommendations. More importantly, we encourage you to do this as well. So for all the measurers of social media out there, take into deep consideration the value you place on influence. I do believe that it’s a meaningful metric and I am optimistic about < foreshadowing > new developments on the horizon from Social Analytics vendors in this area < /foreshadowing >, yet you have to understand what your metrics are made of and how they’re calculated.

That’s the thing that irked me most about the WSJ article was that it implied in the subtitle that all the vendors out there keep their influence rankings secret. Twitalyzer doesn’t do this, in fact they expose all of the factors that go into their calculated metrics for all to see. While some metrics within the Twitalyzer dashboard do rely on scores from other technologies like PeerIndex and Klout, they’re labeled as such with nothing secret about them. I’m not bringing this up to tout the greatness of Twitalyzer, but more so to call out the fact that transparency in the metrics you use and rely on is critically important.

Hopefully, most of you are migrating away from counting measures like fans and followers that offer little more than a measure from an uncalibrated yardstick and adopting business value metrics that actually mean something to your organization. If you are working toward this end — and if influence is a measure that will factor into your marketing efforts — then take the time to see through inflated scores and popularity hounds that are gaming the system. It’s likely that you don’t want these people doing your bidding anyhow. Instead, use measures of success like Impact to correlate influence to action. When you begin to look at your social marketing efforts in this way, you may just find that those with the most “popular” profiles aren’t actually good for your business.

Reporting

Pocket Guide to Identifying Great KPIs

Here’s a quick post sharing a printable reference for establishing clean, clear, and appropriate KPIs for a project. This was something that Matt Coen, one of my peers at Resource Interactive, and I developed in response to some internal requests coming out of a measurement class that we teach both internally and for some of our clients. But, we agreed it was worth sharing with the broader measurement community. The goal was to put something in the hands of analysts or marketers that would actually give a practical guide to the questions to ask when heading into a project to ensure the establishment of effective KPIs up front.

I see this as a complement to one of my favorite Avinash Kaushik posts, which I think I’ve been referencing almost since he wrote it…and I now realize that was over three years ago! The meat of the post is his list of “four attributes of a great metric” (they’re Uncomplex, Relevant, Timely, and “Instantly Useful”). I see this guide as a guide to how to ask the right questions such that you wind up with great KPIs (it works for non-KPI measures, too, but the focus is on KPIs specifically). It’s not rocket science by any means, but it’s handy! Click on the image for a larger version, but see below if you want to print it.

Guide to Great KPIs

A small (half-a-page), black-and-white version of the guide is available in this PDF. The PDF actually has the same diagram twice. Print it, cut it in half, and pass the second diagram along to a colleague who might find it handy!

What do you think? What’s missing?

General, Social Media

Measuring the Super Bowl Ads through a Social Media Lens

Resource Interactive evaluated the Super Bowl ads this year from a digital and social media perspective — how well did the ads integrate with digital channels (web sites, social media, mobile, and overall user experience) before and during the game. I got tapped to pull some hard data. It was an interesting experience!

A Different Kind of Measurement

This was a different kind of measurement from what I normally do. I definitely figured out a few things that we’ll be able to apply to client work in the future, but, while, on the surface, this exercise seemed like just a slight one-off from the performance measurement we already do day in and day out, it actually has some pretty hefty differences:

  • Presumption of Common Objectives — we used a uniform set of criteria to measure the ads, which, by definition, means that we had to assume the ads were all, basically, trying to reach the same consumers and deliver the same results. Or, to be more accurate, we used a uniform set of criteria and then made some assumptions about the brand to inform how an ad and it’s digital integration was judged. That’s a little backwards from how a marketer would normally measure a campaign’s performance.
  • Over 30 Brands — the sheer volume of brands that advertise at the Super Bowl introduces a wrinkle. From Teleflora to PepsiMax to Kia to Groupon, the full list was longer than any single brand would normally watch as its “major competitors.”
  • Real-Time Assessment — we determined that we wanted to have our evaluation completed no later than first thing Monday morning. The reality of Marketing, though, is that, even as there is a high degree of immediacy and real-time-ness…successful campaigns actually play out over time.  In this case, though, we had to make a judgment within a few hours of the end of the game itself.
  • No Iterations — I certainly could (and did) do some test data pulls, but I really had no idea what the data was going to look like when The Game actually hit. So, we chose a host of metrics, and I laid out my scorecard with no idea as to how it would turn out once data was plugged in. Normally, I would want to have some time to iterate and adjust exactly what data was included and how it was presented (certainly starting with a well-thought-out plan of what was being included and why, but knowing that I would likely find some not-useful pieces and some additions that were warranted).

It was a challenge, for sure!

The Approach

While the data I provided — the most objective and quantitative of the whole exercise — was not core to the overall scoring…the approach we took was pretty robust (I had little to do with developing the approach — this is me applauding the work of some of my co-workers).

Simply put, we broke the “digital” aspects of the experience into several different buckets, assigned a point person to each of those buckets, and then had that person and his/her team develop a set of heuristics against which they would evaluate each brand that was advertising. That made the process reasonably objective, and it acknowledged that we are far, far, far from having a way to directly and immediately quantify the impact of any campaign. Rather, we recognized that digital is what we do.  Ad Age putting us at No. 4 on their Agency A-List was just further validation of what I already knew — we have some damn talented folk at RI, and their experience-based judgments hold sway.

For my part, I worked with Hayes Davis at TweetReach, Eric Peterson at Twitalyzer, and my mouse and keyboard at Microsoft Excel to set up seven basic measures of a brand’s results on Twitter and in Facebook. For each measure, there were either two or three breakdowns of the measure, so I had a total of 17 specific measures. For each measure, I grouped each brand into one of three buckets: Top performer (green), bottom performers (red), all others (no color). My hope was that I would have a tight scorecard that would support the core teams’ scoring — perhaps causing a second look at a brand or two, but largely lining up with the experts’ assessment. And, this is how things wound up playing out.

The Metrics

The metrics I included on my scorecard came from three different angles with three different intents:

  • Brand mentions on Twitter — these were measures related to the overall reach of the “buzz” generated for each brand during the game; we worked with TweetReach to build out a series of trackers that reported — overall and in 5-minute increments — the number of tweets, overall exposure, and unique contributors
  • Brand Twitter handle — these were measures of whether the brand’s Twitter account saw a change in its effective reach and overall impact, as measured by Twitalyzer; Eric showed me how to set up a page that showed the scores for all of the brands we were tracking, which was nifty for sharing.
  • Facebook page growth — this was a simple measure of the growth of the fans of the brand’s Facebook page

The first set of measures were during-the-game measures, and we normalized them using the total number of seconds of advertising that the brands ran. The latter two sets of measures we assessed based on a pre-game baseline. We used Monday, 1/31/2011, as our baseline date. Immediately following the game, there was a lot of manual data refreshing — of Facebook pages and of Twitalyzer — followed by a lot of data entry.

As it turned out, many of the brands came up short when it came to integrating with their social media presence, which made for a pretty mixed bag of unimpressive results for the latter two categories above. Sure, BMW drove a big growth in fans of their page, but they did so by forcing fans to like the page to get to the content, which seems almost like having a registration form on the home page of a web site in order to access any content.

The Results

In the end, I had a “Christmas Tree” one-pager: for each metric, the top 25% of the brands were highlighted in green and the bottom 25% were highlighted in red. I’m not generally a fan of these sorts of scorecards as an operational tool, but, to get a visual cue as to which brands generally performed well as opposed to those that generally performed poorly, it worked. It also “worked” in that there were no hands-down, across-the-board winners.

What Else?

In addition to an overall scoring, we captured the raw TweetReach data and have started to look at it broken down into 5-minute increments to see which specific spots drove more/less social media conversations:

THAT analysis, though, is for another time!

General

Should Google Offer a Paid Version of Google Analytics?

Recently there has been some rumor buzz about Google releasing a “paid” version of Google Analytics (beyond what is currently available through Urchin). Assuming, for a second, that something like this is coming in the future, the real question is whether this is a good or bad idea. In this post, I’ll examine some of the pros and cons to this potential move by Google.

Why Google Should Offer a Paid Version

So what are some of the reasons that Google should offer a paid version of its web analytics offering? I can think of the following:

  • There will always be a group of web analytics users that want advanced functionality and are willing to pay for it. These advanced features are often resource-intensive and I could see Google wanting to recoup some money to enable these features or the additional data storage they necessitate.
  • There are millions of websites using Google Analytics for free and if Google can extract even a small amount of revenue from these, it can add up quickly. Since I don’t think Google is hurting for revenue, I assume that the money generated would be filtered back into the product which would mean even more enhancements to a product that pretty robust already.
  • One of the reasons Google may be thinking about offering a paid version of the product is to open the door to its sales team to cross-sell other Google products and services. By being free, Google Analytics has infiltrated millions of websites which creates an easy entrée for a Google sales rep to say: “I see that you are using Google Analytics, did you know that Google also offers Google Ad Words, Google Apps, etc…” While they can already do this, if a company has already started paying for Google Analytics (and it has made it through procurement!), that makes the cross-sell so much easier. It also helps weed out the companies that are serious, which will often be the ones willing to pay.
  • Services baby! It is no secret that professional services are a huge money maker. When I was at Omniture, we had a sizable consulting group and there are a host of other firms (including Analytics Demystified of course!) offering services around web analytics. While I am not sure if it would be a good move or not, Google could offer paid-for services around a paid-for web analytics tool itself or through its certified partners.
  • Competition! I love competition. I think it helps drive innovation. In my opinion, the consolidation of the web analytics industry over the last few years has reduced the amount of innovation and I think Google having a paid product will ultimately mean that everyone in the industry gets more.

Why Google Should Be Careful About Offering a Paid Version

So what are the pitfalls that Google might want to look out for? Here are a few worth considering:

  • Too much functionality! One of the strengths of Google Analytics is its simplicity. Since it is a free tool for most users, it has not been beholden to the axiom that more features must always be added to continue justifying the investment. Like all software products, as time goes by, more features are added to meet the needs of the most advanced users, which often results in casual users leveraging 10% of the functionality. While it looks like Phil & Nick have done a great job adding the features their users want to date, once someone is paying you money, the balance of power tends to shift in a big way (think difference between privately held vs. publicly traded company). I hope that Google will not lose its simplicity “mojo” that got it to where it is today.
  • Customer Support? One of the biggest expenses for software products is the cost associated with supporting its customers. When I worked at Omniture, we had a massive customer support organization of account managers and client care that grew exponentially. If Google has paid clients, I would imagine that it would need to provide support at a level that far exceeds what it is offering today. This is not an easy task and Google is known for being somewhat hands off for most of its products. When your product is free, people accept that they are going to be on their own more than when they are paying for something and if support isn’t good, I could see Google Analytics losing a bit of its current luster. I also imagine that Google loses quite a bit of money on Google Analytics (which I assume it makes up for on the AdWords side), and this will be even worse once it has to staff up to support users unless it can find a way to get its partners to offer that support.
  • SLA’s (Service Level Agreement). Paid-for vendors have legal requirements around the availability of the product and the handling of product issues. To date, it is my understanding that Google Analytics has not had SLA’s since it is a free product, but I would imagine Google would need to provide a reasonable SLA for the paid side. SLA’s are never fun and usually end up costing time and money…
  • What happens if no one buys it? Google has done a lot of things that have changed the market and some that have not done quite as well (i.e. Google Wave). Google shook up the web analytics industry in a huge way with free Google Analytics, but what would it say if only a small % of companies decide to pay for its product? Does this serve as a boost to its paid competitors? I guess the real question comes down to this. If I am a Fortune 500 company and am currently using Google Analytics and a paid product from Omniture, Webtrends, Coremetrics or Unica (which is very often the case!), what features will Google Analytics add to its paid product that will get me to only use Google Analytics and get rid of my other paid vendor? I would guess that the things I would be looking for are 1) my own dedicated servers so I know my data is really my data and can be kept as long as I want, 2) knowledge that Google is not seeing any of my data and using it in its search algorithms, 3) support and SLA’s at the same caliber I am getting from my other paid vendors and 4) 90% of the features I can get from my other paid vendor. If Google can deliver on these items (and I am sure it can), I think it will make a compelling case as to why companies should standardize on Google Analytics, but I don’t think this will be something that happens overnight.

Obviously, all of this is still speculation, but I, for one, look forward to seeing what Google does and how they address some of the items I have described here.

I highly recommend you check out this YouTube video on disruptive innovation. I think it is very cool to watch this and think about Google being the “entrant” and the other paid web analytics vendors as being the “incumbents” described in the video. This video talks about what Google has done to the other paid vendors and how Google could one day become the incumbent and fall prey to even newer entrants (or reincarnations of the old incumbents!). Fascinating stuff!

So what do you think? Will they do it? Will people buy it? What things do you think Google needs to do to make it successful? Please share your thoughts by adding a comment here…

Analytics Strategy, Conferences/Community, General

A few thoughts on the upcoming WAA Awards

I got a nice note this morning from Mike Levin at the Web Analytics Association:

“CONGRATULATIONS! You have been nominated for a WAA Award of Excellence in the category of: Most Influential Industry Contributor (individual) Your nomination recognizes the contributions you and/or your company have made to the web analytics industry. It is an honor to be nominated and the WAA congratulates you on your success. “

While I am honored by the recognition and delighted to have been nominated I told Mike that I am declining to participate in the voting.

Mike wrote me back and seemed surprised but my thinking is very simple: I have been very fortunate in my web analytics career and have received lots of recognition from my peers, my clients, and the press. I’m not one to bang my own drum and brag about my accomplishments … I prefer to just do my thing, help my clients and the community, and build a strong company for my partners and associates.

So I humbly and politely decline the honor and instead will cast my vote for folks I believe to be truly deserving of an industry honor. Here are the people I will be voting for:

  • Web Analytics Rising Star: Jason Thompson.  Jason is still a bit rough around the edges but I love his style and commitment to getting things done.  If I can vote twice I am voting for Michele “Jojoba” Hinojosa … her passion is palpable and her enthusiasm is infectious.
  • Most Influential Industry Contributor: John Lovett. I’m not sure John is actually eligible because he is on the WAA Board but his work on the WAA Code of Ethics is a monumental achievement and one that has the potential to shape our industry for years to come.  If I can vote twice my second nod goes to Jim Sterne … who has done more for this industry than Jim Sterne?  Damn right, nobody!
  • Most Influential Vendor: Google.  Most of the positive changes we have seen in the past two years in web analytics can be derived either directly or indirectly to the work that Brett Crosby and the team at Google Analytics put out there.  Second vote goes to Omniture given the critical mass they have been able to create and the big strides they made since the Adobe acquisition on customer support and overall focus.

UPDATE: OMG I didn’t realize that Corry Prohens was running a shameless and ruthless campaign to win the “Influential Agency/Vendor” award.  You should read his “shameless campaign” blog post and consider voting for Corry.

  • Client/Practitioner of the Year: Best Buy. Difficult to not vote for one of your own favorite clients but I hope you will all come to my keynote presentation with Lynn Lanphier at Emetrics and hear why I cast this vote.  Second vote? Dell, for taking the advice I gave them last year to heart and who are now kicking ass and taking names for testing and optimization. Bravo!
  • Technology of the Year: Analysis Exchange. Now, of course, I’m not really going to vote for something I helped create, but I am pretty damn proud of the work we have done and with Wendy Greco at the helm things are only getting better.  If I could vote twice … I wouldn’t, because I’d be tempted to vote for Twitalyzer LOL!

Again, I do appreciate the nod from the WAA and am looking forward to the party — the Analytics Demystified and Keystone Solutions crews will be there in force. I wish everyone nominated for the WAA awards the best of luck and, as a native of Chicago, remember to vote early and vote often!

Don’t forget to nominate your favorite web analytics superstar!

Adobe Analytics, Analytics Strategy, Conferences/Community, General

Conference Season is Upon Us

Wow, I just got done looking more closely at the Analytics Demystified team calendar for the next few months and it is a doozy! Chances are if you live in the U.S. and do any type of digital measurement, analysis, or optimization professionally we are going to see you between now and the end of March.

If that is the case, we’d like to buy you a drink!

Despite each of us presenting, often multiple times, we are always happy to make time for our clients and potential clients when we are out-and-about.  If you realize you’re going to be at one of the following events why not drop us a line and we’ll see if we can connect. Who knows, maybe we’re planning a great party or something …

After all that the three of us are going to slink home to our loved ones and try and convince them we are in fact their fathers, husbands, and sons.

Seriously, though, we never get enough opportunities to meet with partners, friends, and prospects at these events so if you’d like to meet with any or all of us please drop us a line sooner than later so that we can block time and make plans.

Reporting

How Marketing is Like Homelessness

I’ve officially succumbed to the Blog an Intriguing Title Syndrome (BITS). My payback for that, I suppose, is that I’ve blown the SEO power of my <h2> tags, my <title> tag, and keywords in the URL such that almost certainly no one who would actually be interested in this post will find it via Google or Bing. So it goes.

But, the title isn’t a pure gimmick. It’s the outgrowth of one of those, “I bet I’m the only person sitting in this church hall with 50+ other people at 3:30 AM who is having this thought,” moments. We all have those moments occasionally, right? Right?!

Gilligan, Are You Drunk? WHAT Are You Talking About?

The basic thought: Marketing is like homelessness, in that they face similar challenges when it comes to measurement.

Earlier this week, I participated in an annual homelessness count in downtown Columbus coordinated by the Community Shelter Board (CSB), which is an organization that drives coordination, collaboration, and consistency across the various homeless shelters in the area. It’s been nationally recognized as a model for how communities can efficiently and effectively meet the basic needs of the homeless. As it turns out, they’re also an organization that does a great job of measurement (which, I now realize, I’ve discussed before).

One of the questions that CSB tries to answer for the community is, “Are we reducing the number of people who are homeless over time?” It turns out that that is a pretty tough question to answer. CSB can certainly track how many beds are filled each night in the various shelters they work with, but those shelters tend to pretty much run at capacity and find creative ways to adjust their capacity as needed so they seldom turn people away. And, the weather affects how many people seek shelter on any given night. So, it’s messy to measure the true change in overall homelessness. That’s sort of like measuring marketing.

Whopper of a Disclaimer: I’m going to spend the rest of this post comparing measuring homelessness to measuring marketing. I’m pretty passionate about both, but the latter pays the bills, while the former actually has a degree of Noble Purpose attached to it. I am in no way comparing the the marketing profession to the group of underpaid and overworked people whose careers are dedicated to reducing homelessness. There is simply no contest there.

Marketing and Homelessness, Huh?

The way that we measure marketing at the highest level is often by measuring revenue, profitability, brand awareness, brand affinity, etc. These are all messy to measure in one way or another, and some of them are expensive to measure, too! It turns out, measuring homelessness is the same way.

So, there I was at 3:15 AM in a church hall waiting for all of the volunteers to arrive. Each team in the hall was made up of 5-6 people, and each team was assigned a different area of the city to physically walk around counting the homeless in that area. It’s not that it takes 5-6 people to do the counting, but there is safety in numbers. I was pretty much just one of “the numbers.” My team’s leader was Dave S., who I’ve known for several years, and whose team I explicitly asked to be assigned to. I mean, if your pre-count pep talk includes flashlight under the chin, how can you not be inspired?

Scary Leader Dave

The Outcome Alone Isn’t All That Helpful

So, we headed out and did our counting. For our group, our total homeless count was: zero. Does that mean that we’re solving homelessness in Columbus? Of course not. That might be the case (we certainly hope it is), but we were only providing one input to an overall count that included the other teams, a shelter census, and self-reported “homeless-but-not-somewhere-you-could-count-me” (i.e., a car) data. And, there was a lot of construction under way in our area, which doesn’t make for conducive overnight outdoor stays, so we were not all that surprised with what we found (two of the members of our team had covered the same area last year, and they did count a handful of people).

Marketing Analog: It’s messy to measure overall marketing outcomes, and it’s almost always impossible to draw a meaningful conclusion from a single data set. In the world of digital and social media, we don’t want to go crazy and try to assess an unorganized and overwhelming sea of data, but we do want to deliberately plan and measure using different tools and sources as appropriate to get as clear a picture as possible of a messy world.

Regardless of how the final tally turns out on the homeless count, having a solid annual measure of the key outcome we’re hoping to change is just the frame around a rather intricate and involved picture. Homelessness, like marketing results, are impacted by myriad  underlying factors. The most commonly recognized causes of homelessness are:

  • The economy — when a local economy is down, there is less prosperity, and the “barely keeping our heads above water” populace become the “drowning” populace
  • The availability of jobs with a living wage — related to the economy, but includes issues such as job skills and quality of the local public education system
  • Mental illness — without access to mental health services and medications, it can be impossible for many people to maintain a stable life
  • Drug and alcohol abuse — often, this goes hand in hand with mental illness, but, even when it doesn’t, once an addiction has set in, wheels can rapidly fall off the steady income wagon
  • Personal catastrophe — a health crisis of the individual or a family member often wrecks limited savings and can draw a person away from his/her job, which triggers a spiral that, ultimately, ends on the streets

It’s a daunting challenge to address all of these, and it’s even more daunting to try to disentangle which of these issues are interrelated and to what degree — both for an individual and at a macro level.

Marketing Analog: Trying to tease apart how economic factors, cultural trends, competitor activity, TV advertising, print advertising, radio advertising, web site content, SEO and SEM, Facebook, and Twitter all interact with each other to affect marketing outcomes is daunting and messy. The fact is, we need to effectively use multiple channels, and we need to identify cross-channel effects and measure those as best as we can. But, it’s not easy, and it’s definitely not perfect.

I’m starting to feel a little silly making this comparison, but, having volunteered on a number of “basic needs” (the lower levels of Maslow’s Hierarchy) committees over the years, it’s been interesting to watch how often the question gets asked: “What’s the single root cause that we can address to have the biggest impact?” The answer? There isn’t one. It’s got to be a multi-faceted approach. And, it’s also a fact that no one person or group can address all of the facets at once.

As it happens, one of the other volunteers on my count team was Matt K., who is the United Way of Central Ohio staff member now responsible for the main United Way committee on which I’ve been a member for the past few years. As we walked along the Scioto River early Tuesday morning, we chatted about how hard it was to identify a clean set of “leading indicators” as to whether we were making progress in our assigned community impact area: emergency food, shelter, and financial assistance. I told Matt that he and I were living in similar worlds — we both are supporting people with expectations and desires for easy, accurate, and accessible measures of something that is very complicated and messy!

Planning Is Important

One final point: our assigned area was a mile or two away from the church where we started…and no one had a car that would easily fit six people. Luckily, it was relatively warm (mid-30s), and the back of my truck was relative dry, so four of us piled into the front, while Matt K. and  Joe M. (also of United Way) climbed into the back of my truck:

Matt and Joe Ready to Ride to the Count Area

Now, had I known we would need to transport six people ahead of time, I easily could have swapped vehicles with my wife for the day and brought along a minivan rather than a small truck. But, we didn’t coordinate that up front. We didn’t fully plan for our measurement.

Marketing Analog: Planning does matter. It doesn’t mean that, without planning, you can’t gather some data, but you may not get the data you want, and it may be a little more painful (or at least chilly) to get the data you need.

I guess, as an analyst, I see data challenges all around me. I also have lower expectations for the quality and completeness of data, I’m more comfortable with wildly imperfect proxy measures, and I expect gathering meaningful data to be a messy process.

Whether counting the homeless or counting web site conversions, though, it’s definitely a whole lot more pleasant to do it with fun and interesting people. I’ve been pretty fortunate on that front!

Analytics Strategy

The Web Analyst’s Code of Ethics

The Web Analyst’s Code of Ethics is a reality! This Code represents an industry effort to promote ethical data practices and treat consumer data with the respect and attention it deserves.

I’m writing this on the eve before the official launch announcement of the Web Analyst’s Code of Ethics here at the WAA Symposium in Austin Texas. As you can see in the video above, this effort is the culmination of a ton of hard work by a community of contributors.

Yet, the conversation isn’t a new one. My partner Eric has been writing about the fact that We are our own worst enemy since August and our internal conversations about privacy regulation and public opinion of tracking practices have been going on long before that. The issue received mainstream attention from the Wall Street Journal in their What They Know series, which took a bias view in our opinion. Anything that starts out with the phrase; Marketers are spying on Internet users… is FUD in my opinion.

So, in September of last year we decided to do something about it. I must say that Eric never fails to amaze me in his ability to make things happen, because not 24 hours after our conversation about launching a Code of Ethics, he had one drafted and in my inbox. We decided that the best avenue for getting this code out to the community was to work in conjunction with the WAA, where I am a member of the board. Thus, I shopped it around to my fellow board members and we all agreed that it was something that our industry needed. The issue was brought before the WAA Standards Committee and a sub-committee was formed to hash out the details. And the Code was offered to the community for public comment. After numerous iterations and literally dozens of comments and contributors, we arrived at the final Code you see here.

It’s important to recognize that this Code is a pledge for individuals and not organizations. We created it as such because we know that not every individual will be able to enforce policy within their company, but every individual can inform and educate their peers. Yet, as we state in the pledge itself, “I recognize that we are far stronger as a community…”. And this effort is about a community showing it’s commitment to ethical data collection and utilization practices.

Momentum for this project has been incredible thus far, but our work is far from over. It’s just beginning. Like any good analyst, I’ve created goals and success metrics for the code of ethics that I’ll be tracking and reporting on over time. The video above is the first effort to share a glimpse of the metrics, but ultimately I’m shooting for the following goals:

      1) Gain 1,000 Pledges to the Code of Ethics in 2011

 

      2) Attract mainstream media attention to this community effort within the first 90 days of launch (e.g., recognition by @WhatTheyKnow)

 

    3) Ensure that our collective voice is heard by legislators and policy makers before regulation is forced upon us

Let us know what you think about the Code of Ethics here by leaving comments and joining the conversation. Or simply show your support by pledging to follow the Web Analyst’s Code of Ethics.

Adobe Analytics, Analytics Strategy, General

Free webcast on Tag Management Systems on Jan 25th

Given the considerable buzz in the marketplace regarding Tag Management Systems and vendors like Ensighten, TagMan, and BrightTag I wanted to call your collective attention to a free webcast I am participating in next week on “The Myth of the Universal Tag.” On Tuesday, January 25th at 1:00 PM Pacific time I will presenting with Josh Manion, CEO of Ensighten and Brandon Bunker, Senior Manager of Analytics at Sony, detailing some of the advantages I see in the adoption of a tag management platform.

What’s more, the nice folks at Ensighten have taken the registration form off of my white paper on tag management systems and so everyone is free to read all of my thoughts on Tag Management without prompting a sales call.  How cool is that?

Spread the word:

“The Myth of the Universal Tag” free webcast sponsored by Ensighten
Tuesday, January 25th, 1:00 PM Pacific / 4:00 PM Eastern
Register online now at GoTo Meeting!

Don’t forget to download that free copy of my white paper on tag management systems!

Adobe Analytics, Analytics Strategy, Conferences/Community, General

Want to meet Adam Greco? Go to OMS 2011 in San Diego!

By now I hope you have heard that Adam Greco is joining John and I as a Senior Partner in Analytics Demystified. While his official start date isn’t still for a few weeks he’s already on the road as part of the Demystified team. If you’d like to meet Adam in person and talk with him about the practice he is building there are a few places I just happened to know he will be in the coming months:

  • Adam will be participating in the Web Analytics Association (WAA) Symposium in Austin, Texas on Monday, January 24th. Adam is talking about integrating web analytics and CRM which is core to his practice area given his past work at Salesforce.com and Omniture.
  • Adam will also be presenting at the Online Marketing Summit in San Diego, California on Tuesday, February 8th. He’ll be giving the same presentation on web analytics and CRM, discussing how to move marketing analytics from the server room to the board room.
  • Adam will also be joining me in Minneapolis on Wednesday, February 16th for a special Web Analytics Wednesday sponsored by our good friends at SiteSpect and with the generous help from our friends at Stratigent.  We don’t have the details on the site yet but the event will be downtown Minneapolis and Adam and I will be doing some prognostication and fielding questions from Twin Cities locals.

Adam will also be at Webtrends Engage, Adobe’s Omniture Summit, and the Emetrics Marketing Optimization Summit but we’ll post more on that when additional details emerge.  Suffice to say Adam will be busy in his first few months on the job.

If you haven’t met Adam I would encourage you to head out to one of these events and introduce yourself. Especially if you’re a marketer and are considering the Online Marketing Summit — if you haven’t been to OMS you really need to go.  Every year I am absolutely blown away by the job that Aaron Kahlow and the OMS team do bringing that conference together.  OMS draws amazing speakers, amazing sponsors, and most importantly amazing conference participants and delivers an absolute fire-hose of information.

I’m sincerely bummed that Adam is taking my place at OMS this year — I haven’t actually missed a big OMS event in California ever — but I am confident that the audience will benefit greatly from Adam’s message about CRM integration, his direct experience at Salesforce.com, and his distinct presentation style.

Social Media

Dear Facebook: As an Analyst, It’s Hard to Be Your Friend

Update: More Facebook Insights updates rolling out, and, so far, they are buggy: Facebook Quietly Updates Insights to Show Real-Time Data on Page Posts, Bugs Appear.

Dear Facebook,

I really want to stay your friend, you see, but I’m an analyst. I’m someone who daily gets asked by marketers: “How do I know if my Facebook investment is paying off?” They want to be your friend, too, but you sure don’t make it easy.

For starters, I couldn’t figure out where to send this note. And, honestly, I’ve never been able to figure out how to actually contact you. It seems kinda’ silly — you do a fantastic job of helping people interact with each other, and you do a lot to enable brands to engage with consumers. Yet, you don’t make it very easy for us data types to engage with you. It would be one thing if there were a handful of uber-analysts who had an “in” with you, and if those folk were out there chiming in to the myriad aborted threads of frustrated analysts trying to extract meaningful data from your systems, that would be one thing. But there aren’t.

Alas! This note is destined to be a bit of a pissy rant. You can accuse me of using social media in all the wrong ways, of launching a Festivus-style airing of grievances. All I can say is that I’ve been around long enough to know that I should not post in anger, but, rather, should pen this note and then let my heels cool overnight. I did. Over a couple of nights, actually. And it still seemed like the right thing to do.

You see, I’ve been doing this web analytics thing for a while now. I lived through the maturing of the industry from “counting hits” all the way to “measuring conversion, segmenting traffic, and testing and optimization experiences.” As an industry, we’ve learned a lot on that front, but, Facebook, you seem hell-bent on reinventing the wheel, and, so far, your wheel looks like a square drawn by a drunken monkey. I want to be able to cleanly measure who’s interacting with my brand within Facebook and how they are engaging with my content. I don’t want to know names and e-mail addresses, but I sure would like to know how first-timers with my brand engage with me as compared to long-time fans. I want to be able to segment my fans and analyze their behavior by segment. Each successive Facebook Insights rollout gets a lot of buzz, but that buzz tends to turn out to be a swarm of horseflies…circling a fresh cow pie. And that stinks.

I don’t know if you know this about me, Facebook, but I was a technical writer early in my career. That’s made me a few things: 1) a pretty fast typist, 2) a guy who occasionally does RTFM, and 3) someone who expects formal documentation to be pretty pristine and comprehensive. With that in mind, let me show you what happens when I go to Facebook Insights and click the Export button. This is the box that pops up:

Seperated? SepErated?!! Maybe you don’t have copy editors on your dev team, but that’s just embarrassing, and, it turns out, an indication of deeper issues. I killed several hours trying to get Excel 2010 + PowerPivot + OData working to “sync data with Excel,” and I overcame several hurdles before running into a brick wall a hundred feet from the finish line. One of these days, maybe I’ll fully crack the code and can write a nice guide on how to make that work — it would be a handy capability — but your single page of documentation simply blithely points to dead ends! Never mind that Excel 2010 is far from a mass-adopted tool (and let me point out that all major web analytics vendors, including, Google Analytics, which is every bit as free as Facebook, have Excel integrations that are well-documented and work with multiple versions of Excel). I’m an analyst — not a programmer. That means I’m reasonably technically savvy, I can generate, hack up, and even do a little debugging of VBA and Javascript here and there. But, I’m not really equipped to jump into a poorly documented API to start extracting data. I need a little help, and you simply do not provide it.

Let’s say I just export an Excel file manually, though. At least, finally, you provide daily data for a few more metrics so that I can do some roll-ups and trending. Now, mind you, I still can’t get trended data for individual custom tab traffic without jumping through a painful number of scroll-and-click hoops, and that’s a pretty run-of-the-mill need. But I digress. I’ve exported my Excel file and I’m checking out the Key Metrics tab. I’m not going to even bother to quibble with how on earth you could know what my key metrics are, or the fact that you provide 18 “key” metrics. Let’s put that aside and, instead, just take a close look at the first three columns of data and the metric names and descriptions provided:

  • Daily Active Users — Daily 1 day, 7 day, and 30 day counts of users who have engaged with your Page, viewed your Page, or consumed content generated by your Page (Unique Users)
  • Weekly Active Users — Weekly 1 day, 7 day, and 30 day counts of users who have engaged with your Page, viewed your Page, or consumed content generated by your Page (Unique Users)
  • Monthly Active Users — Monthly 1 day, 7 day, and 30 day counts of users who have engaged with your Page, viewed your Page, or consumed content generated by your Page (Unique Users)

Wha…?!!! Keeping in mind that each of these metrics has a single column of data that has a value for the metric for each date…what the HECK is a “Monthly 1 day count of users?” I guess I can make an assumption that this was just some of the sloppiest bit of documentation ever written (maybe it was those drunken monkeys again?), and that Daily Active Users are, for each day, the number of unique users who “engaged with the Page” (more on that in a minute) on that one day; that Weekly Active Users are, for each day, the number of unique users who engaged with the page over the prior 7 days (so it’s a rolling 7-day count); and that Monthly Active Users, for each day, are the number of unique users who engaged with the Page over the prior 30 days (so it’s a rolling 30-day count).

Unfortunately, that’s not what the definitions say. What the definitions say is…gibberish.

But wait! There’s more! Let’s look at “…or consumed content generated by your Page.” That’s, like, three multi-syllable words put back to back, which, seemingly, indicates a coherent command of the language. Alas! It’s actually a pretty vague statement. Again, I have to make an assumption that this means, “any user who generated an impression by having a status update by the page render in their news feed.” If that’s what it means, then why not say so? And, if that’s what it means, should that really count as an “active” user? Sure, “engaging with your Page” (my assumption being that that is a Like of or a Comment on content from my page) is a sign of “Active,” as is visiting the page itself (“viewed your Page”), but an impression? Hardly. Unfortunately, I can’t carve that out and use a metric definition that makes sense for me.

The vagueness of the documentation points to a larger issue of transparency as to the mechanics of how you capture and report data. With Google Analytics, Sitecatalyst, Coremetrics, Webtrends, Twitalyzer, Localytics, Flurry, and other analytic tools, I can roll up my sleeves, dive into the documentation and the interwebtubes, do a little experimentation, and wind up with a fundamental understanding of how bits and bytes are flying around to capture the data. While these sorts of underlying mechanics aren’t something that the  business users I support need to understand, it’s critical for my ability to translate the business questions they ask into the interpretation of the reporting and analysis I do. If I had a nickel for every time I had to say, “Well, it’s pure guesswork as to how Facebook is actually capturing and counting that (video views is a biggie there),” I’d have a nice chunk o’ change that I could transfer to an offshore account and then buy a little piece of Facebook. I don’t have those nickels, though, so I’ll settle for just having you pull back the covers a bit and share your data capture mechanisms and data model.

That actually leads to a real head-scratcher on some data you don’t provide. Call me crazy if you must, but I actually care if people are spreading my content to their social graph. You know how they do that? Of course you do! You were instrumental in bringing the concept of “Share” into the mainstream! Yet, you provide no native reporting on share volume (much less segmentation of who shares, or any indication of the lifecycle of a share)! I can get basic content Share counts for content that I manage through Vitrue, but I’m not running Vitrue on all of the pages I work with.

Don’t even get me started on the random nonsensical holes in your data — “my page plummeted from hundreds of thousands of fans to zero fans for two days and then mysteriously returned to its pre-plunge levels!” — or the firm commitment you’ve made to have data available within 48 hours <choke!> of the activity occurring. 48 hours? It’s a real-time world, baby, and, even if “real-time” doesn’t truly need to mean absolute zero latency, 48 hours is ridiculous.

Now, a workaround that occurred to me back in late 2009 was to simply give up on Facebook when it came to getting the data I wanted and, instead, to just deploy web analytics code on my pages. But, you made it clear from the get-go that you had no interest in me bringing anyone else into this relationship, even if they could totally offer something that you’re not interested in providing. Javascript only runs in the narrowest of circumstances, your image caching stymies many workarounds to that limitation, and, even when I am successful, you manage to make me feel vaguely dirty about my success, like I’m doing something wrong. I’m not. I just want to understand how people are engaging with my pages!

I could go on and on. Unfortunately, I don’t have time to make this note shorter, and I do apologize for that. I’m going to go hang out with some Twitter data for a bit to calm down. Maybe, while I’m out, you could take a good hard look at the way you’ve been treating me? A few months ago, Brian Clifton predicted that, in order to survive, Webtrends needs to get acquired, and he suggested that Facebook would be be a good suitor. When I initially read that, I thought it was a pretty “out there” idea. I don’t think that any more. You need to get help. You need a friend, and having some seasoned web analysts and web analytics developers sharing their thoughts and ideas with you would really help your and my relationship with each other.

Facebook, as a user, I am your friend. And I’m loyal. You give me a lot. It’s as an analyst that I’m being forced to remain your friend, even though I soooo Unlike how you reciprocate.

Best regards,

Gilligan on Data

General

weThink Podcast — Digital Trends and What They Mean for Marketers

Matthew Santone, Dan Shust, Chris BarcelonaLast fall, I started listening to the weThink podcast (here’s the iTunes link) that is unique in that I personally know all of the people who work on it. They’re some of my co-workers at Resource Interactive, and most of them are part of the RI Lab — our “R&D” wing: Matthew SantoneDan Shust, and Chris “Barce” Barcelona, with Lisa Richardson as the moderator. They’re go-to folk when it comes to what’s hot and happening in the digital and social space, and what those happenings mean for consumers and for marketers. Seeing as how I’m both a consumer AND a marketer, I pick up great info from every episode. Even better, the format and style of these bi-weekly chats are entertaining and engaging.

The most recent episode was a bit longer than usual, but it’s a good sample of the breadth of material they cover.

Predictions for 2011

Lisa asked the guys to complete the statement: “2011 will be The Year of…” and she got a range of responses:

  • Barce: Facebook Credits and the superphone
  • Matthew: data — the year we actually start making sense of and great experiences out of all of the data we’re collecting from consumers
  • Dan: the year of “the internet of things” and the year of Kinect-like technology (using motion to deliver great experiences)

CES 2011 Recap

Dan attended CES, while Matthew and Barce followed the event closely from afar. The highlights they discussed:

  • Tablets — the Motorola Xoom, which runs Google’s Android Honeycomb OS; the RIM Playbook, the Samsung Galaxy Tab, the Razer Switchblade, and all of the questions and issues around how the myriad form factors and applications will evolve (and how marketers and developers will deliver content to such a wide range of devices)
  • Superphones — the Motorola ATRIX made a splash at the show, but the larger discussion was around how a single device would truly become the centerpiece of a consumer’s digital life
  • 3D — 3D experiences are here to stay, but there was some debate as to whether this is really going to be driven more by consumers or more by manufacturers (and not just device manufacturers — Oakley and other sunglasses manufacturers are now introducing 3D glasses). Glassless group viewing may never happen (lenticular displays, even as they evolve, are still reliant on the viewer being in a small sweet spot to get the 3D effect), and what kind of human interaction barriers do 3D glasses introduce that limit the practical application of 3D?
  • Automotive — Audi’s attempts to deploy vehicle-to-vehicle communication such that vehicles can automatically collect data about weather and road conditions and share that information with other vehicles. This, I believe, is one example of “the internet of things” — all sorts of devices floating around the world that have both data collection and network connectivity capabilities
  • Motion — centered around Microsoft as the lead press conference at the event and Steve Ballmer discussing what’s next for the Kinect — controlling both Netflix and Hulu Plus using hand gestures, as well as Kinect-based avatars interacting in a virtual space (the Second Coming of SecondLife, perhaps?). And, the gang discussed how Kinect-like technologies can make for richer and more relevant consumer experiences both in-home and in-store.

The Mac App Store

Apple has now released an app store for the Mac — think iTunes, but for Mac laptops and desktops rather than just for iPhones. This appears to be a harbinger of a future that sounds a little funny: a future where laptops and desktops run apps. But, these are apps in the smartphone/superphone/tablet paradigm, rather than the “heavy overhead installed software applications” that have been a mainstay of computers for years. These apps will have much more of a platform-agnostic and cloud-centric orientation — enabling cross-device usage of an app in a seamless manner. The Chrome Web Store is another example of this shifting paradigm, with the Tweetdeck, Mashable, and Amazon Window Shop apps available there being examples of where it appears this world is heading.

The iPhone on Verizon

The consensus was that the announcement that, as of February 10th, the iPhone will be available on Verizon, rather than solely with AT&T, will be one of the biggest non-news events of the year. While iPhone users are frustrated with the dropped calls they get with AT&T, they’re going to be equally frustrated by the fact that they cannot simultaneously make a phone call and maintain a data connection with their iPhone when they switch to Verizon. AT&T’s 3G service is GSM-based, which allows data and phone service simultaneously…but is prone to call dropping. Verizon’s 3G service is CDMA-based, which is less prone to dropped calls, but which cannot run data and phone at the same time. Both AT&T and Verizon are migrating to the GSM-based 4G LTE technology, so users, presumably, will have similar experiences and similar limitations once that happens.

One way to look at this announcement is that it is a further leveling of the playing field for a 2-horse race </mixedmetaphor> between the iPhone and Android-based phones: Windows Mobile 7 is awesome, but it’s wayyyyy too late to the game, and RIM just can’t seem to get out of its own way.

Picks of the Week

  • Barce: personal hotspots coming to all iPhones in March (Verizon and AT&T)
  • Matthew: over the holidays, he purchased and installed a Filtrete WiFi Enabled Programmable Thermostat — controllable via an iPhone app or an web interface — and is loving it
  • Dan: the new eBay Fashion iPhone app — a very cool augmented reality app whereby you put your eyes between a couple of markers and you can then “try on” sunglasses

Pretty cool stuff. If you listen to podcasts, it’s worth subscribing (iTunes link)!

Analytics Strategy, Conferences/Community, General

Big Changes at Analytics Demystified

I suspect by now many of you have noticed but this week we made two pretty amazing announcements here at Analytics Demystified. Now that the dust is settling I have some time to take a step back and offer up some comments on the announcements and what I believe they mean for our clients, our prospects, and the web analytics industry in general.

On Tuesday we announced that respected industry veteran Adam Greco had joined John and I as a Senior Partner. Adam is well-known to many in our community thanks to his high-visibility work during his tenure at Omniture, his popular “Omni-Man” blog, and his fine, fine work on the Beyond Web Analytics podcast series.

For John and I bringing Adam on board was a no-brainer. The guy is as bright as they come, he is articulate, and most importantly he knows how to squeeze every last drop of value out of the most widely deployed digital measurement solutions in use today — Adobe SiteCatayst and Google Analytics. Adam is committed to extending that expertise to all of the popular platforms as quickly as possible, and our hope is that by mid-year he will be providing the same great insights he has for SiteCatalyst to Webtrends, Unica, Coremetrics, Nedstat, and other customers.

Adam will be running our Operational Use Audit and Framework Development practice as well as providing custom training and generally supporting the rest of the Demystified service offerings.  Which brings me to our second announcement …

On Wednesday we announced an exclusive partnership with tactical and technical consulting practice leaders Keystone Solutions. Keystone is a slightly better-kept secret than Adam Greco, although their current clients certainly know who they are. Founded years ago by former Omniture super-star Matthew Gellis, Keystone has grown into a talent magnet comprable to, well, Analytics Demystified.  Matt Wright from HP, Kurt Slater from Expedia, Rudi Schumpert from Ariba, and a host of other amazing analytics technicians.

We have doubled-down with Keystone for one simple reason: in our experience they are the best of the best when it comes to providing fundamental and foundational support for any digital measurement practice. Especially against those same two “most popular” solutions — Google Analytics and Adobe SiteCatalyst — Keystone delivers in a way that few others out there are capable, and that is the kind of talent we prefer to work with in the field.

Through this partnership Analytics Demystified clients will be able to benefit from a dramatically expanded set of web analytics consulting service offerings ranging from on-the-ground implementation support to ongoing reporting and analysis to some pretty amazing custom solutions. They will also be taking the lead on our Tag Management Systems Audit and Deployment practice, an offering I expect to be red-hot in 2011 and beyond.

Now, unfortunate as it is, we were not able to pursue this type of relationship with Keystone without some cost. The immediate fall-out is that Analytics Demystified will no longer be participating in the X Change conference. While this breaks my heart after having put three years of sweat equity into the event, relationships change and so it is time to move on.

I do, however, promise every one of the hundreds of consultants, vendors, and practitioners we have personally invited to this conference over the past three years that we will be back, live and in-person, with something far more “Demystified” in nature. Based on our work with Web Analytics Wednesday, the Analysis Exchange, and hundreds of other events around the globe, we have a pretty good idea of what is truly missing from the web analytics event landscape … and now, thanks to Adam and the team at Keystone, we have the means to deliver.

I welcome your comments and questions about both pieces of news, and I hope you’ll keep your eyes open in the coming few weeks for even more news from our growing company. It is exciting times, indeed.

Reporting, Social Media

The Future of Advertising Is Clear — Measurement, not So Much

Fast Company published a lengthy article last November titled The Future of Advertising, and it’s a good read. It traces the evolution of the advertising industry over the past 50 years, and it does a great job of assessing the business model(s) that have worked over time and why. That all serves as a backdrop for how the author posits digital and social media, and the crowdsourced-fragmented-wiki world we now live in is blowing those models up. And, it highlights a number of examples of agencies that are successfully shaking up the ways they operate. It’s a great read.

As I read through the article, I was eager to see what, if anything, came up regarding measurement and analytics. The sole mention turned up on the fourth page of the article (bold/underline added by me):

Every CEO in the [advertising agency] business…wants to be financially rewarded for performance, and thanks to all those new data-analytics tools, for the first time ever, their effectiveness can be measured. Says IPG chairman [Michael] Roth: “We should get higher [compensation] if it works and lower if it doesn’t. That’s how this industry can return to the profitability level.” It’s a nice thought, but those tools aren’t infallible: While Wieden’s innovative Web campaign for P&G’s Old Spice garnered tons of publicity, Ad Age speculated that the boost in sales may well have been due to a coupon.

So much for the silver bullet.

First off, the “every CEO in the business wants” statement is a little odd. Unquestionably, every CFO would love to be able to pay for performance, both the leaders in the company’s own marketing organization, as well as every agency with whom the company works. And, sure, every agency executive would agree that it is fair and reasonable to be paid based on performance. But, I don’t exactly think the advertising industry is flush with agencies wishing they could have performance-based compensation. Sure, agencies want to be able to measure the business impact of their work, but that’s so they can demonstrate their value to their clients, so, in turn, they can retain and grow those clients.

Just as my dander was good and raised, I hit the second sentence that I bolded and underlined above: “It’s a nice thought, but those tools aren’t infallible.” <whew> The voice of reason. But, the “new data-analytics tools” wording implies that there is some whole new class of business impact measurement platforms, and there simply is not. There are scads of emerging tools for measuring new channels like blogs and Facebook and Twitter, and there are lots of really smart people trying to build models that can supplement or supplant the broken reality of marketing mix modeling. But, we’re far, far, far from simply having “tools that aren’t infallible.”

Finally, the snippet above brings up the Old Spice campaign that featured Isaiah Mustafa in an eye-popping number of clever and consumer-engaging videos. No rational marketer would look at that campaign and try to judge it solely based on near-term sales. Word-of-mouth impact, consumers talking positively about the brand, existing customers quietly puffing out their chests because “their” brand is making a splash. How can that not lead to increased awareness of the brand, a positive shift in brand perception, and, I would think, 12-24 months of lingering positive effects? Is all of that worth $100 million or $1 million? I don’t know. But, from a “results based on what the conceivers of the campaign hoped to achieve,” it’s hard to argue that it delivered. But, I’m really not going to continue that debate — just want to point out that “immediate sales impact” is, well, the same sort of old school thinking that the rest of the article takes to task.

I still liked the article, but the brief measurement nod was a bit bizarre.

General

Adam Greco, Demystified

I am extremely excited to begin this next chapter in my career and wanted to get started at Analytics Demystified by describing my past, the present, and what I hope to do in the future.

The Past

I began my career in consulting working for Arthur Andersen’s technology group, first in the field of Customer Relationship Management and eventually in Marketing where I ended up managing the website of the Chicago Mercantile Exchange. It was there, as one of Omniture’s first customers, that I began to learn about web analytics and the data behind websites.

For some reason web analytics came naturally to me, and I loved figuring out fun, new ways to use the technology at my disposal to answer interesting business questions. As I had done in my consulting days, I dug into every Omniture training manual available and soon found that I knew the technology almost as well as those at Omniture. I think that is what led me to ultimately apply to work at Omniture in their newly founded Omniture Consulting group.

While at Omniture, I had the pleasure of working on many different clients, large and small, and helping them get the most out of Omniture products. As I showed Omniture clients how to use the technology, I often heard clients say “I had no idea you could do that…” so I decided to start a blog to teach people what they really ought to know.

Fast forward a few years and I decided that I wanted to go back to the “practitioner” side of the house and was given a great opportunity to head up web analytics at Salesforce.com. In the role of Senior Director for Web Analytics over the past two years at Salesforce I have had a great time reviving the web analytics program. More importantly, thanks to the generosity of the Salesforce organization, I have been able to continue writing about Omniture technology, speaking at industry events, and helping to establish and grow the Beyond Web Analytics podcast series.

The Present

While working at Salesforce.com has been one of the highlights of my career, when you are used to working on ten or more web analytics projects at a time as a consultant, sometimes working on just one for a few years in a row can be tough. Plus, despite being fully employed these past two years, I have been constantly approached by great people asking how to get the most out of their web analytics efforts — requests that I more often than not had to turn down.

Having been in consulting for most of my career, I had to face the inevitable truth; I have always been destined to become a consultant again … to work with dozens of companies at a time, sharing my knowledge of Omniture with companies large and small working to maximize their investment in web and digital analytics.

The Future

Once I decided that I wanted to go back into web analytics consulting, the difficult part was figuring out which organization would be the best fit for me. There are so many great web analytics consultancies, and over the years I have become friendly with many of the leaders of these firms. However, in my mind, Analytics Demystified was my first choice because of the brand that they have built, the caliber of their clients, and the overall thought-leadership their principals have displayed throughout the years.

When I think about the web analytics industry as a whole, I think about the books that helped launch the industry, the Yahoo Discussion Forum, the camaraderie found on Twitter, Web Analytics Wednesdays and most recently, the Analysis Exchange, and the new Web Analysts Code of Ethics. The common theme in all of these pillars is Eric Peterson and the Analytics Demystified brand he has created.

It was only last year when I first met Eric face-to-face but I have always been impressed by his ability to understand where our industry has been and, more importantly, where it is going. When John Lovett joined the organization, I enjoyed reading about all that he was doing in helping clients with their strategy, vendor evaluations, and social media efforts. In the last year, I got to spend some time with John and have been equally impressed with his passion for the industry and work he has done as a board member for the WAA.

I know there are any number of organizations that I could have joined where I could have helped teach people what I knew and managed teams of consultants. But at Analytics Demystified I believe that I am in a position to be both teacher and also a student, working with two of the smartest, most respected people in the field.

At Analytics Demystified, my hope is to help as many clients as possible get the greatest value from their investments in web analytics technology. I am excited to continue helping clients using Omniture technologies, but also look forward to branching out into other vendor tools and help their clients as well. It is my hope that the combination of Eric and John’s strategic work with my design and architecture background will have a synergistic effect, helping Analytics Demystified clients to further achieve their digital business objectives.

I look forward to hearing from all of you here in my new blog, and by all means if I can help your business, please don’t hesitate to contact me.

Adobe Analytics, General

Form Submit Button Clicks

At the end of last year, I spent a bunch of time showing how you could dissect your website forms to see which were performing well and not so well. While this post will be different from those, it is still related to website forms. In this post, I am going to share a concept that will let you determine which of those visitors seeing your forms have the intention to complete them and which do not. This information can be very valuable as I hope to show.

Which Forms Get Visitors to Take Action?
If you have forms on your website, I hope that you are at least doing the basics and tracking how many people View each Form and how many Complete each Form like this:

This will allow you to have a rudimentary view about how each website form is performing. However, one short-coming of this is that you only have two points of comparison. As a web analyst, I always like to have more data points to slice, dice and analyze. The report above answers the question: “How many people who see each form decide to complete it?” What if you wanted to know how many people who see each form try to complete it? That might be an interesting data point, since sometimes when you do a lot of Paid Search or Display Advertising you could be driving less qualified traffic to your website. Therefore, what I like to do is to create a new metric that I call Form [Submit] Button Clicks. This Success Event is set when website visitors click the button that you place on your form (duh!). By doing this, you have essentially created a wedge between the Form Views and Form Completes metrics shown above such that you can create a report that looks like this:

As you can see here, in the first report above we knew that only 786 of the 2,246 Form Views turned into Form Completions. However, with the second report, we now know that visitors to that specific form clicked the Form Submit button 830 times. That means that 44 times they tried to complete the Form, but were unable to for one reason or another (maybe Form Errors).

Dig Deeper With Calculated Metrics
Once you have this cool new Form Button Clicks metric, you can then create some fun new Calculated Metrics that let you dig even deeper. Here are two that I suggest: Form Button Click Rate & Form Button Click Fail Rate. The Form Button Click Rate is the number of Form Button Clicks divided by the number of Form Views. This metric shows you what percent of people viewing the Form actually click the button as shown here:

In this report you can see which forms on your website are doing a good job at getting visitors to click the button. Forms with low percentages might indicate that there are too many fields, poor content or a bad offer. You can use this report to zero in on which forms represent the biggest opportunity for improvement. I like to bubble-chart this data such that the forms with the most Form Views and the lowest Button Click Rate move to the “magic quadrant.”

The next Calculated Metric is the Form Button Click Fail Rate. This represents the percentage of times visitors click the Form Submit button, but fail to have a Form Complete. These people represent your “lowest hanging fruit” as by clicking the button, they have implicitly told you they are somewhat interested in you! You create this metric by dividing the difference between Form Button Clicks and Form Completes by the number of Form Button Clicks as shown here:

In this case, for the first form, about 5% of people who click the button don’t make it to a Form Complete, but the last form shown in the report seems to have some issues since 62% of Form Button Clicks don’t make it to a Form Complete. You may want to start doing some testing on that form!

As is always the case, whenever you create new Calculated Metrics you can see them as general metrics in addition to using them in eVar reports. Therefore you can set Alerts and see trends for both of the metrics described above:

What I like about these two metrics is that one shows you how good you are at getting people to click the button on the form (how good your offer/content is) and the other tells you how good you are at closing the deal once a visitor has decided to give you a chance. Those who have managed websites realize that there are very different tactics used to solve these two very different questions so having these metrics can really help you focus and use your precious website resources as efficiently as possible.

Don’t Forget Your Other Reports!
While the above reports hopefully get you excited, don’t forget that you already have many reports that can be combined with the information above to get even more value. For example, one of the reports I use a lot is the Traffic Driver (Unified Sources) report which shows me how each visitor got to my website. Wouldn’t it be cool if I could see Form Button Clicks and the above two new Calculated Metrics by Traffic Source? Well…you can! All you have to do is add these metrics to your existing Traffic Sources report like this:

Now you can see how each channel is doing! Looks like Paid Search (SEM) is generating lots of Form Views, but only gets 12% of these to turn into Form Button Clicks. If they do get someone to click the button, it looks like 55% of them don’t end up successfully making it to a Form Complete. This can be contrasted by SEO which seems to fare a bit better by getting 30% of its Form Viewers to click the button and of those 75% make it through to Form Completion. You can imagine how powerful this data could be and how you could use a product like Test&Target to come up with ways to improve these conversion rates by traffic source.

If you want to get even more granular, you can break this report down by the root traffic driver so you can take specific actions. In the following report, I can see the Paid Search ID’s that make up the Form Views and the other metrics and see how each performs individually:

Here we can see that there are some Paid Search keywords that are doing well (get people to click on the submit button over 20% of the time) and others that are under-performing (less than 15%). You can use these metrics to help drive your Paid Search strategy or possible automate this using SearchCenter. Finally, in this fictitious example, I have made row three have zero Form Completes, but a 32% Form Button Click Rate, which would indicate a major issue with the form that should be addressed.

One last example of leveraging an existing report would be the Visit Number report:

Here we can see that the Form Button Click Rate is pretty consistent, but up a bit in the 3rd visit, but interestingly, our Form Button Click Fail Rate appears to decrease over time. Perhaps the more time visitors take to get to know us, the more likely they are willing to deal with all of the information we are asking for on our forms!

Final Thoughts
Well there you have it. I always find it so amazing that adding one simple Success Event in the right place can open up so many new web analysis opportunities. If you have forms on your website, I hope this will help you learn more about your users and how they are interacting with your forms. Let me know if you have any questions…

Analytics Strategy, General, Social Media

It's not about you, it's about the community …

Happy New Years my readers! I hope the recent holidays treated you well regardless of your faith, persuasion, or geographic location. I wanted to take a quick break from all the heavy privacy chatter these past few months and tell a little story about the generosity of our community and one individual in particular.

If you follow me on Twitter you may have noticed me cryptically tweeting “it’s not about you, it’s about the community” from time to time. I started sending this update as a subtle hint to a few folks who harp on and on about their accomplishments, products, and “research” in the Twitter #measure community … but sadly those folks never got the hint (so much for being subtle, huh?)

Over time the tweet became something larger — it became a reminder about what we all are capable of when we think about more than our own little world.  “It’s not about you, it’s about the community” is about some of the greatest contributors in the history of web analytics, people like:

  • Jim Sterne, who years ago realized that we needed a place to gather, and who wisely picked the Four Seasons Biltmore in Santa Barbara, California.  While Emetrics may have become a profit-generating machine, those of you who know Jim and know history understand that the conference is as much about and for the community as it is anything else;
  • Jim Sterne, Bryan Eisenberg, Rand Schulman, Greg Drew, Seth Romanow, and others who founded the Web Analytics Association years ago when it was clear that we needed some type of organizing body, committing themselves to hundreds of hours of work without thinking about how they would make money off of the effort;
  • Jim Sterne (again!!!!) who has been making sure that we all know who is doing what where and when via his “Sterne Measures” email newsletter for as long as I can remember;
  • Avinash Kaushik, Google’s famed Analytics Evangelist, who has long committed the profits from his books on web analytics to two amazing charities;
  • Super-contributors to the Web Analytics Forum at Yahoo Groups, folks like Kevin Rogers, Yu Hui, Jay Tkachuk, and dozen more who still take the time to answer questions from newer members of this rapidly expanding community;
  • Past and current Web Analytics Association Board members and super-volunteers, folks like Alex Yoder, Jim Novo, Raquel Collins, Jim Humphries, and so many more who give their time and energy every month to make sure the Association continues to evolve and grow;
  • Activists and evangelists like my partner John Lovett, who in the midst of writing his first book on social media analytics has taken the time to shepherd our Web Analysts Code of Ethics effort through the Web Analytics Association Board of Directors;
  • Everyone who has ever hosted a Web Analytics Wednesday event, including luminaries like Judah Phillips, June Dershewitz, Tim Wilson, Bob Mitchell, Emer Kirrane, Perti Mertanen, Alex Langshur, Anil Batra, Ruy Carneiro, Dash Lavine, Jenny Du, David Rogers, and way too many more folks to list who contribute their valuable time to help grow organic web analytics communities locally;
  • All of the over 1,000 members of the Analysis Exchange, many of whom have contributed to multiple projects to make sure that nonprofit organizations around the world have access to web analytics insights;
  • Dozens of others I am forgetting, and probably hundreds more I have never even met …

When I think about this list of people and their individual contributions to the web analytics community it is almost overwhelming — how lucky we are to have such considerate and giving friends!  Still, people have been giving back for years and so it is rare that I see something or someone in the community that really blows me away …

Until recently.

Not everyone knows Jason Thompson, and I suspect he would be the first to admit that not everyone who knows him actually likes him, but if I had to pick one “web analytics super-hero” for 2010 Jason would be my hand’s-down, number one choice.  See, Jason was smart enough to not just get the web analytics community to give back to our community, he managed to get our community to help provide clean water to an entire community in a developing nation.

Having worked repeatedly as a volunteer with Analysis Exchange Jason was introduced to charity:water, a nonprofit organization who’s vision is very simple: to provide clean, safe drinking water for everyone on the planet.

Water.

Not a great blog or free books, not data or solution profilers, but water that mothers can bring to their children. Clean, pure water that I would venture each and every one of the members of the web analytics community takes for granted and rarely even considers the source and its availability.

But Jason thought about it, and what’s more, Jason did something about it. Thanks to some cool new technology Jason was able to donate his 36th birthday to help raise $500. By leveraging Twitter and his web analytics community he was able to raise that $500 by December 18th.  Having met his goal before his birthday Jason didn’t stop and settle, he set the bar higher, working first to raise $1,000, then $3,000, and finally $5,000, enough to provide water for an entire village – 80 people for 20 years.

Jason’s effort brought out the best in our community again, collecting donations from luminaries and lay-users alike … hell, he even got money from his mom! Some of the biggest names in web analytics helped Jason along, and donations large and small rolled in right up until Ensighten’s Josh Manion put in the last $300 on Jason’s birthday, putting him over the top and completing his final goal.

Honestly I don’t know Jason very well, but I do know passion and greatness when I see it. Jason once again served as a reminder that “it’s not about you, it’s about the community” and he did more than just tweet obnoxiously … he put his time and money where his mouth is and did something real.

Bravo, Mr. Thompson.  Bravo.

If you don’t know Jason I highly recommend following him in Twitter (@usujason, if you’re into Twitter) and, if you see him at a conference or event do like I will and buy the man a drink. I for one am going to let Jason be an example of how I can work even harder to make a difference both inside and outside of the web analytics community in 2011 and beyond.

Hopefully some of you will do the same.

Analysis, Social Media

If the Data Looks too Amazing to Be True…

I’ve hauled out this same anecdote off and on for the past decade:

Back in the early aughts [I’m not Canadian, but I know a few of ’em], I was the business owner of the web analytics tool for a high tech B2B company. We were running Netgenesis (remember Netgenesis? I still have nightmares), which was a log file analysis tool that generated 100 or so reports each month and published them as static HTML pages. It took a week for all of the reports to process and publish, but, once published, they were available to anyone in the company via a web interface. One of the product marcoms walked past my cubicle one day early in the month, then stopped, backed up, and stuck his head in: “Did you see what happened to traffic to <the most visited page on our site other than the home page> last month?” I indicated I had not. We pulled up the appropriate report, and he pointed to a step function in the traffic that had occurred mid-month — traffic had jumped 3X and stayed there for the remainder of the month.

“I made a couple of changes to the meta data on the page earlier in the month. This really shows how critical SEO is! I shared it with the weekly product marketing meeting [which the VP of Marketing attended most weeks].”

I got a sinking feeling in my stomach, told him I wanted to look into it a little bit, and sent him on his way. I then pulled up the ad hoc analysis tool and started doing some digging and quickly discovered that a pretty suspicious-looking user-agent seemed to be driving an enormous amount of traffic. It turned out that Gomez was trying to sell into the company and had just set up their agent to ping that page so they could get some ‘real’ data for an upcoming sales demo. Since it was a logfile-based tool, and since the Gomez user agent wasn’t one that we were filtering out, that traffic looked like normal, human-based traffic. When the traffic from that user-agent was filtered out, the actual overall visits to the page had not shown any perceptible change. I explained this to the product marcom, and he then had to do some backtracking on his claims of a wild SEO success (which he had continued to make in the course of the few hours since we’d first chatted and I’d cautioned him that I was skeptical of the data). The moral of the story: If the data looks too dramatic to be true, it probably is!

This anecdote is an example of The Myth of the Step Function (planned to be covered in more detail in Chapter 10 of the book I’ll likely never get around to writing) — the unrealistic expectation that analytics can regularly deliver deep and powerful insights that lead to immediate and drastic business impact. And, the corollary to that myth is the irrational acceptance of data that shows such a step function.

Any time I do training or a presentation on measurement and analytics, I touch on this topic. In an agency environment, I want our client managers and strategists to be comfortable with web analytics and social media analytics data. I even want them to be comfortable exploring the data on their own, when it makes sense. But, (or, really, it’s more like “BUT“), I implore them that, if they see anything that really surprises them, to seek out an analyst to review the data before sharing it with the client. More often than not, the “surprise” will be a case of one of two things:

  • A misunderstanding of the data
  • A data integrity issue

All of this is to say, I know this stuff. I have had multiple experiences where someone has jumped to a wholly erroneous conclusion when looking at data that they did not understand or that was simply bad data. I’d even go so far as to say it’s one of my Top Five Pieces of Personal Data Wisdom!

And yet…

When I did a quick and simple data pull from an online listening tool last week, I had only the slightest of pauses before jumping to a conclusion that was patently erroneous.

Maybe it’s good to get burned every so often. And, I’m much happier to be burned by a frivolous data analysis shared with the web analytics community than to be burned by a data analysis for a paying client. It’s tedious to do data checks — it’s right up there with proof-reading blog posts! — and it’s human nature to want to race to the top of the roof and start hollering when a truly unexpected result (or a more-dramatically-than-expected affirming result) comes out of an analysis.

For me, though, this was a good reminder that taking a breath, slowing down, and validating the data is an unskippable step.

Analytics Strategy

The Privacy Apogee

The biggest topic that you will grapple with in 2011 is consumer privacy. We are at the most liberal and lenient point of consumer privacy in the history of time. It’s primarily because digital data is spewed by consumers with each click, like, Tweet, share, and update with reckless abandon. Consumers are barely aware of the digital footprints they’re creating and we don’t know how to handle it. There are no rules here.

Consumers are racing to new digital medium at breakneck speeds to be early adopters of the next best thing and are literally addicted to digital. Our obsession is so ravenous that almost half of smartphone users will wake up in the middle of the night to check for digital updates. It’s not their fault really, in fact I include myself in this frantic race to get the newest browser, the latest app, or to connect with nearly anyone who asks. Heck, I downloaded the Owner’s Manual to a Hyundai on my iPad within seconds of watching a TV commercial just because I could. I have no idea what data Hyundai now has on me and if or when I’ll start receiving ads or emails containing must-have offers for a car that I probably won’t ever buy (although it looks sweet!). My point is that we’re on the precipice of a substantive change in the way that consumer data is collected and utilized. If we (and by “we” I mean we digital measurers, organizations and institutions) don’t get our acts together in the first quarter of Q1 then we will have regulation forced upon us.

In my opinion, the number one most critical component for even getting off the ground with privacy protection is education. We must educate consumers, organizations, developers and governments to have a meaningful conversation about privacy. If we fall short of that, ignorance about how data is collected, how it’s used, and who uses it, will continue to be vilified by consumers and media sources that don’t know What they Know.

To that end, I’m working on a concept that I’m calling the Privacy Apogee.

Those of you who are up to speed on your celestial mechanics will know that an apogee reflects the furthest point of orbit from earth. What I seek to explore is the farthest point of ethical data collection from a consumer. My working diagram above depicts your average consumer at the epicenter of privacy and the way we track his digital activities using technology that extends from innocuous to invasive. My plan is to flesh out this concept with current tracking capabilities and potential consumer benefits. Moreover, I intend to create a blueprint for accountability. Ultimately the goal is to produce an infographic that conveys several things:

For consumers

      – The Privacy Apogee will illustrate data tracking capabilities that exist today and highlight some of the benefits of opting-in to these tracking practices.

For developers – It will offer guidance on what methods of data to collect and how to communicate data collection, storage and utilization practices in clear language.

For organizations – The Privacy Apogee will illustrate just how far – is too far – by showing what’s technically possible and what’s morally ethical.

In creating this work, I hope to educate and inform the masses by offering a public service that will open some eyes to the critical imperative for self-regulation before we have governmental mandates forced upon us. The Privacy Apogee will illustrate current technological capabilities for tracking consumers’ digital actions and offer both positive and negative repercussions of those actions.

So back to you Captain Blackbeak…I’m listening and this is what I’m doing to create change. It’s a change in perception. A change in education. And a change in direction for our industry. But like you, I cannot do this alone and need the support and mindshare of our industry. With the help of my partner Eric and the industry #measure pros out there my goal is to crowd source this idea to ensure that I’ve fully considered the technology capabilities and the benefits of tracking practices, So I need your help. The Web Analyst’s Code of Ethics is one part of this, but I’ll be working to define the pros and cons of data collection and the methods by which we accomplish our task. Stay tuned for more, as this is just the beginning…

But in the meantime, what do you think?

Analytics Strategy, Social Media

Is It Just Me, or Are There a Lot of #measure Tweets These Days?

<Standard “good golly I haven’t been blogging with my planned weekly frequency / been busy / try to get back on track in 2011” disclaimer omitted>

Update: This update almost warrants deleting this entire post…but I’m going to leave it up, anyway. See Michele Hinojosa’s link in the comment for a link to an Archivist archive of #measure tweets that goes back to May 2010 and doesn’t show anything like the spike the data below shows, and also shows an average monthly tweet volume of roughly 3X what the November spike below shows. Kevin Hillstrom also created a Twapper Keeper archive back in early November 2010, and the count of tweets in that archive to date looks to be in line with what the Archivist archive is showing. So…wholly invalid data and conclusion below!!!

Corry Prohens’s holiday e-greeting email included a list of hist “best of” for web analytics for 2010, and he really nailed it. That just further validates what all web analysts know: Corry is, indeed “Recruiter Man” for our profession. He’s planning to turn the email into a blog post, so, I’ll sit back and wait for that. But, I did suggest that the #measure hashtag probably deserved some sort of shout out (I actually dubbed #measure my “web analytics superhero-sans-cape” in my interview as part of Emer Kirrane‘s “silly series”).

That got me to thinking: how much, really, has the #measure community grown since it’s formal rollout in late July 2009 via an Eric Peterson blog post?

10 minutes in my handy-dandy online listening platform, and I had a nice plot of messages by month:

Yowza! My immediate speculation is that the jump that started in October was directly related to the Washington, D.C. eMetrics conference in the first week of October — the in-person discussions of social media, combined with the continuing adoption of smartphones, combined with the live tweeting that occurred at the conference itself (non-Twitter users at the conference picking up on how Twitter was being effectively used by their peers). That’s certainly a testable hypothesis…but it’s not one I’m going to test right now (add a comment if you’ve got a competing hypothesis or two — maybe I will dive a little deeper if we get some nice competing theories to try out; this will definitely — the horror! — fall in the “interesting but not actionable” category, so, shhhh!!!, don’t point your business users to this post!).

It’s also possible that the data is not totally valid — gotta love the messiness of social media! I’d love to have someone else do a quick “conversation volume” analysis of #measure tweets to see if similar results crop up. Unfortunately, Twitter doesn’t make that sort of historical data available, I shut off my #measure RSS feed archive a few months ago, and, apparently, no one (myself included) ever set up a TwapperKeeper archive for it. So, I can’t immediately think of an alternative source to use to check the data.

Thoughts? Observations? Harsh criticisms? Comment spammers (I know I can always count on you to chime in, you automated, Akismet-busting robots, you!)?


Analytics Strategy

Santa Puts Aprimo Under the Tree!

2011 is shaping up to be the year of big marketing. And luckily for us measurers, smart marketing is founded in data and measurement. With IBM’s recent acquisition rampage and now Teradata’s plans to buy Aprimo, there is unprecedented choice for integrated enterprise marketing solutions. Teradata announced today it’s intentions to buy the Enterprise Marketing Management leader for $525M with a closing date anticipated for sometime in Q1 2011. It’s a smart move in my opinion because the days of big data management and the ability to harness the consumer data firehose for elevated marketing are upon us.

On the executive briefing this morning, I pointed a question by asking if this acquisition was a response to IBM’s recent buying spree and the answer was a definitive no. Bill Godfrey, Aprimo’s Chief Executive Officer, quickly pointed out that Aprimo’s technology set covers 8 categories and that only one competes directly with the IBM/Unica offering. He iterated, “This is not a copy-cat move” with mild umbrage. Mr. Godfrey went on to eloquently explain that the merger pursues an independent strategy that brings a unified platform covering a very broad end-to-end spectrum of functionality. While the story sounded familiar, it’s a good one. It leverages the database storage and business analytics capabilities of Teradata and layers the marketing management and operations proficiency of Aprimo on top. This enterprise-ready integrated solution fuels a marketers’ paradise where insights are churned from data, which pumps intelligent life into automated marketing. All this happens within a closed-loop system that improves over time. Sounds rosy doesn’t it? To paraphrase Teradata’s CMO Darryl McDonald, “The combined solution will help accelerate revenue generating campaigns and leverage data for strategic insights and quick response.

Keep in mind that this isn’t entirely new territory for Teradata who has been offering marketing products to its customers for some time. With IWI (Integrated Web Intelligence) and TRM (Teradata Relationship Manager), it’s already servicing digital data integration and intelligent marketing to it’s customers. Yet, it will be interesting to see how many existing clients and new organizations adopt this complete functionality. My hunch is that this stack is not for the feint of heart nor the bootstrapped organization. It will work best with deeply integrated datasets, stored within big iron and activated using some complex Marketing Resource Management capabilities. All things that both Teradata and Aprimo excel at. But fair warning: Mom & Pop shops need not apply. However, if you’re a large enterprise looking to accelerate your marketing prowess, then this may be the solution you’ve had on your wish list all these years.

While integrating these technologies may take a while, and the promise of an end-to-end solution is no trivial pledge, I’m bullish on the deal. This is a step forward for marketers because it has the potential to deliver the ERP system they never had. It still doesn’t cover everything, but the combined solution sure does handle some critical moving parts.

Congrats to everyone at Aprimo for building an attractive offering and to Teradata for recognizing it. And Happy Holidays to all!

Adobe Analytics, General

Tracking Form Errors (Part 3)

(Estimated Time to Read this Post = 4 Minutes)

In this series of blog posts, I have been talking about how to see what types of Form Errors your website visitors are receiving so you can improve conversion. So far, we have learned how to see how many Form Errors your website is getting, which fields are causing those and how many Form Errors you get per Form and Visit. As my regular readers know, I like to go beyond the basics, so now we are going to kick it up a notch and get into some real fun stuff. Fasten your seat belts!

Which Fields on Which Forms?
In my first post of this series I shared a simplistic way to learn which form fields caused errors using a List sProp. However, correlating this to specific forms was a bit trickier. Here I will show how to do this, even if you don’t have Discover. The trick here is to set a Form Errors eVar that stores all of the fields which had an error when the above Form Errors Success Event is set. Since eVars have a longer character length, this should be possible for most forms that aren’t too long (which they shouldn’t be anyway!). I like to do this by concatenating the field values into one long string with a separator between each field. Here is an example of the report you want to have:

This report will look a bit like the one I described in the previous post, but as you will see, it is much more powerful since it is in the conversion area and can take advantage of Conversion Subrelations. Besides being able to see which combination of field errors are troubling users, you can open your Form ID reports, find a specific form and then break it down by this new Form Error eVar to see the specific form fields causing problems by form as shown here:

Using this report, we can see that for the first form shown above, 66% of the times visitors get a Form Error, they had eight form field errors (or left them blank). This data, when coupled with observational data using a tool like ClickTale can be invaluable in driving increased form conversions!

What % of Required Form Fields Have Errors?

While the above report, which shows Form Field Errors by Form, is powerful, one question it doesn’t answer is: How many of the required fields on my forms are not being filled out by users? The answer to this question can help you figure out which fields should/shouldn’t be required. So to answer this question, what you want to do is to look at each form that loads on your website and calculate how many fields the user received an error for and then divide that number by the total number of required form fields. For example, if you have a form with eight required fields, and the current user received two errors on that form, the calculation would be 2/8 or 25%. You should then pass this 25% value to an eVar when you are setting the Form Errors Success Event. Once you do this for all forms, you will have a report that looks like the one shown here. Using this report we can see that the highest number of Form Errors are cases where users are getting errors on every field (which is most likely people leaving all fields blank). Maybe our users don’t realize that these fields are required and we can do some testing to create a better experience or reduce the number of required fields?

If we want to see which forms are the ones that have the highest 100% Form Field Error Rate, all we need to do is break the above report down by Form ID:

Finally, if you are doing a good job of grouping your website forms using SAINT Classifications, you can see some super-cool reports. In the following report, I have grouped all of my website forms into high-level buckets of Demo and Free Trial. Then I broke this report down by the percentage of required fields that result in Form Errors.

You can see here that most website visitors on Demo forms are getting errors for 100% of the fields (probably leaving them blank!), while for the Free Trial, the largest percentage of required fields with errors is 10%. Interesting data indeed!

Final Thoughts
In this post, we have covered some advanced ways to see which fields produce errors on each form, see this by form and seen how to know which forms have the highest total required field error rates. These reports can provide an enormous amount of insight into what is happening on your forms with respect to errors and once you understand your visitor’s form behavior, you can apply these learnings to all forms on your site. In my next post, I will cover a tangentially related item (related to Forms, but not as much about Form Errors) that I think is super-cool.

Between this post and the last post, hopefully you have some food for thought when it comes to tracking how your website forms are doing so you improve your conversion rates…

General

Commerce Department and WAA Code of Ethics

Thanks to Tim Evans I was alerted to a report about the Commerce Department weighing in on privacy issues online.  Suffice to say I agree with the direction Commerce is giving the Obama administration.  Specifically the idea that, according to CNN’s Money, “the government ‘enlist the expertise and knowledge of the private sector’ to create ‘voluntary codes of conduct that promote informed consent and safeguard personal information.'”

More or less exactly what John Lovett and I proposed back in September of this year.

I have started reaching out to the media on this point — that we in the digital measurement community are already taking matters into our own hands and stepping up — but we could use your help! Please, if you know anyone in the press, send a link to this blog post along to them and help spread the word that as a community we can take responsibility for our own actions and we are willing to do what is right for consumers around the globe.

This issue affects all of us in the digital measurement sector — vendors, consultants, and practitioners alike. Please help us create awareness about our efforts.

Here are links to the relevant background materials:

Here is the link to the near-final draft of the Web Analysts Code of Ethics:

  • Last chance to shape the Web Analysts Code of Ethics (WAA Blog)

The Standards sub-committee for the Code of Ethics met yesterday and as I publish this blog post John Lovett is presenting the final version to the Web Analytics Association Board of Directors.  We expect the Code to be available to sign at the WAA web site in the coming weeks.

Adobe Analytics

Phew…!

Phew. It’s been crazy weeks for me lately. At the moment, we just put up the tree, kids are all quiet and I’m drinking a glass of red. It’s one of the rare moments these days that I have in solace…and it’s gone…the littlest one is squirmy with hiccups.

Okay, I’m back. Made a bottle and made the hand off to Mommy. I haven’t blogged in a long while and there so much to say but I just haven’t had time. So here’s the johnlovett highlight reel for Fall 2010:

  • We welcomed a new baby into our home. And that makes three. Three boys that is. I always thought the jump from one kid to two was really no problem. But, I can tell you that increasing the number of kids another 33% 50% is a big jump indeed. [The 33% designates the percentage of quantitative reasoning skills I’ve lost in the past month.] Our house is busier than ever with an 18-month old climbing the walls and an eldest brother at five running the show. Everybody is happy and healthy so I’m immensely grateful for the lack of sleep and craziness.
  • I’m writing a book for Wiley on Social Media Metrics. And it’s one of the hardest things I’ve ever done. I’ve got the story in my head and know what I want to write, yet cranking out 40 page chapters every other week is really tough. I’m nearly half way through my manuscript and I love the way its coming together. Although, if you’ve got a social analytics story of smashing success, miserable failure or sheer brilliance, I’d love to talk with you. I could always use more.
  • My business is off-the-charts busy. Looking back on twelve months since joining Demystified and I couldn’t be happier. It’s been a great year and the work I’m doing is motivating me to maintain work-a-holic proportions. Since Labor Day I spent 8 weeks on the road visiting clients, working on changing our industry and speaking at events from coast to coast with a business trip to Italy as a big November finale. I made it home with four days to spare before the baby was born. Whew.
  • And I’m happier than I’ve ever been. Who knew that chaos could be so rewarding? I always knew this was the case, but I love my job and I truly love the #measure industry. As measurers of digital medium, our roles are about to become indispensable. We’re on the precipice of a big data explosion and we’ll have the skills to float to the top. Big data is going to rush like a flood over enterprises and marketers alike and we measurers will be ready to slice and dice our way to sensibility. I like our chances.

More to follow on all these topics as I’m working three concurrent projects, writing two white papers and working through book chapters at present… Oh, and it’s my turn to change diapers, so I’m out.

Talk to y’all soon.
John

Adobe Analytics

Tracking Form Errors (Part 2)

In my last post, I started the process of identifying which form fields were producing the most errors. In this post, I will cover some related topics that will allow you to quantify how often you are getting Form Errors and how effective, in general, your forms are at converting website visitors.

How Many Form Errors Are You Producing?
While the solution I identified in my last post showed which form fields had more errors than others, in the web analytics space, we like hard, concrete numbers! Therefore, I would recommend that you set a Success Event each time website visitors encounter at least one form error (assuming you do validation when the Form Submit button is clicked). By setting a Success Event, you will have a nice chart that shows you the overall trend of Form Errors as shown here:

If you are passing a Name or ID for each form you have on your website, you can also use this Success Event to see which forms are getting the most number of errors like this:

In addition, you can set an Alert for the overall Form Error metric or for a specific Form Name/ID:

 

How Is Each Form Doing?
While knowing how many Errors a form gets is cool, as is often the case, we in the web analytics field care more about ratios! In the report above, it is alarming to see that the first form had 85 Form Errors but how do we know if that is good or bad? If we create a Calculated Metric to compare Form Errors to Form Views, we can see how many Form Errors visitors had in relation to each time the same Form was viewed. Based upon the data below, we can see a wide range of Form Error percentages depending upon the form:


Some of these percentages are quite high and represent amazing opportunities to do testing to see if they can be improved! In addition, when you create a calculated metric, besides just seeing it in an eVar report like the one above, you can also see it as a standalone metric. This means that you can see the overall trend of Form Errors per Form View (or Visit) to see if we are getting better or worse over time. This might make a great KPI metric for the team focused on Forms and Form Completions:

Final Thoughts
In my last post I covered a simple way to see which fields are causing problems for your visitors. In this post, I showed you how to quantify your Form Errors, see how much of an issue you may have and even see which Forms have the most Errors. In my next post I will show you some advanced ways to see which fields are causing errors and how to break this down by Form. Stay tuned!

Between this post and the last post, hopefully you have some food for thought when it comes to tracking how your website forms are doing so you improve your conversion rates…

Analysis, Reporting

Reporting: You Can't Analyze or Optimize without It

Three separate observations from three separate co-workers over the past two weeks all resonated with me when it comes to the fundamentals of effective analytics:

  • As we discussed an internal “Analytics 101” class  — the bulk of the class focusses on the ins and outs of establishing clear objectives and valid KPIs — a senior executive observed: “The class may be mislabeled. The subject is really more about effective client service delivery — the students may see this as ‘something analysts do,’ when it’s really a a key component to doing great work by making sure we are 100% aligned with our clients as to what it is we’re trying to achieve.”
  • A note added by another co-worker to the latest updated to the material for that very course said: “If you don’t set targets for success up front, someone else will set them for you after the fact.”
  • Finally, a third co-worker, while working on a client project and grappling with extremely fuzzy objectives, observed: “If you’ve got really loose objectives, you actually have subjectives, and those are damn tough to measure.”

SEO search engine optimization indiaIt struck me that these comments were three sides to the same coin, and it got me to thinking about how often I find myself talking about performance measurement as a critical fundamental building block for conducting meaningful analysis.

“Reporting” is starting to be a dirty word in our industry, which is unfortunate. Reporting in and of itself is extremely valuable, and even necessary, if it is done right.

Before singing the praises of reporting, let’s review some common reporting approaches that give the practice a bad name:

  • Being a “report monkey” (or “reporting squirrel” if you’re an Avinash devotee) — just taking data requests willy-nilly, pulling the numbers, and returning them to the requestor
  • Providing “all the data” — exercises of listing out every possible permutation/slicing of a data set, and then providing a many-worksheeted spreadsheet to end users so that they can “get any data they want”
  • Believing that, if a report costs nothing to generate, then there is no harm in sending it — automation is a double-edged sword, because it can make it very easy to just set up a bad report and have it hit users’ inboxes again and again without adding value (while destroying the analyst’s credibility as a value-adding member of the organization)

None of these, though, are reasons to simply toss reporting aside altogether. My claim?

If you don’t have a useful performance measurement report, you have stacked the deck against yourself when it comes to delivering useful analyses.

Let’s walk through a logic model:

  1. Optimization and analysis are ways to test, learn, and drive better results in the future than you drove in the past
  2. In order to compare the past to the future (an A/B test is a “past vs. future” because the incumbent test represents the “past” and both the incumbent and the challenger represent “potential futures”), you have to be able to quantify “better results”
  3. Quantifying “better results” mean establishing clear and meaningful measures for those results
  4. In order for measures to be meaningful, they have to be linked to meaningful objectives
  5. If you have meaningful objectives and meaningful measures, then you have established a framework for meaningfully monitoring performance over time
  6. In order for the organization to align and stay aligned, it’s incredibly helpful to actually report performance over time using that framework, quod erat demonstrandum (or, Q.E.D., if you want to use the common abbreviation — how in the hell the actual Latin words, including the correct spelling, were not only something I picked up in high school geometry in Sour Lake, TX, but that has actually stuck with me for over two decades is just one of those mysteries of the brain…)

So, let’s not just bash reporting out of hand, okay? Entirely too many marketing organizations, initiatives, and campaigns lack truly crystallized objectives. Without clear objectives, there really can’t be effective measurement. Without effective measurement, there cannot be meaningful analysis. Effective measurement, at it’s best, is a succinct, well-structured, well visualized report.

Photo: Greymatterindia

Adobe Analytics, General

Tracking Form Errors (Part 1)

Almost all websites have forms. Whether you are a B2B/Lead generation site, an eCommerce site, a travel site, etc… you most likely have forms. More importantly, you have people who don’t fill out your forms correctly and get some sort of error message. While error messages are a fact of life, in the web analytics/optimization world these are painful since you work so hard to get people to your site, to read your content and then agree to give you personal information. That is a lot of time and money spent only to have someone potentially abandon because they have problems with your forms. This represents your “low hanging fruit” so to speak – people who have already decided they like you and want to give you their information! In this series of posts, I am going to share some techniques for seeing how much of a problem your website has with form errors and in the next few posts I will cover some more advanced things you can do to diagnose these form error issues.


Which Fields Produce the Most Errors?
The first step in diagnosing form error issues is understanding which form fields are causing issues. Unfortunately, since a user might receive more than one error message, you have to pass in multiple values to a SiteCatalyst variable. This can be done using the Products variable, but since that is often already being used for more important purposes, I will suggest that you use a List Traffic Variable (sProp) to capture these values. Unfortunately, List sProps are not well documented and have some specific limitations (see Knowledge Base ID# 2305). All you need to know is that List sProps allow you to pass in delimited values and when you view them in the sProp report, these values will be split out. Let’s look at an example. Here we see a form in which a user has attempted to submit the form without filling out some required fields. What we want to do is capture which fields this user messed up (could mean incorrect value or leaving blank) so we can see which ones are messed up the most often. In this case, we see that the form errors are related to Job Title, E-mail Address, Phone #, Company Name and the MSA checkbox.

So in this case we can use a List sProp to capture the fields giving us errors. Here is how it would look in the JavaScript Debugger:

Unfortunately, List sProps are still constrained to the 100 character limit so if you have long forms you are out of luck or you can select the most important form fields to capture. Once you have captured the fields, you can open the sProp report and you will see something that looks like this:

In this case, we can see that we are getting the greatest number of errors on the Phone Number form field on the US website (I have added the site since forms exist in multiple sites). I could also filter this sProp report for just US or Japan form fields by using a text search of “us:” or “jp:” as needed. This report should help steer you in the right direction when it comes to fixing basic form field issues.

Correlating Form Field Errors to Forms
Once you have seen which form field errors, the next logical question is to see which forms had which errors. Unfortunately, one of the limitations of List sProps is that they cannot be used in Traffic Data Correlations. Therefore, if you want to breakdown form field errors by Form, you will need to use the Discover product as shown here:

If you don’t have access to Discover and seeing this type of breakdown is important to you, you may want to consider using the Products variable instead of a List sProp since the Products variable comes with full Subrelations by default (though this implementation will be significantly more difficult). I will also be covering a different way to approach this in my next post so stay tuned!

Final Thoughts
If you are not currently tracking form field errors, hopefully this will give you some ideas on how you can start the process of seeing where you are tripping up your visitors. Keep in mind that this post is just a start and that the next few posts will go into more advanced stuff you can do and how you can identify your biggest opportunities for improving conversion.

Analytics Strategy, Conferences/Community, General

FTC "Do Not Track?" Bring it on …

As the hubub around consumer privacy continues I was gently prodded by a friend to pipe up in the conversation.  While my feelings about how we have ended up in this position are pretty clear, and while my partner John and I have proposed what we believe is a step in the right direction regarding online privacy and the digital measurement community, it seems that some type of ban or limitation on online tracking is becoming inevitable.

Without getting political or debating the reality of what we can and cannot know about online visitors I have a single word response to the FTC:

Whatever.

Before you accuse me of changing my stripes or going completely nuts consider this: If the FTC is able to somehow pull off the creation of a universal opt-out mechanism, and if the browser developers support this mechanism despite clear and compelling reasons not to, and if consumers actually widely adopt the mechanism — all pretty big “ifs” in my humble opinion — then I believe the digital measurement industry will do what I have already described as inevitable:

We will hold a revolution!

Since my tenure at JupiterResearch back in 2005 I have been telling anyone who would listen to stop worrying about counting every visitor, visit, and page view and instead start thinking about statistically relevant samples, confidence intervals, and the algorithmic use of data to conduct analysis.  Yes, you need to work to ensure data quality — of course you do — but you don’t have to do it at the expense of your sanity, your reputation, or your job …

See, it turns out in our community it doesn’t really matter whether we are able to measure 100% of the population, 90% of the population, or even 80% of the population — what matters is that we are able to analyze our visitor populations and that are able to draw reasonable conclusions from that analysis.  Oh, we have to be empowered to conduct analysis as well, but that’s a whole other problem …

Statistical analysis of the data … trust me, it’s going to be all the rage in a few years. I’m not saying this simply because I have a white paper describing the third generation of digital measurement tools that will empower this type of analysis … although I would encourage you to download and read “The Coming Revolution in Web Analytics” (freely available thanks to the generous folks at SAS!)

I’m saying this because every day I see the writing on the wall.  Data volumes are increasing, data sources are increasing, and demands for insights are increasing, all while professional journalists, politicians, and political appointees are supposedly protecting our “God-given right to surf the Internet in peace” without any regard to the businesses, employees, and investors who depend to a greater or lesser degree on web-collected data to provide a service, pay their bills, and make a profit …

Okay, sorry, that was editorializing.  My bad.

Still, rather than wring our hands and gripe about how much the credit card companies know (which is a silly argument given that credit card companies provide tangible value in exchange for the data they collect … it’s called “money”) I believe it is time to do three things:

  1. Suck it up.
  2. Hold yourself to a higher standard.
  3. Buy “Statistics in Plain English” and start reading.

The good news is that we have access to lots and lots of great statistical analysis of sampled data today — we just might not realize it.  Consider:

Have I mentioned Excel, Tableau, and R?  Hopefully by now you get the gist … statistics is already all around us all the time, perhaps just not exactly where we expect it or, in the context of lower rates of data collection, where we will ultimately need it to be.

Perhaps the most encouraging evidence that we will be able to make this shift is the increasing attention the digital world is getting from traditional business intelligence market leaders like Teradata, FICO, IBM, and SAS.  I, for one, am more or less convinced that the gap between “web analytics” and “Analytics” is about to be closed even further … and here’s one guy that seems to agree with me.

We don’t need to thumb our noses at the privacy people — quite the opposite, and to this end John and I will be sitting down with a representative from the Center for Democracy and Privacy and Adobe’s Chief Privacy Officer MeMe Rasmussen at the next Emetrics in San Francisco! We also don’t need to stick our head’s back in the sand and hope this issue will simply go away — it won’t, trust me.

We need to prepare.

Prepare by committing yourself to not being that scary data miner that consumers are supposedly so afraid of; prepare by improving your data quality to the extent that you are able; and prepare by starting to communicate to leadership that it really doesn’t matter if you can count every visitor, every visit, and every page view — what matters is your ability to analyze data using the tools at your disposal to deliver value back to the business.

If you’re not sure how to do that, call us.

Viva la revolution!

DISCLOSURE: I mentioned and linked to lots of vendors in this post which I normally do not do. Some are clients of Analytics Demystified, others are not. If you have concerns about why we linked to one company and not another please don’t hesitate to email me directly.

Adobe Analytics, General

A/B Test Bounce Rates

(Estimated Time to Read this Post = 4 Minutes)

In the past, I have written about Bounce Rates, Traffic Source Bounce Rates , Segment Bounce Rates and Site Wide Bounce Rates. In the latter, I even promised I was finished writing about Bounce Rates, but, alas, I have yet another Bounce Rate installment. I was recently in a conversation with a peer and she asked me how they could see the bounce rates of the various landing page A/B tests they were running via Test&Target. I told her that this was easy to do if you follow my instructions in the Segment Bounce Rate post, but she asked if I could write a brief post with more specifics so here it is…

Why A/B Bounce Rates?
Before getting into the solution, let’s re-visit why this is of interest. Test&Target (and other tools like GWO) are wonderful when it comes to optimizing landing pages. They allow you to alter content/creative elements and see what works and what doesn’t. I have seen many cases where clients have used tools like Test&Target to change content based upon when brought the user to the website (i.e. Search Keyword) or demographic information (i.e. Location). Regardless of the reason you want to test, if it is a landing page, one of the questions you often get asked is related to Bounce Rate. Understanding how many people saw “Version A” of a test and bounced vs. those who saw “Version B” and bounced usually comes up for discussion. To answer this question using Omniture/Adobe tools you have the following options:

  • Create a unique page name for each test variation and use the regular Pages report and Bounce rate metric. However, this can get very messy, so unless your website is small, I don’t recommend this approach.
  • Use ASI or Discover to build a segment for people coming from “Version A” or “Version B” and then compare the bounce rates. This is a viable option if you have access to these tools and are well versed in Segmentation.
  • Attempt to track Bounce Rates from within Test&Target. This does not come out-of-the-box, but if you have mboxes on all of the pages the landing page links to, I have heard of some people setting conversion events on the landing page and the subsequent pages, but I don’t think this is for novices (if you are interested, I’m sure @brianthawkins could figure out a way to hack this together!)
  • Do what I suggest below!

Implementing A/B Bounce Rates
Luckily, implementing this in SiteCatalyst is relatively simple. All you need to do is the following:

  1. Enable a new Traffic Variable (sProp)
  2. In this new sProp, concatenate the Test&Target ID and the Page Name on each page of your website
  3. Enable Pathing on the new sProp

That’s it! By concatenating the Test&Target ID and the Page Name, you create a unique join between the two and can find the combination of the Test ID you care about and the page name that you expect them to have landed on. Once you find this combination in the report, you can add your Bounce Rate Calculated Metric (Single Access/Entries – which hopefully you already have as a Global Calculated Metric) and you are done. Here is an example of a report:

In this report, you have all of the ID’s associated with the US Home Page, how many Entries each received and the associated Bounce Rate. If you wanted, you could perform a search for the specific Test&Target Test ID you care about and then your report would be limited to just those ID’s. In the example above, we have multiple tests taking place on the US Home Page. However, in the following example we can see a case where there is just one test taking place on the UK Home Page and the associated Bounce Rate of each:

Other Cool Stuff
But wait…there’s more! Since you have enabled Pathing on this new A/B Test sProp, there are some other cool things you can do. First, you can look at a trended view of the report above to see how the Bounce Rate fluctuates during the course of the test. To do this, simply switch to the trended view and choose your time frame:

Another benefit of having Pathing enabled on this sProp is that you can see how visitors from various tests navigated your site using all of the out-of-the-box Pathing reports. Here is an example of a next page flow for one of the tests:

You can run the preceding report for each test variation and compare the path flows to see if one version pushes people more often to the places you want them to go. Another report you could run is a Fall-Out report which can show you how often people from a specific test made it through your desired checkpoints:

In this example, instead of seeing how the general population falls-out from the Home Page to a Product Page and then to a Form Page, we can limit the funnel to only those people who were part of Test ID “18964:1:0.” I like to run this report and the corresponding one for the other test version(s) and add them all to a SiteCatalyst Dashboard where I can see the fall-out rates side by side.

Final Thoughts
As you can see, by doing a little up-front work, you can add an enormous amount of insight into how your A/B tests are performing on your site including Bounce Rates, Next Page Flows, Fall-Out, etc…Enjoy!

Analysis, Social Media

Twitter Analytics — Turmoil Abounds, and I'm a Skeptic

Last week was a little crazy on the Twitter front, with two related — but very different —  analytics-oriented announcements hitting the ‘net within 24 hours of each other. Let’s take a look.

Selling Tweet Access

On Wednesday, Twitter announced they would be selling access to varying volumes of tweets, with 50% of all tweets being available for the low, low price </sarcasm> of $360,000/year. It appears there will be a variety of options, with “50%” being the maximum tweet volume, but with other options in the offing to get 5% of all tweets, 10% of all tweets, or all tweets/references/retweets that are tied to a specific user. All of these sound like they’re going to come with some pretty tight usage constraints, including that they can’t be resold and that the actual tweet content can’t be published.

Twitter has made an API available almost from the moment the service was created. That’s one of the reasons the service grew so explosively — developers were able to quickly build a range of interfaces to the tool that were better than what Twitter’s development team was able to create. But, the API came with limitations — a very tight limit on how often an application could get updates, and a tight limit on just how many updates could be pushed/pulled at once.

As various Twitter analytics-type services began to crop up, Twitter opened up a “garden hose” option — developers could contact Twitter, show that they had a legitimate service with a legitimate need, and they could get access to more tweets more often through the API. Services like Twitalyzer, TweetReach, and Klout jumped all over that option and have built out robust and useful solutions over the course of the last 6-12 months. Now it looks like Twitter is looking to coil up the garden hose, which could spell a permanent end to the growing season for these services. This will be a shame if it comes to pass.

For a steep price, these paid options from Twitter will have limited use: limited to some basic monitoring/listening and some basic performance measurement. Even with the $360K/year option, providing half of the tweets seems problematic when you consider Twitter from a social graph perspective — in theory, half of the network ripple from any given tweet will be lost, or, more confusingly, will crop up as a 2nd or 3rd degree effect with no ability to trace it back to its source because the path-to-the-source passes through the “unavailable 50%!”

This data also won’t be of much use as a listen-and-respond tool. Imagine a brand that has a fantastic ability to monitor Twitter and appropriately engage and respond…but appears schizophrenic because they’re operating with one eye closed (and paying a pretty penny to do even that!). To be clear, for any given brand or user, only a tiny fraction of all tweets are actually of interest, but that tiny fraction is going to be spread across 100% of the Twitterverse, so only having access to a 5%, 10%, or even 50% sample means that relevant tweets will be missed.

Online listening platforms — Radian6, SM2, Buzzmetrics, Crimson Hexagon, Sysomos, etc. — may actually have deep enough pockets to pay for these tweets to improve their own underlying data…but they will have to significantly alter the services they provide in order to comply with the usage guidelines for the data.

Ugh.

Twitter Analytics

On Thursday, Mashable reported that Twitter Analytics was being tested by selected users. Unfortunately, I’m not one of those users (<sniff><sob>), so I’m limited to descriptions in the Mashable article. Between that article and Pete Cashmore’s (Mashable CEO) editorial on cnn.com, I’ve got pretty low expectations for Twitter Analytics.

Both pieces seem somewhat naive in that they overplay the value to brands that Facebook has delivered with Facebook Insights, and they confuse “pretty graphs” with “valuable data.” All I can think to do is rattle off a series of reactions from the limited information I’ve been able to dig up:

  • Replies/references over time: um…thanks, but that’s always been something that’s pretty easy to get at, so no real value there.
  • Follows/unfollows: this seems to be taking a page directly from Facebook Insights with it’s new fans/removed fans reporting (which, by the way, never agrees with the “Total Fans” data available in the same report, but I digress…); this has marginal value — in practice, unless a user is really pissing off followers or baiting them to follow with a very specific promotional giveaway (“Follow us and retweet this and you’ll be entered to win a BRAND NEW CAR!!!”), there’s probably not going to be a big spike in unfollows, and it isn’t that hard to trend “total followers” over time, so I can’t get too excited about this, either
  • Unfollows (cont’d.): “tweets that cause people to unfollow” is another apparent feature of Twitter analytics. Really? Was that something that someone living on planet Earth came up with? This sounds nifty initially, but, in practice, isn’t going to be of much use. If a user posts offensive, highly political (for a non-political figure user), or obnoxiously self-promoting tweets…he’s going to lose followers. I don’t think “analytics” will really be needed to figure out the root cause (if it was a single tweet) driving a precipitous follower drop. Common sense should suffice for that.
  • Retweets: this is like references, in that it’s not really that hard to track, and I wouldn’t be surprised at all if Twitter Analytics only counts retweets that use the official Twitter retweet functionality, rather than using a looser definition that includes “RT @<username>” occurrences (which are retweets that are often more valuable, because they can include additional commentary/endorsement by the retweeters)
  • Impressions: I’m expecting a simplistic definition of impressions that is based just on the number of followers, which is misleading, because most users of Twitter see only a fraction of the tweets that cross their stream. Twitalyzer calculates an “effective reach” and Klout calculates a “true reach” — both make an attempt to factor in how receptive followers are to messages from the user. None of these measures is going to be perfect, but I’m happier relying on companies whose sole focus is analytics trying to tinker with a formula than I am with the “owner” of the data coming up with a formula that they think makes sense.

With the screen caps I’ve seen, there is no apparent “export data” button, and that’s a back-breaker. Just as Facebook Insights is woefully devoid of data export capabilities (the “old interface” enables data export…but not of some of the most useful data, and API access to the Facebook Insights data doesn’t exist, as best as I’ve been able to determine), Twitter looks like they may be yet another technology vendor who doesn’t understand that “their” dashboard is destined to be inadequate. I’m always going to want to combine Twitter data with data that Twitter doesn’t have when it comes to evaluating Twitter performance. For instance, I’m going to want to include referrals from Twitter to my web site, as well as short URL click data in my reporting and analysis.

Ikong Fu speculated during an exchange (on Twitter) that Twitter may also, at some point, include their internal calculations of a user’s influence in Twitter Analytics:

I didn’t realize that Twitter was calculating an internal reputation score. It makes sense, though, that that would be included when they make recommendations of who else a user might want to follow. I found a post from Twitter’s blog back in July that announced the rollout of  “follow suggestions,” and that post indicated these were based on “algorithms…built by our user relevance team.” The only detail the post provided was that these suggestions were “based on several factors, including people you follow and the people they follow.” That sounds more like a social graph analysis (“If you’re following 10 people who are all following the same person who you are not following, then we’re going to recommend that you follow that person”) than an analysis of each user’s overall influence/quality. Again…I’m more comfortable with third party companies who are fully focussed on this measurement and who make their algorithms transparent providing me with that information than I am with Twitter in that role.

So, Where Does This Leave Us?

Maybe, for once, I’m just seeing a partially filled glass of data as being half empty rather than half full (okay, so that’s the way I view most things — I’m pessimistic by nature). In the absence of more information, though, I’m forced to think that, just as I was headed towards analytics amour when it came to Twitter data, Twitter is making some unfortunate moves and rapidly smudging the luster right off of that budding relationship.

Or, maybe, I’m unfairly pre-judging. Time will tell.

Social Media

WAW Recap: Marketing to Hispanics Using Social Media

We jumped a little afield of web analytics at this month’s Columbus Web Analytics Wednesday: Why Marketing to Hispanics Using Social Media works. The event was hosted and sponsored by Social Media Spanish, and it was chock full of good information. Natasha Pongonis and Eric Diaz presented a host of statistics about both the growth of the Hispanic population in the U.S., the many ways that Hispanics are heavier users of social media than the population as a whole, and how smart brands are targeting Hispanics using social media. They posted the full presentation on the Social Media Spanish blog, and it’s definitely worth checking out.

It’s a complex topic, which Natasha Pongonis highlighted early on with this chart (yes, sadly, it’s a pie chart) showing a breakdown of Hispanics in the U.S. (click to view a larger version):

One of the takeaways here was that there are a significant number of American Hispanics who prefer to communicate in English…and a significant number who prefer to communicate in Spanish! Some of the discussion later in the presentation centered on this challenge — simply “targeting Hispanics” is too broad of a classification, as even something as basic as which language to use in that targeting varies!

It’s a challenge: with social media platforms evolving rapidly in conjunction with evolving consumer expectations, marketers are faced with pros and cons of just about any strategy using these tools.

Not only was the content great, but Social Media Spanish secured a great venue, great food, and even a real photographer! I brought my camera, but Alison Horn really took some great shots, which she has posted as a set on Flickr. Check ’em out!

Adobe Analytics, Analytics Strategy, General

Tracking Lead Gen Forms by Page Name

Every once in a while, as a web analyst, I get frustrated by stuff and feel like there has to be a better way to do what I am trying to do. Many times you are able to find a better way, often times you are not. In this case, I had a particular challenge and did find a cool way to solve it. You may not have the same problem, but, if for no other reason than to get it off my chest, I am writing this as a way to exhale and bask in my happiness of solving a web analytics problem…

My Recent Problem
So what was the recent problem I was facing that got me all bent out of shape? It had to do with Lead Generation forms, which are a staple of B2B websites like mine. Let me explain. Many websites out there, especially B2B websites, have Lead Generation as their primary objective. In past blog posts, I have discussed how you can track Form Views, Form Completes and Form Completion Rates. However, over time, your website may end up with lots of forms (we have hundreds at Salesforce.com!). In a perfect world, each website form would have a unique identifier so you can see completion rates independently. That isn’t asking too much is it? However, as I have learned, we rarely live in a perfect world!

Through some work I did in SiteCatalyst, I found that our [supposedly unique] form identifier codes were being copied to multiple pages on multiple websites. While this causes no problems from a functionality standpoint – visitors can still complete forms – what I found was that the same Form ID used in the US was also being used in the UK, India, China, etc… Therefore, when I ran our Form reports and looked at Form Views, Form Completes and Form Completion Rate by Form ID, I had no idea that I was looking at data for multiple countries. For example, if you look at this report nothing seems out of the ordinary right?

However, look what happened when I broke this report (last row of above report) down by a Page Name eVar:

At first, I thought I was going crazy! How can this unique Form ID be passed into SiteCatalyst on eleven different form pages on nine country sites? This caused me to dig deeper, so I did a DataWarehouse report of Form ID’s by Page Name and found that an astounding number of Form Pages on our global websites shared ID’s. Suddenly, I panicked and realized that whenever I had been reporting on how Forms were performing, I was really reporting on how they were performing across several pages on multiple websites. In the example above, I realized that the 34.669% Form Completion Rate I was reporting for the US version of the form in question was really reporting data with the same ID for forms residing on websites in Germany, China, Mexico, etc… While the majority was coming the the form I was expecting, 22% was coming from other pages! Not good!

The Solution
So there I was. Stuck in web analytics hell, reporting something different than I thought I was. What do you do? The logical solution was be to do an audit and make sure each Form page on the website had a truly unique ID. However, that is easier said than done when your web development team is already swamped. Also, even if you somehow manager to fix all of the ID’s, what is preventing these ID’s from getting duplicated again? We looked at all types of process/technology solutions and then realized that there is an easy way to fix this by doing a little SiteCatalyst trickery.

So what did we do? We simply replaced the Form ID eVar value with a new value that concatenated the Page Name and the Form ID on every Form Page and Form Confirmation Page. By concatenating the Page Name value, even if the same Form ID was used on multiple pages, the concatenated value would still be unique. For example, the old Form ID report looked like the one above:

But the new version looked like this:

With this new & improved report, when I was reporting for a particular form on a particular site/page, I could search by the form pagename and be sure I was only looking at results from that page. Also, a cool side benefit of this approach is that you could add a Form ID to the search function to quickly find all pages that had the same Form ID in case you ever did want to clean up your Form ID’s:

Implementation Gothca!
However, there is one tricky part of this solution. While it is certainly easy to concatenate the s.pagename value with the Form ID on the Form page, what about the Form Confirmation page? The Form Confirmation page is where you should be setting your Form Completion Success Event and that page is going to have a different pagename. If your Form ID report doesn’t have the same Page Name + Form ID value for both the Form View and Form Complete Success Event, you cannot use a Form Completion Rate Calculated Metric. For this reason, you need to use the Previous Value Plug-in to pass the previous pagename on the Form Confirmation page. Doing this will allow you to pass the name of the “Form View” page on both the Form View and Form Complete page of your site so you have the same page name value merged with the Form ID.

A Few More Things
Finally, while the Form ID report above serves this particular function, it is not very glamorous and it might not be the most user-friendly report for your users. If you want to provide a more friendly experience you can do the following with SAINT Classifications:

  1. Classify the Form ID value by its Page Name so your users can see Form Views, Form Completions and the Form Completion Rate by Page Name
  2. Classify the Form ID value by the Form ID if for some reason you want to go back to seeing the report you had previously

Final Thoughts
Well there you have it. A very specific solution to a specific problem I encountered. If you have Lead Generation Forms on your website, maybe it will help you out one day. If not, thanks for letting me get this out of my system!

Analytics Strategy, Conferences/Community, General

Are you in Atlanta? I will be, next week!

Just a quick note to those of you in the greater Atlanta (GA) metropolitan region to let you know I will be in town next week working and participating in two awesome events:

  1. A blow-out Atlanta Web Analytics Wednesday, sponsored by the fine folks at Unica, where I will be moderating a “practitioner panel” with Delta, Home Depot, and How Stuff Works. Sadly we only had room for 60 odd people and that list filled up almost right away … but you can write to Jane Kell and ask to be put on the waiting list just in case people have to back out.
  2. A presentation to Atlanta CHI on “Getting to Know Your Users Using Data” at the Georgia Tech Research Institute Conference Center. As this is a somewhat mixed audience my presentation will be a high-level walk through the systems and processes we all leverage on a daily basis.

As far as I know the CHI event is not sold out and costs $35 for the general public, $10 for students with ID, and nothing (free!) if you are a member of Atlanta CHI!

For those of you not in Atlanta, apologies for this utterly useless blog post. Hopefully I will make it to your town soon and can make it up to you …

Reporting, Social Media

Twitter Performance Measurement with (a Heavy Reliance on) Twitalyzer

My Analyzing Twitter — Practical Analysis post a few weeks ago wound up sparking a handful of fantastic and informative conversations (“conversations” in the new media use of the term: blog comments, e-mails, and Twitter exchanges in addition to one actual telephone discussion). That’s sort of the point of social media, right? The fact that I can now use these discussions as an example of why social media has real value isn’t going to convince people who view it just as a way to tell the world the minutia of your life, because they would point out that gazing at one’s navel to better understand navel-gazing…is still just navel-gazing. So, yeah, if a brand knows that 145 million consumers have signed up for Twitter and knows that they are welcome to leverage it as a marketing channel, but just don’t fundamentally believe that it’s a channel to at least consider using, then neither anecdotes nor good-but-not-perfect data is going to convince them.

Many brands, though, are convinced that Twitter is a channel they should use and are willing to put some level of resources towards it. But, the question still remains: “How do we most effectively measure the results of our investment?” Everything in Twitter occurs at a micro level — 140 characters at a time. A single promotion with a direct response purchase CTA can be measured, certainly, but that’s an overly myopic perspective. So, what is a brand to do? For starters, it’s important to recognize there are (at least) three fundamentally different types of “measurement” of Twitter:

  • Performance measurement — measuring progress towards specific objectives of the Twitter investment
  • Analysis and optimization — identifying opportunities to improve performance in the channel
  • Listening (and responding) — this is an area where social media has really started blurring the line between traditional outbound marketing, PR, consumer research, and even a brand’s web site; with Twitter, there is the opportunity to gather data (tweets) in near real-time and then respond and engage to selected tweets…and whose job is that?

The kicker is that all three of these types of “measurement” can use the same underlying data set and, in many cases, the same basic tools (with traditional web analytics, both performance measurement and analysis often use the same web analytics platform, and plenty of marketers don’t understand the difference between the two…but I’m going to maintain some  self-discipline and avoid pursuing that tangent here!).

This post is devoted to Twitter performance measurement, with a heavy, heavy dose of  Twitalyzer as a recommended key component of that approach. Have I done an exhaustive assessment of all of the self-proclaimed Twitter analytics tools on the market? No. I’ll leave that to Forrester analysts. I’ve gone deep with one online listening platform and have done a cursory survey of a mid-sized list of tools and found them generally lacking in either the flexibility or the specificity I needed (I will touch on at least one other tool in a future post that I think complements Twitalyzer well, but I need to do some more digging there first). Twitalyzer was (and continues to be) designed and developed by a couple of guys with serious web analytics chops — Eric Peterson and Jeff Katz. They’ve built the tool with that mindset — the need for it to have flexibility, to trend data, to track measures against pre-established targets, and to calculate metrics that are reasonably intuitive to understand. They’ve also established a business model where there is “unlimited use” at whichever plan level you sign up for — there is no fixed number of reports that can be run each month, because, generally, you want to see a report’s results and iterate on the setup a few times before you get it tuned to what you really need. So, there’s all of that going for it before you actually dive into the capabilities.

One more time: this is not a comprehensive post of everything you can do with Twitalyzer. That would be like trying to write a post about all the things you can do with Google Analytics, which is more of a book than a post. For a comprehensive Twitalyzer guide, you can read the 55-page Twitalyzer handbook.

Metrics vs Measures

The Twitalyzer documentation makes a clear distinction between “metrics” and “measures,” and the distinction has nothing to do with whether the type of data is useful or not. Measures are simply straight-up data points that you could largely get by simply looking at your account at any point in time — following count, follower count, number of lists the user is included on, number of tweets, number of replies, number of retweets, etc. Metrics, on the other hand, are calculated based on several measures and include things like influence, clout, velocity, and impact. Obviously, metrics have some level of subjectivity in the definition, but there are a number of them available, and everywhere a metric is used, you are one click away from an explanation of what goes into calculating it. The first trick is choosing which measures and metrics tie the most closely to your objectives for being on Twitter (“increase brand awareness” is a a very different objective from “increase customer loyalty by deepening consumer engagement”). The second trick is ensuring that the necessary stakeholders in the Twitter effort buy into them as valid indicators of performance.

For both metrics and measures, Twitalyzer provides trended data…as best they can. Twitalyzer is like most web analytics packages in that historical data is not magically available when you first start using the tool. Now, the reason for that being the case is very different for Twitalyzer than it is for web analytics tools. Basically, Twitter does not allow unlimited queries of unlimited size into unlimited date ranges. So, Twitalyzer doesn’t pull all of its measures and calculate all of the metrics for a user unless someone asks the tool to. The tool can be “asked” in two ways:

  • Someone twitalyzes a username (you get more data if it’s an account that you can log into, but Twitalyzer pulls a decent level of data even for “unauthenticated” accounts)
  • All of the tracked users in a paid account get analyzed at least once a day

When Twitalyzer assesses an account, the tool looks at the last 7 days of data. So, as I understand it, if you’re a paid user, then any “trend” data you look at is, essentially, showing a rolling 7-day average for the account (if you’re not a paid user, you could still go to the site each day and twitalyze your username and get the same result…but if you really want to do that, then suck it up and pay $10/month — it’ll be considerably cheaper if you have even the most basic understanding of the concept of opportunity costs). This makes sense, in that it reasonably smooths out the data.

Useful Measures

There isn’t any real magic to the measures, but the consistent capture of them with a paid account is handy. And, what’s nice about measures is that anyone who is using Twitter sees most of the measures any time they go to their page, so they are clearly understood. Some measures that you should consider (picking and choosing — selectivity is key!) include:

  • Followers — this is an easy one, but it’s the simplest indication as to whether consumers are interested in interacting with your brand through Twitter; and if your follower count ever starts declining, you’ve got a very, very sick canary in your Twitter coal mine — consumers who, at one time, did want to interact with you are actively deciding they no longer want to do so; that’s bad
  • Lists — the number of lists the user is a member of is another measure I like, because each list membership is an occasion where a reasonably sophisticated Twitter user has decided that he/she has stopped to think about his relationship with your brand, has categorized that relationship, AND has the ability to then share that category with other users.
  • Replies/References — if other Twitter users are aware of your presence and are actually referencing it (“@<username>”), that’s generally a good thing (although, clearly, if that upticks dramatically and those references are very negative, then that’s not a good thing)
  • Retweets — people are paying attention to what you’re saying through Twitter, and they’re interested in it enough to pass the information along

Twitalyzer actually measures unique references and unique retweets (e.g., if another user references the tracked account 3 times, that is 3 references but only 1 unique reference — think visits vs. page views in web analytics), but, as best as I can tell, doesn’t make those measures directly available for reporting. Instead, they get used in some of the calculated metrics.

A few other measures to consider that you won’t necessarily get from Twitalyzer include:

  • Referrals to your site — there are two flavors of this, and you should consider both: referrals from twitter.com to your site (are Twitter users sharing links to your site overall?), and clickthroughs on specific links you posted (which you can track through campaign tracking, manually through a URL shortener service like bit.ly or goo.gl, or through Twitalyzer)
  • Conversions from referrals — this is the next step beyond simply referrals to your site and is more the “meaningful conversion” (not necessarily a purchase, but it could be) of those referrals once they arrive on your site
  • Volume and sentiment of discussions about your brand/products — Twitalyzer does this to a certain extent, but it does it best when the brand and the username are the same, and I’m inclined to look to online listening platforms as a more robust way to measure this for now

Calculated Metrics

Now, the calculated metrics are where things really get interesting. Each calculated metric is pretty clearly defined (and, thankfully, there is ‘nary a Greek character in any of the definitions, which makes them, I believe, easier for most marketers to swallow and digest). This isn’t an exhaustive list of the available metrics, but the ones I’m most drawn to as potential performance measurement metrics are:

  • Impact — this combines the number of followers the user has, how often the user tweets, the number of unique references to the user, and the frequency with which the user is uniquely retweeted and uniquely retweets others’ tweets; this metric gets calculated for other Twitter users as well and can really help focus a brand’s listening and responding…but that’s a subject for another post
  • Influence — a measure of the likelihood that a tweet by the user will be referenced or retweeted
  • Engagement — a lot of brands still simply “shout their message” out to the Twitterverse and never (or seldom) reference or reply to other users; Twitalyzer calculates engagement as a ratio of how often the brand references other user compared to how often other users reference the brand; so, this is a performance measure that is highly influenced by the basic approach to Twitter a brand takes, and many brands have an engagement metric value of 0%. It’s an easy metric to change…as long as a brand wants to do so
  • Effective Reach — this combines the user’s influence score and follower count with the influence score and follower count of each user who retweeted the user’s tweet to “determine a likely and realistic representation of any user’s reach in Twitter at any given time.” Very slick.

There are are a number of other calculated metrics, but these are the ones I’m most jazzed about from a performance measurement standpoint. (I’m totally on the fence both with Twitalyzer’s Clout metric and Klout‘s Klout score, which Twitalyzer pulls into their interface — there’s a nice bit of musing on the Klout score in an AdAge article from 30-Sep-2010, but the jury is still out for me.)

Setting Goals

Okay, so the next nifty aspect of Twitalyzer when it comes to performance measurement is that you can set goals for specific metrics:

Once a goal is set, it then gets included on trend charts when viewing a specific metric. “But…what goal should I set for myself? What’s ‘normal?’ What’s ‘good?'” I know those questions will come, and the answer isn’t really any better than it is for people who want to know what the “industry benchmark for an email clickthrough rate” is. It’s a big fat “it depends!” But, assessing what your purpose for using Twitter is, and then translating that into clear objectives, and then determining which metrics make the most sense, it’s pretty easy to identify where you want to “get better.” Set a goal higher than where you are now, and then track progress (Twitalyzer also includes a “recommendations” area that makes specific notes about ways you can alter your Twitter behavior to improve the scores — the metrics are specifically designed so that the way to “game” the metrics…is by being a better Twitter citizen, which means you’re not really gaming the system).

I’d love to have the ability to set goals for any measure in the tool, but, in practice, I don’t expect to do any regular performance reporting directly from Twitalyzer’s interface for several reasons:

  • There are measures that I’ll want to include from other sources
  • The current version of the tool doesn’t have the flexibility I need to put together a single page dashboard with just the measures and metrics I care about for any given account — the interface is one of the cleanest and easiest to use that I’ve seen on any tool, but, as I’ve written about before, I have a high bar for what I’d need the interface to do in order for the tool itself to actually be my ultimate dashboard

Overall, though, goal-setting = good, and I appreciate Eric’s self-admitted attempt to continue to steer the world of marketing performance measurement to a place where marketers not only establish the right metrics, but they set targets for them as well, even if they have to set the targets based on some level of gut instinct. You are never more objective about what it is you can accomplish than you are before you try to accomplish it!

But, Remember, That’s not All!

So, this post has turned into something of a Twitalyzer lovefest. Here’s the kicker: the features covered in this post are the least interesting/exciting aspects of the tool. Hopefully, I’ll manage to knock out another post or two on actually doing analysis with the tool and how I can easily see it being integrated into a daily process for driving a brand’s Twitter investment. Twitalyzer is focussed on Twitter and getting the most relevant information for the channel directly out of the API, unlike online listening platforms that cover all digital/social channels and, in many cases, are based on text mining of massive volumes of data (which, as I understand it, is generally purchased from one of a small handful of web content aggregators). It’s been designed by marketing analysts — not by social media, PR, or market research people.  It’s pretty cool and does a lot considering how young it is (and the 4.0 beta is apparently just around the corner). Like any digital analytics tool, it’s going to have a hard time keeping up with the rapid evolution of the channel itself, but it’s one helluva start!

Analytics Strategy, Conferences/Community

Updated "Web Analysts Code of Ethics"

Just in case you hadn’t seen this already I wanted to call your attention to the updated (version 2) “Web Analysts Code of Ethics” over at the Web Analytics Association blog. John Lovett and the members of the Standards Subcommittee did a wonderful job condensing my original work down into a more easily digested document.

The committee is still looking for comments on this version so please, please head over, read the update, and let us know what you think.

Thanks to John and the WAA for making this happen for all of us!

Adobe Analytics, General

Tracking Recurring Revenue

Recently, I had the pleasure of meeting a web analyst whose business relied on a subscription model. In a subscription model, you often sell a product initially and then there is a subsequent Recurring Revenue stream (normally monthly). During the conversation, I explained how I would address this in SiteCatalyst and since it is a somewhat advanced concept, thought I would share the same info here in case there are blog readers out there who also have Recurring Revenue models.

Why Is Recurring Revenue A Challenge?
So why is tracking Recurring Revenue in SiteCatalyst difficult? As always, I like to explain through an example. Let’s imagine that you sell a popular CRM product that has an initial sale price and then a monthly subscription. A visitor comes to your website from a Bing keyword of “CRM” and they end up purchasing your product for $10,000. You can track this $10,000 sale online and attribute it to the Bing keyword. However, what do you do after one month? Let’s say the customer pays $1,000 each month after the initial $10,000. How do you attribute the recurring monthly $1,000 to the Bing keyword that originally brought the customer? Many clients I have seen stop at the initial sale, but this is problematic. What if there are some marketing campaigns that bring in a lot of initial sales, but those campaigns produce customers who quit the subscription after two months? Perhaps there are other marketing campaigns that generate lower initial sale amounts, but result in customers who are retained for several years. How do you compare “apples to apples” in this case if you cannot tie both initial and subscription revenue to the original marketing campaign?

The answer for most clients is to simply pass the original marketing campaign to their back-end system and do all of the reporting outside of a tool like SiteCatalyst. However, this has the following negative consequences:

  1. As a web analyst, you are now out of the loop which is not good for your program (or your career!)
  2. There are hundreds of online web-only data points that you know about the original sale (i.e. visit number, internal search terms used, internal promos used, etc…). Are you going to pass all of these data points to your company’s data warehouse and do analysis there? If you are a big company that may be possible, but what if you are a small or mid-sized business?

If you are like me, at a minimum, I like to have all important data in SiteCatalyst so I can have a seat at the table. With this in mind, the following section will describe what you need to do if you want to get Recurring Revenue into your SiteCatalyst implementation so it can be tied to the same data points as the initial sale.

Recurring Revenue in SiteCatalyst Reports
If you have read my past blog posts or even just my last post on Product Returns, you may know that one of my favorite SiteCatalyst features is Transaction ID (I suggest re-reading this post!). At a high level, Transaction ID allows you to set an ID associated with a transaction and later upload offline metrics that are dynamically associated with any Conversion Variable (eVar) values which were active at the time the Transaction ID was set. Just as Transaction ID was important to solving our Product Returns issue, it can be used similarly to solve the aforementioned Recurring Revenue challenge.

When visitors make their initial subscription purchase, you can set a Transaction ID value on the confirmation page. Doing this allows you to establish “key” that can be used later to upload Recurring Revenue and tie it to all of the eVar values associated with the original sale. Keep in mind that you will have to work with your Adobe account manager to get Transaction ID set-up. Additionally, Transaction ID is normally only used for 90 days, but in this case you will need to work with your account manager to get it extended perpetually (or for as long as you want to include Recurring Revenue). For example, if you want to associate two years of revenue with the eVar values that contributed to the original sale, then your Transaction ID data must persist for two years.

Once you have Transaction ID enabled and have started passing Transaction ID’s for online purchases, the next step is to create a new “Recurring Revenue” [Currency] Incrementor Event. Transaction ID uploads are similar to Data Sources and, as such, can only import data as Incrementor Success Event. Setting up a new Incrementor Event is easily done through the Admin Console.

Once you have Transaction ID set-up and your new Recurring Revenue currency Success Event, you need to generate a Data Sources template file that you can upload on a monthly/weekly/daily basis. This file will consist of each subscription account that is still active and the amount of [monthly] Recurring Revenue that should be recorded for each date. Normally, you will have subscriptions that expire at varying dates so you may decide to upload a file on a daily basis which represents all those who are starting a new subscription cycle on that date. The Data Sources template that you will create should have the following columns:

  1. Date – Use the date that the new subscription cycle starts, not the date of the original sale. This date will determine which month the Recurring Revenue will appear in when using SiteCatalyst. NOTE: Please keep in mind that there is currently a SiteCatalyst restriction that you cannot upload a Transaction ID file that has dates spanning more than 90 days. You can upload dates that are more than 90 days old, but the date ranges for the entire file upload cannot be more than 90 days (kind of lame in my opinion!).
  2. Transaction ID – The ID set when the initial subscription sale took place
  3. Product Name/ID – Same value that was passed to the Products Variable during the original online subscription purchase
  4. Recurring Revenue – Amount that the client will be charged for the next subscription cycle

When you are done, your Data Sources upload file might look something like this:

Seeing Recurring Revenue in SiteCatalyst Reports
Once you have successfully uploaded some Recurring Revenue data, it is time to see how all of this looks in SiteCatalyst. To do this, open the Products report and add Revenue and our new Recurring Revenue metrics. The report should look like this:

Next, you can create a Calculated Metric which combines the two metrics to create a “Total Revenue” metric as shown here:

Finally, since Transaction ID allows you to apply the eVar values that were associated with the original transaction to the new Recurring Revenue data, you can use Subrelation break downs for the Products report by Campaign and see both Revenue and Recurring Revenue (and the Total Revenue Calculated Metric)!

In the above report, we can see an example of the quandary I described in the beginning of this post. When we break down our first product (Sales Cloud) by marketing campaign, if we look at the Revenue column, it looks like we should be focusing our marketing spend on Bing Branded Keywords. However, when we add our Recurring Revenue, we can see that the majority of our Recurring Revenue and the most of our Total Revenue is coming from E-mail. Perhaps that is the best place to concentrate our marketing budget…

Final Thoughts
So there you have it. A simple, yet [hopefully] effective approach for making sure that you show all Revenue you are helping generate whether it takes place during the initial sale or subsequently… If you have comments/questions, please leave a comment here…

Analysis, Social Media

Four Ways that Media Mix Modeling (MMM) Is Broken

Many companies rely on some form of media mix modeling (or “marketing mix modeling”) to determine the optimal mix of their advertising spend. With the growth of “digital” media and the explosion of social media, these models are starting to break down. That puts many marketing executives in a tough bind:

  1. Marketing, like all business functions, must be data-driven — more so now than ever
  2. Digital is the “most measurable medium ever” (although their are wild misperceptions as to what this really means)
  3. Ergo, digital media investments must be precisely measured to quantify impact on the bottom line

For companies that have built up a heavy reliance on media mix modeling (MMM), the solution seems easy: simply incorporate digital media into the model! What those of us who live and breathe this world recognize (and lament over drinks at various conferences for data geeks), is that this “simple” solution simply doesn’t work. Publicly, we say, “Well…er…it’s problematic, but we’re working on it, and the modeling techniques are going to catch up soon.”

My take: don’t hold your breath that MMM is going to catch up — even if it catches up to today’s reality, it will already be behind, because digital/social/mobile will have continued its explosive evolution (and complexity to model).

Believe it or not, I’m not saying that MMM should be completely abandoned. It still has it’s place, I think, but there are a lot of things it’s going to really, really struggle to address. I’d actually like to see companies who provide MMM services weigh in on what that is. At eMetrics earlier this month, I attended a session where the speaker did just that. Skip ahead to the last section to find out who!

Geographic Test/Control Data

Both traditional and digital marketing have a mix of geo-specific capabilities. The cost of TV, radio, print, and out-of-home (OOH) marketing provides an imperative to geo-target when appropriate (or simply to minimize the peanut butter effect of spreading a limited investment so thinly that it doesn’t have an impact anywhere). Many digital channels, though, such as web sites and Facebook pages, are geared towards being “available to everyone.” Other channels – SEM, banner ads, and email, for instance – can be geo-targeted, but there often isn’t a cost/benefit reason to do so. Without different geographic placements of marketing, the impact on sales in “exposed areas” vs. “unexposed areas” cannot be teased out:

Cross-Channel Interaction

While marketers have long known that multi-channel campaigns produce a whole that is greater than the sum of the parts, the sheer complexity that digital has introduced into the equation forces MMM to guess at attribution. For example, we know (or, at least, we strongly suspect) that a large TV advertising campaign will not only provide a lift in sales, but it will also produce a lift in searches for a brand. Those increased searches will increase SEM results, which will drive traffic to the brand’s web site. Consumers who visit the site can then be added to a retargeting campaign. Those are four different marketing channels that all require investment…but which one gets the credit when the consumer buys?

This is both data capture and a business rules question. Entire companies (Clearsaleing being the one that I hear the most about) have been built just to address the data capture and application of business rules. While they provide the tools, they’re a long way from really being able to capture data across the entire continuum of a consumer’s experience. The business rules question is just as significant — most marketers’ heads will explode if they’re asked to figure out what the “right” attribution is (and simply trying different attribution models won’t answer the question — different models will show different channels being the “best”). Is this a new career option for Philosophy majors, perhaps?

Fragmentation of Consumer Experiences

This one is related to the cross-channel interaction issue described above, but it’s another lens applied to the same underlying challenge. Consumer behavior is evolving — there are exponentially more channels through which consumers can receive brand exposure (I picked up the phrase “cross-channel consumer” at eMetrics, which is in the running for my favorite three-letter phrase of 2010!). Some of these channels operate both as push and pull, whereas traditional media is almost exclusively “push” (marketers push their messages out to consumers through advertising):

We’re now working with an equation that has wayyyyyyyy more variables, each of which has a lesser effect than the formulas we were trying to solve when MMM first came onto the scene. HAL? Can you help? This is actually beyond a question of simply “more processing power.” It’s more like predicting what the weather will be next week — even with meteoric advancements in processing power and a near limitless ability to collect data, the models are still imprecise.

Self-Fulfilling Mix

Finally, there is a chicken-and-egg problem. While there are reams of secondary research documenting the shifting of consumer behavior from offline to online consumption…many brands still disproportionately invest in offline marketing. It’s understandable — they’re waiting for the data to be able to “prove” that digital marketing works (and prove it with an unrealistic degree of accuracy — digital his held to a higher standard than offline media, and the “confusion of precision with accuracy” syndrome is alive and well). But, when digital marketing investments are overly tentative (and those investments are spread across a multitude of digital channels), the true impact of digital can’t be detected because it’s dwarfed by the impact of the massive — if less efficient — investments in offline marketing:

If I shoot a pumpkin simultaneously with a $1,500 shotgun and a $30 BB gun and ask an observer to tell me how much of an impact the BB gun had…

So, Should We Just Start Operating on Faith and Instinct?

I wrote early in this post that I think MMM has its place. I don’t fully understand what that place is, but the credibility of anyone whose bread is buttered by their MMM book of business who stands up and says, “Folks, MMM has some issues,” immediately skyrockets. That’s exactly what Steve Tobias from Marketing Management Analytics (MMA) did at eMetrics. In his session, “Marketing Mix Modeling: How to Make Digital Work for a True ROI,” he talked at length about many of the same challenges I’ve described in this post (albeit in greater detail and without the use of cartoon-y diagrams). But, he went on to lay out how MMA is using traditional MMM in conjunction with panel-based data (in his examples, he used comScore for the analysis) to get “true ROI” measurement. All I’ve seen is that presentation, so I don’t have direct experience with MMA’s work in action, but I liked what I heard!

Adobe Analytics, General, Industry Analysis

Our Engagement Metric in use at Philly.com

Those of you who have read my blog for long know that I have written a tremendous amount about measures of visitor engagement online. In addition to numerous blog posts we have published a 50 page white paper describing how to measure visitor engagement and every year I give a half-dozen presentations on the subject. Unlike some people who seem to fear new ideas and others who disapprove of anything they themselves do not create I have long been a champion for evolving our use of metrics in web analytics to satisfy business needs.

But don’t take my word for it, read about how the nice folks at Philly.com are using a near complete version of my calculation to better understand their audience.

Cool, huh?

The thing I love about this article is that Philly.com is openly talking about their use of my engagement metric.  What’s better is that their sharing prompted another super-great organization (PBS) to comment that they too have been using my engagement metric for years.

Awesome.

I have been honored to work with several companies in the past three years who have implemented my metric and variations thereof but most treat the metric as a competitive secret. Given that most are in the hard-pressed and hyper-competitive online media world I understand, but I’m certainly happy to see Philly.com and Chris Meares share their story with the world.

Anyway, check out the article and, if you’re brave, download our white paper on visitor engagement and give it a read. If you are in media and are stuck trying to figure out how to get web analytics to work for you (instead of the other way around) give me a call. I’m more than happy to discuss how our measure of engagement might be able to help your business grow.

Analytics Strategy

My Letter To The C-Suite

The following originally posted in Exact Target’s 10 Ideas To Turn Into Results report. It’s part of their Letters to the C-Suite Series and this is my letter…

To The Executive Team:

Do you even know who your customers are anymore? Chances are, you probably don’t. You
catch fleeting glimpses of them as they open your emails or pop onto your website for a quick
visit. You might even momentarily engage with them when they drop into your store to browse
around or see your products firsthand. Or maybe you meet them ever so briefly as they feign
interest in your brand by “liking” something you posted on Facebook.

If you’re doing it right, your business is collecting feedback across many customer
touch points.

But you only really hear them when they shout from the rooftops, irate and full of vim. That’s
probably where you begin to learn what’s on their minds. But do you even know that it’s the
same person who was showing you all that love during your last promotion? Probably not.
In actuality, few companies really know their customers. Whether your customers are end
users or other businesses, how they interact with your brand, where they discover new
information, and how they communicate is changing at an astounding rate. Customers
are increasingly unaffected by traditional marketing conventions, and their tolerance for
redundant messaging, static content, and conflicting brand information is nonexistent. They
don’t see your organization like you do—in departmentalized silos of categories, products,
business units, and operating divisions. To them, you’re just that brand they either love, hate,
or treat with ambivalence. That is, until you knock their socks off by impressing them with your
service, support, and relevance. Yet, to really deliver value to your customers, you need to get
to know them. This starts by remembering the interactions you have with them and building
off of these activities.

Digital communication is the new reality, and treating customers through digital channels is
synonymous with how you’d treat someone you meet in person. Listen to what they’re saying
and respond with appropriate dialog. But most importantly, remember these things (because
upon your next conversation, your customer might just remember you):

• Your memory of customers exists at the database level.

• By maintaining customer profiles and appending them with attributes that contain history,
activity, and propensity (among other things), you can truly begin to have meaningful
interactions.

• To do this effectively, the database must contain information from all your touch points.

This includes transactional systems, web analytics, call centers, mobile devices, social
media, ATMs, stores, email systems, and whatever else you’re using to reach out.
Bringing your data together through integrations enables you to achieve a holistic picture of
your customers. A little scared by this? Well, you should be. Customer behaviors are going to
fundamentally change the way you engage with your audience. If you’re not equipped, they’re
going to take their conversations (and their wallets) elsewhere. By integrating your data, you
open opportunities for new customer dialogs.

Take my word for it—it’s happening NOW.

Your Agent For Change,
John Lovett

 

Analysis, Analytics Strategy, Reporting, Social Media

Analyzing Twitter — Practical Analysis

In my last post, I grabbed tweets with the “#emetrics” hashtag and did some analysis on them. One of the comments on that post asked what social tools I use for analysis — paid and free. Getting a bit more focussed than that, I thought it might be interesting to write up what free tools I use for Twitter analysis. There are lots of posts on “Twitter tools,” and I’ve spent more time than I like to admit sifting through them and trying to find ones that give me information I can really use. This, in some ways, is another one of those posts, except I’m going to provide a short list of tools I actually do use on a regular basis and how and why I use them.

What Kind of Analysis Are We Talking About?

I’m primarily focussed on the measurement and analysis of consumer brands on Twitter rather than on the measurement of one’s personal brand (e.g., @tgwilson). While there is some overlap, there are some things that make these fundamentally different. With that in mind, there are really three different lenses through which Twitter can be viewed, and they’re all important:

  • The brand’s Twitter account(s) — this is analysis of followers, lists, replies, retweets, and overall tweet reach
  • References of the brand or a campaign on Twitter — not necessarily mentions of @<brand>, but references to the brand in tweet content
  • References to specific topics that are relevant to the brand as a way to connect with consumers — at Resource Interactive, we call this a “shared passion,” and the nature of Twitter makes this particularly messy, but, to whatever level it’s feasible, it’s worth doing

While all three of these areas can also be applied in a competitor analysis, this is the only mention (almost) I’m going to make of that  — some of the techniques described here make sense and some don’t when it comes to analyzing the competition.

And, one final note to qualify the rest of this post: this is not about “online listening” in the sense that it’s not really about identifying specific tweets that need a timely response (or a timely retweet). It’s much more about ways to gain visibility into what is going on in Twitter that is relevant to the brand, as well as whether the time spent investing in Twitter is providing meaningful results. Online listening tools can play a part in that…but we’ll cover that later in this post.

Capturing Tweets?

When it comes to Twitter analysis, it’s hard to get too far without having a nice little repository of tweets themselves.  Unfortunately, Twitter has never made an endless history of tweets available for mining (or available for anything, for that matter). And, while the Library of Congress is archiving tweets, as far as I know, they haven’t opened up an API to allow analysts to mine them. On top of that, there are various limits to how often and how much data can be pulled in at one time through the Twitter API. As a consumer, I suppose I have to like that there are these limitations. As a data guy, it gets a little frustrating.

Two options that I’ve at least looked at or heard about on this front…but haven’t really cracked:

  • Twapper Keeper — this is a free service for setting up a tweet archive based on a hashtag, a search, or a specific user. In theory, it’s great. But, when I used it for my eMetrics tweet analysis, I stumbled into some kinks — the file download format is .tar (which just means you have to have a utility that can uncompress that format), and the date format changed throughout the data, so getting all of the tweets’ dates readable took some heavy string manipulation
  • R — this is an open source statistics package, and I talked to a fellow several months ago who had used it to hook into Twitter data and do some pretty intriguing stuff. I downloaded it and poked around in the documentation a bit…but didn’t make it much farther than that

I also looked into just pulling Tweets directly into Excel or Access through a web query. It looks like I was a little late for that — Chandoo documented how to use Excel as a Twitter client, but then reportd that Twitter made a change that means that approach no longer works as of September 2010.

So, for now, the best way I’ve found to reliably capture tweets for analysis is with RSS and Microsoft Outlook:

  1. Perform a search for the twitter username, a keyword, or a hashtag from http://search.twitter.com (or, if you just want to archive tweets for a specific user, just go to the user’s Twitter page)
  2. Copy the URL for the RSS for the search (or the user)
  3. Add a new RSS feed in MS Outlook and paste in the URL

From that point forward, assuming Outlook is updating periodically, the RSS feeds will all be captured.

There’s one more little trick: customize the view to make it more Excel/export-friendly. In Outlook 2007, go to View » Current View » Customize Current View » Fields. I typically remove everything except From, Subject, and Received. Then go to View » Current View » Format Columns and change the Received column format from Best Fit to the dd-Mmm-yy format. Finally, remove the grouping. This gives you a nice, flat view of the data. You can then simply select all the tweets you’re interested in, press <Ctrl>-<C>, and then paste them straight into Excel.

I haven’t tried this with hundreds of thousands of tweets, but it’s worked great for targeted searches where there are several thousand tweets.

Total Tweets, Replies, Retweets

While replies and retweets certainly aren’t enough to give you the ultimate ROI of your Twitter presence, they’re completely valid measures of whether you are engaging your followers (and, potentially, their followers). Setting up an RSS feed as described above based on a search for the Twitter username (without the “@”) will pick up both all tweets by that account as well as all tweets that reference that account.

It’s then a pretty straightforward exercise to add columns to a spreadsheet to classify tweets any number of ways by some use of the IF, ISERROR, and FIND functions. These can be used to quickly flag each tweet  as a reply, a retweet, a tweet by the brand, or any mix of things:

  • Tweet by the brand — the “From” value is the brand’s Twitter username
  • Retweet — tweet contains the string “RT @<username>
  • Reply — tweet is not a retweet and contains the string “@<username>

Depending on how you’re looking at the data, you can add a column to roll up the date — changing the tweet date to be the tweet week (e.g., all tweets from 10/17/2010 to 10/23/2010 get given a date of 10/17/2010) or the tweet month. To convert a date into the appropriate week (assuming you want the week to start on Sunday):

=C1-WEEKDAY(C1)+1

To convert the date to the appropriate month (the first day of the month):

=DATE(YEAR(C1),MONTH(C1),1)

C1, of course, is the cell with the tweet date.

Then, a pivot table or two later, and you have trendable counts for each of these classifications.

This same basic technique can be used with other RSS feeds and altered formulas to track competitor mentions, mentions of the brand (which may not match the brand’s Twitter username exactly), mention of specific products, etc.

Followers and Lists

Like replies and retweets, simply counting the number of followers you have isn’t a direct measure of business impact, but it is a measure of whether consumers are sufficiently engaged with your brand. Unfortunately, there are not exactly great options for tracking net follower growth over time. The “best” two options I’ve used:

  • Twitter Counter — this site provides historical counts of followers…but the changes in that historical data tend to be suspiciously evenly distributed. It’s better than nothing if you don’t have a time machine handy. (See the Twitalyzer note at the end of this post — I may be changing tools for this soon!)
  • Check the account manually — getting into a rhythm of just checking an account’s total followers is the best way I’ve found to accurately track total followers over time; in theory a script could be written and scheduled that would automatically check this on a recurring basis, but that’s not something I’ve tackled

I also like to check lists and keep track of how many lists the Twitter account is included on. This is a measure, in my mind, of whether followers of the account are sufficiently interested in the brand or the content that they want to carve it off into a subset of their total followers so they are less likely to miss those tweets and/or because they see the Twitter stream as being part of a particular “set of experts.” Twitalyzer looks like it trends list membership over time, but, since I just discovered that it now does that, I can’t stand up and say, “I use that!” I may very well start!

Referrals to the Brand’s Site

This doesn’t always apply, but, if the account represents a brand, and the brand has a web site where the consumer can meaningfully engage with the brand in some way, then measuring referrals from Twitter to the site are a measure of whether Twitter is a meaningful traffic driver. There are fundamentally two types of referrals here:

  • Referrals from tweeted links by the brand’s Twitter account that refer back to the site — these can be tracked by a short URL (such as bit.ly), by adding campaign tracking parameters to the URL so the site’s web analytics tool can identify the traffic as a brand-triggered Twitter referral, or both. The campaign tracking is what is key, because it enables measuring more than simply “clicks:” whether the visitors are first-time visitors to the site or returning visitors, how deeply they engaged with the site, and whether they took any meaningful action (conversions) on the site
  • “Organic” referrals — overall referrals to the site from twitter.com. Depending on which web analytics tool you are using on your site, this may or may not include the clickthroughs from links tweeted by the brand.

By looking at referral traffic, you can measure both the volume of traffic to the site and the relative quality of the traffic when compared to other referral sources for the site.

(If the volume of that traffic is sufficiently high to warrant the effort, you may even consider targeting content on the landing page(s) for Twitter referral traffic to try to engage visitors more effectively– you know the visitor is engaged with social media, so why not test some secondary content on the page to see if you can use that knowledge to deliver more relevant content and CTAs?)

Word Clouds with Wordle

While this isn’t a technique for performance management, it’s hard to resist the opportunity to do a qualitative assessment of the tweets to look for any emerging or hot topics that warrant further investigation. Because all of the tweets have been captured, a word cloud can be interesting (see my eMetrics post for an example). Hands-down, Wordle makes the nicest word clouds out there. I just wish it was easier to save and re-use configuration settings.

One note here: you don’t want to just take all of the tweet content and drop it straight into Wordle, as the search criteria you used for the tweets will dwarf all of the other words. If you first drop the tweets into Word, you can then do a series of search and replaces (which you can record as a macro if you’re going to repeat the analysis over time) — replace the search terms, “RT,” and any other terms that you know will be dominant-but-not-interesting with blanks.

Not Exactly the Holy Grail…

Do all of these techniques, when appropriately combined, provide near-perfect measurement of Twitter? Absolutely not. Not even close. But, they’re cheap, they do have meaning, and they beat the tar out of not measuring at all. If I had to pick one tool that I was going to bet on that I’d be using inside of six months for more comprehensive performance measurement of Twitter, it would be Twitalyzer. It sure looks like it’s come a long way in the 6-9 months since I last gave it a look. What it does now that it didn’t do initially:

  • Offers a much larger set of measures — you can pick and choose which measures make sense for your Twitter strategy
  • Provides clear definitions of how each metric is calculated (less obfuscated than the definitions used by Klout)
  • Allows trending of the metrics (including Lists and Followers).

Twitalyzer, like Klout, and Twitter Counter and countless other tools, is centered on the Twitter account itself. As I’ve described here, there is more going on in Twitter that matters to your brand than just direct engagement with your Twitter account and the social graph of your followers. Online listening tools such as Nielsen Buzzmetrics can provide keyword-based monitoring of Twitter for brand mentions and sentiment — this is not online listening per se, really, but it is using online listening tools for measurement.

For the foreseeable future, “measuring Twitter” is going to require a mix of tools. As long as the mix and metrics are grounded in clear objectives and meaningful measures, that’s okay. Isn’t it?

Adobe Analytics, General

Tracking Product Returns

If you sell products or services on your website, you are probably working diligently to be sure that you are tracking the appropriate Orders, Units and Revenue associated with each product sold (you may even be doing some advanced stuff like I described here). Doing this allows you to see all sorts of wonderful things, like what pages lead to sales and what online campaigns have better ROI than others. However, one formidable challenge that web analysts don’t like to talk about is Product Returns. How often have you bought something online only to ship it back or return it to a brick & mortar store associated with the website? If customers return products in significant enough numbers, all of the great online data you have collected may be inaccurate.

I have seen some companies apply a “rule of thumb (dumb?)” in which they discount sales by 20% across the board to account for returns, but how does that help you determine if a specific marketing campaign is good or bad when your SiteCatalyst reports only show the good? By not tying product returns directly to their corresponding online sales, your web analytic reports will be inherently flawed. The truth is that I have seen very few clients who have adequately addressed this issue, so I thought I would suggest an idea that I think is an appropriate way to deal with Product Returns. Even if you don’t sell things through a shopping cart, I encourage you to read this post as its principles are applicable to any situation in which you have an online success that later is retracted in some manner offline.

Tracking Product Returns
The best way to understand the tracking of Product Returns, is through an example. Let’s pretend that are a web analyst for Apple and a first time visitor comes to the website from the Google paid search keyword “ipod” and purchases two iPods for $50 each. In SiteCatalyst, when we open the Products report, we would see $100 for the Product labeled “ipod” and if we broke it down by visit number, the same $100 would be attributed to visit number one. So far, so good…

However, let’s imagine that one of these iPods was returned to a local Apple Store. Now our reality has changed. The paid search keyword and first visit combination has now only led to $50, but SiteCatalyst still shows $100. If we create calculated metrics to compare our Revenue per Marketing spend, suddenly our ROI just got cut in half, but this is not reflected in SiteCatalyst. This might cause us to misallocate marketing dollars to campaigns that look to be good at first, but in reality are not as profitable as others when Product Returns are added to the mix (especially if we automate Paid Search using SearchCenter!). I don’t know about you, but I certainly wouldn’t want to be the one telling my boss to invest in marketing campaigns that turn out to be “duds!”

So how do we fix this mess? In order to track Product Returns, you’ll need to re-familiarize yourself with the Transaction ID feature of SiteCatalyst (I suggest re-reading this post!). At a high level, Transaction ID allows you to set an ID associated with a transaction and later upload offline metrics that are dynamically associated with any Conversion Variable (eVar) values which were active at the time the Transaction ID was set (phew!). In this case, you need to ensure that you are setting a Transaction ID value when the original online sale takes place. By doing this, you create a “key” that will allow you to upload Product Return data later and back it out of its corresponding online sale. Keep in mind that you will have to work with your Adobe account manager to get Transaction ID set-up and you’ll want to be sure that the Transaction ID table persists for as long of a time frame that you require to upload return data (default is 90 days, but this can be extended).

Once you have Transaction ID enabled and have started passing Transaction ID’s for online purchases, the next step is to create a new “Product Return Amount” [Currency] Incrementor Event. Transaction ID uploads are similar to Data Sources and, as such, import data as Incrementor Success Events. Setting up a new Incrementor Event is easily done through the Admin Console.

Once you have Transaction ID set-up and your new Product Return Amount currency Success Event, you need to use Data Sources to generate a Product Returns template which you can populate and upload on a daily/weekly/monthly basis. This file will contain the following columns:

  1. Date – I suggest you use the date that the original purchase took place, not the date of the return, so there is no lag time. NOTE: Please keep in mind that there is currently a SiteCatalyst restriction that you cannot upload a Transaction ID file that has dates spanning more than 90 days. You can upload dates that are more than 90 days old, but the date ranges for the entire file upload cannot be more than 90 days (kind of lame in my opinion!).
  2. Transaction ID – The ID associated with the original online sale
  3. Product Name/ID – Same value that is passed to the Products Variable during the original online purchase
  4. Product Return Amount – This is the total $$ amount per product that is being returned

When you are done, your upload file might look something like this:

Using Product Returns in SiteCatalyst Reports
Once you have successfully uploaded some Product Return data, it is time to see how all of this looks in SiteCatalyst. To do this, open the Products report and add Revenue and our new Product Return Amount metrics. The report should look like this:

Next, we can create a Calculated Metric which subtracts the Product Return Amount from Revenue to create a “Net Revenue” metric as shown here:

Finally, since Transaction ID allows you to apply the eVar values that were associated with the original transaction to the new “return” transaction, you can even use Subrelation break downs for the Products report by Visit Number and see both Revenue and Returns (and the Calculated Metric) by Visit Number (pretty cool huh?)!

Orders & Units (Advanced)
For those that are a bit more advanced, I wanted to let you know that the above solution does not back out Orders or Units for Product Returns. Backing out Orders is a bit tricky since you may only want to remove the Order if the entire Order is returned. Units is a bit easier as you can simply create a second Incrementor Success Event for “Returned Units” as we did above. However, I suggest that you start with Revenue since most of your questions around Product Returns will be related to Revenue.

Finally, SiteCatalyst does provide an out of the box Data Sources template for Product Returns which can be found in the Data Sources Manager:

However, I have not used this template myself and I would have the following potential concerns:

  1. I don’t believe that this template uses Transaction ID which can be problematic as you will be unable to use the Product Return metric with all of your pre-existing eVar reports
  2. It looks like this template uses the same Revenue, Orders and Units Success Events to back out Product Return data. I feel like this can be a recipe for disaster if something goes wrong. With my approach, the worst case scenario (if you upload some bad Product Return data) is that your new Product Return metrics are temporarily off. If you use the standard Revenue, Orders and Units metrics, a mistake can be fatal and hidden amongst your normal online metrics (I never mess with Revenue, Orders or Units!).

For these reasons, I suggest you talk to your Adobe Account Manager if you want to pursue this route.

Final Thoughts
So if Product Returns are something that you have to deal with, the above is my suggested way to handle them. Those of you who work in Retail day in and day out may have come up with some other ways to deal with Product Returns, so if there are other best practices out there, please leave a comment here. Thanks!

Analytics Strategy, Social Media

eMetrics Washington, D.C. 2010 — Fun with Twitter

I took a run at the #emetrics tweets to see if anything interesting turned up. Rather than jump into Nielsen Buzzmetrics, which was an option, I just took the raw tweets from the event and did some basic slicing and dicing of them.

[Update: I’ve uploaded the raw data — cleaned up a bit and with some date/time parsing work included — in case you’d like to take another run at analyzing the data set. It’s linked to here as an Excel 2007 file]

The Basics of the Analysis

I constrained the analysis to tweets that occurred between October 4, 2010, and October 6, 2010, which were the core days of the conference. While tweets occurred both before and after this date range, these were the days that most attendees were on-site and attending sessions.

To capture the tweets, I set up a Twapper Keeper archive for all tweets that included the #emetrics hashtag. I also, certainly, could have simply set up an RSS feed and used Outlook to capture the tweets, which is what I do for some of our clients, but I thought this was a good way to give Twapper Keeper a try.

The basic stats: 1,041 tweets from 218 different users (not all of these users were in attendance, as this analysis included all retweets, as well as messages to attendees from people who were not there but were attending in spirit).

Twapper Keeper

Twapper Keeper is free, and it’s useful. The timestamps were inconsistently formatted and/or missing in the case of some of the tweets. I don’t know if that’s a Twapper Keeper issue, a Twitter API issue, or some combination. The tool does have a nice export function that got the data into a comma-delimited format, which is really the main thing I was looking for!

Twitter Tools Used

Personally, I’ve pretty much settled on HootSuite — both the web site and the Droid app — for both following Twitter streams and for tweeting. I was curious as to what the folks tweeting about eMetrics used as a tool. Here’s how it shook out:

So, HootSuite and TweetDeck really dominated.

Most Active Users

On average, each user who tweeted about eMetrics tweeted 4.8 times on the topic. But, this is a little misleading — there were a handful of very prolific users and a pretty long tail when you look at the distribution.

June Li and Michele Hinojosa were the most active users tweeting at the conference by far, accounting for 23% of all tweets between the two of them directly (and another 11% through replies and retweets to their tweets, which isn’t reflected in the chart below — tweet often, tweet with relevancy, and your reach expands!):

Tweet Volume by Hour

So, what sessions were hot (…among people tweeting)? The following is a breakdown of tweets by hour for each day of the conference:

Interestingly, the biggest spike (11:00 AM on Monday) was not during a keynote. Rather, it was during a set of breakout sessions. From looking at the tweets themselves, these were primarily from the Social Media Metrics Framework Faceoff session that featured John Lovett of Web Analytics Demystifed and Seth Duncan of Context Analytics. Of course, given the nature of the session, it makes sense that the most prolific users of Twitter attending the conference would be attending that session and sharing the information with others on Twitter!

The 2:00 peak on Monday occurred during the Vendor Line-Up session, which was a rapid-fire and entertaining overview of many of the exhibiting vendors (an Elvis impersonator and a CEO donning a colonial-era wig are going to generate some buzz).

There was quite a fall-off after the first day in overall tweets. Tweeting fatigue? Less compelling content? I don’t know.

Tweet Content

A real challenge for listening to social media is trying to pick up hot topics from unstructured 140-character data. I continue to believe that word clouds hold promise there…although I can’t really justify why a word frequency bar chart wouldn’t do the job just as well.

Below is a word cloud created using Wordle from all 1,041 tweets used in this analysis. The process I went through was that I took all of the tweets and dropped them in MS Word and then did a handful of search-and-replaces to remove the following words/characters:

  • #emetrics
  • data
  • measure
  • RT

These were words that would come through with a very strong signal and dominate potentially more interesting information. Note: I did not include the username for the person who tweeted. So, occurrences of @usernames were replies and retweets only.

Here’s the word cloud:

What jumped out at me was the high occurrence of usernames in this cloud. This appears to be a combination of the volume of tweets from that user (opening up opportunities for replies and retweets) and the “web analytics celebrity” of the user. The Expedia keynote clearly drove some interest, but no vendors generated sufficient buzz to really drive a discussion volume sufficient to bubble up here.

As I promised in my initial write-up from eMetrics, I wasn’t necessarily expecting this analysis to yield great insight. But, it did drive me to some action — I’ve added a few people to the list of people I follow!

Analytics Strategy

Gilligan's eMetrics Recap — Washington, D.C. 2010

I attended the eMetrics Marketing Optimization Summit earlier this week in D.C., and this post is my attempt to hash out my highlights from the experience. Of all the conferences I’ve attended (I’m not a major conference attendee, but I’m starting to realize that, by sheer dint of advancing age, I’m starting to rack up “experience” in all sorts of areas by happenstance alone), this was one that I walked away from without having picked up on any sort of unintended conference theme. Normally, any industry conference is abuzz about something, and that simply didn’t seem to be the case with this one.

(In case you missed it, the paragraph above was a warning that this post will not have a unifying thread! Let’s plunge ahead nonetheless!)

Voice of the Customer

It’s good to see VOC vendors aggressively engaging the “traditional web analytics” audience. Without making any direct effort, I repeatedly tripped over Foresee Results, iPerceptions, OpinionLabs, and CRM Metrix in keynotes, sessions, the exhibit hall, and over meals.

My takeaway? It’s a confusing space. Check back in 12-18 months and maybe I’ll be able to pull off a post that provides a useful comparison of their approaches. If I had my ‘druthers, we’d pull off some sort of bracketed Lincoln-Douglas style debate at a future eMetrics where these vendors were forced to engage each other directly, and the audience would get to vote on who gets to advance – not necessarily judging which tool is “better” (I’m pretty sure each tool is best-in-class for some subset of situations…although I know at least one of the vendors above who would vigorously tell me this is not the case), but declaring a winner of each matchup so that we would get a series of one-on-one debates between different vendors that would be informative for the audience.

Cool Technology

I generally struggle to make my way around an exhibit hall, so I didn’t come anywhere close to covering all of the vendors This wasn’t helped by the fact that I talked to a couple of exhibitors early on that were spectacularly unappealing. That wasn’t exactly a great motivator for continuing the process. There were, however, several tools that intrigued me:

  • Ensighten – if you’re reading this blog, then chances are you read “real” blogs, too, and you likely caught that Eric Peterson recently wrote a paper on Tag Management Systems (sponsored by Ensighten). It’s worth a read. Ensighten was originally developed in-house at Stratigent and then spun off as a separate business with Josh Manion at the helm. Their corny (but highly effective) schtick at the conference was that they were starting a “tagolution” (a tagging revolution). That gave them high visibility…but I think they’ve got the goods to back it up. Put simply, you deploy the Ensighten javascript on your site instead of all of the other tags you need (web analytics, media tracking, VOC tools, etc.). When the page loads, that javascript makes a call to Ensighten, which returns all of the tags that need to be executed. Basically, you get to manage your tags without touching the content on your site directly. And, according to Josh, page performance actually improves in most cases (he had a good explanation as to why — counter-intuitive as it seems). Very cool stuff. Currently, they’re targeting major brands, and the price point reflects this – “six figures” was the response when I asked about cost for deploying the solution on a handful of domains. Ouch.
  • DialogCentral – this is actually an app/service from OpinionLabs, and I have no idea what kind of traction it will get. But, as I stood chatting with the OpinionLabs CIO, I pulled out my Droid and had had a complete DialogCentral experience in under a minute. The concept? Location-based services as a replacement for “tell us what you think” postcards at physical establishments. You fire up their app (iPhone) or their mobile site (dialogcentral.com will redirect to the mobile site if you visit it with a mobile device). DialogCentral then pulls up your location and nearby establishments (think Foursquare, Gowalla, Brightkite-type functionality to this point), and then lets you type in feedback for the establishment. That feedback then gets sent to the establishment, regardless of whether the venue is a DialogCentral customer. Obviously, their hope is that companies will sign on as customers and actually promote the feedback mechanism in-store, at which point the feedback pipeline gets much smoother. It’s an intriguing idea — a twist-o’-the-old on all of the different “publicly comment on this establishment” aspects of existing services.
  • Clicktale – these guys have been around for a while, and I was vaguely familiar with them, but got an in-depth demo. They use client-side code (which, presumably, could be managed through Ensighten — I’m just sayin’…) to record intra-page mouse movements and clicks. They then use that data to enable “replays” of the activity as well as to generate page-level heatmaps of activities and mouse placement. Their claim (substantiated by research) is that mouse movements are a pretty tight proxy for eye movement, so you get a much lower cost / broadly collected set of (virtual) eye-tracking data. And, the tool has all sorts of triggering and filtering capabilities to enable honing in on subsets of activity. Pretty cool stuff.
  • ShufflePoint – this wasn’t an exhibiting vendor, but, rather, the main gist of one of the last sessions of the conference. The tool is a poor man’s virtual-data-mart enabler. Basically, it’s an interface to a variety of tool APIs (Google Analytics, Google Adwords, Constant Contact, etc. – Facebook and Twitter are apparently in the pipeline) that allows you to build queries and then embed those queries in Excel. I’ve played around with the Google Analytics API enough to get it hooked into Excel and pulling data…and know that I’m not a programmer. Josh Katinger of Accession Media was the presenter, and he struck me as being super-pragmatic, obsessive about efficiency, and pretty much bullshit-free (I found out after I got to the airport that a good friend of mine from Austin, Kristin Farwell, actually goes wayyy back with Josh, and she confirmed that this was an accurate read). We’ll be giving ShufflePoint a look!

Social Media Measurement

I was expecting to hear a lot more on social media measurement at the conference…but it really wasn’t covered in-depth. Jim Sterne kicked off with a keynote on the subject (he did recently publish a book on the topic, which is now sitting on my nightstand awaiting a read). And, there was a small panel early on the first day where John Lovett got to discuss the framework he developed with Jeremiah Owyang (which is fantastic) this past spring. But, other than that, there really wasn’t much on the subject.

MMM and Cross-Channel Analytics

Steve Tobias from Marketing Management Analytics conducted a session that focused on the challenges of marketing mix modeling (MMM) in a digital world. I felt pretty smart as he listed multiple reasons why MMM struggles to effectively incorporate digital and social media, because many of his points mirrored what I’ve put together on the exact same subject (to be clear, he didn’t get his content from me!). It was good to get validation on that front from a true expert on the subject.

Where things got interesting, though, was when Steve talked about how his company is dealing with these challenges by supplementing their MMM work (their core strength) with “cross-channel analytics.” By “cross-channel analytics,” he meant panel-based measurement. Again, I felt kinda’ smart (and, really, it’s all about me and my feelings, isn’t it?), as I keep thinking (and I’ve got this in some internal presentations, too), that panel-based measurement is going to be key in truly getting a handle on cross-channel/cross-device consumer interactions and their impact.

The People

One of the main reasons to go to a conference like eMetrics is the people — catching up with people you know, meeting people you’ve only “known” digitally, and meeting people you didn’t know at all.

For me, it was great to again get to chat with Hemen Patel from CRM Metrix, John Lovett from Analytics Demystified, Corry Prohens from IQ Workforce, and the whole Foresee Results gang (Eric F., Eric H., Chris, Maggie,…and more). And, it wound up being a really special treat to see Michelle Rutan, who I take credit for putting on the web analytics career path way back when we worked at National Instruments together…and she was presenting (as an amusing aside, I credit Michelle’s husband, Ryan — although they weren’t even dating at the time — as being pretty key to helping me understand the mechanics of page tagging; he’s credited by name in one of the most popular posts on this blog)!

I actually got to meet Stéphane Hamel in person, which was a huge treat (I saw a lot of other web analytics celebrities, but never wound up in any sort of conversation with them — maybe next time), as well as Jennifer Day, who I’ve swapped tweets with for a while.

Digital Analytics folk are good peeps. That’s all there is to it.

Twitter (and Twapper Keeper) Means More to Come!

I actually managed to have the presence of mind to set up a Twapper  Keeper archive for #emetrics shortly before the conference started, and I’m hoping to have a little fun with that in the next week or two. We’ll see if any insights (I’m not promising actionable insights, as I’ve decided that term is wildly overused) emerge. I picked up a few new people to follow just based on the thoroughness and on-pointed-ness of their tweets — check out @michelehinojosa (who also is blogging her eMetrics takeaways) if you’re looking to expand your follower list.

It was a good conference!

Adobe Analytics, Analytics Strategy, General, Reporting

Presentations from Analytics Demystified

This week is somewhat bittersweet for me because it marks the very first time I have missed an Emetrics in the United States since the conference began. And while I’m certainly bummed to miss the event, knowing that my partner John is there representing the business makes all the difference in the world. If you’re at Emetrics this week, please look for John (or Twitter him at @johnlovett) and say hello.

If you’re like me and not going to the conference perhaps I can interest you in one of the four (!!!) webcasts and live events I am presenting this week:

  • On Tuesday, October 5th I will be presenting my “Web Analytics 201” session to the fine folks at Nonprofit Technology Network (NTEN) who we partner with on The Analysis Exchange. You need to be a NTEN member to sign up but if you are I’d love to talk with you!
  • On Wednesday, October 6th I will be doing a free webcast for all our friends in Europe talking about our “no excuses” approach towards measuring engagement in the online world. Sponsored by Nedstat (now part of comScore) all attendees will get a free copy of our recent white paper on the same topic.
  • Also on Wednesday, October 6th (although at a slightly more normal time for me) I will be presenting our Mobile (and Multi-channel) Measurement Framework with both our sponsor OpinionLab and a little consumer electronics retailer you may have heard of … Best Buy! The webcast is open to everyone and all attendees will also get a copy of our similarly themed white paper.
  • On Thursday, October 7th I will be at the Portland Intensive Social Media workshop presenting with Dean McBeth (of Old Spice fame) and Hallie Janssen from Anvil Media. I will be presenting John and Jeremiah’s Social Marketing Analytics framework and am pretty excited about the event!

All-in-all it promises to be a very busy week presenting content so I hope to hear from some of you on the calls or see you in person on Thursday.

Adobe Analytics, General

Hidden SiteCatalyst Features

(Estimated Time to Read this Post = 4 Minutes)

One of the funny things about SiteCatalyst (you will notice I can’t yet bring myself to call it Adobe SiteCatalyst!) is that there are some really cool features that are hidden. In some cases, it almost seems like someone has gone out of their way to hide them, but I like to look at these “hidden gems” as a sort of rite of passage. In this post I will share some of the ones I have found and hope that maybe you know of others so that all of us can learn! Also, if you haven’t read my old blog post on SiteCatalyst Time Savers, I encourage you to do so!

The Magic Triangle and Checkboxes
If you are like me, it may seem like you spend most of your day adding/removing metrics from reports! This can be a very time consuming process, so you might as well be as efficient as possible. However, I often find that new SiteCatalyst users add extra steps to the process because they don’t know a few easy tricks in the Metrics window. The first trick is that you can change the column that is used for sorting by clicking the [very] little triangle next to each metric. It amazes me how many people add metrics, wait for the report to load and then click on a column to sort and wait for the report to load again! Multiply that by twenty reports and it becomes a real time suck! Instead, simply click the triangle until it turns green (soon to be Adobe red?) and you are done!

But wait! There’s more…You will also notice that there are a bunch of check boxes next to each metric. Those check boxes are used to choose which metrics you want to graph with your report. You don’t have to graph every metric in the report, which may confuse your audience. Also, I find that many clients don’t take advantage of the fact that you can display two graphs per report. To do this, all you need to do is check off one of the boxes on the left and right side. This is helpful if some of your metrics are numbers and them are percentages. It is the closest SiteCatalyst comes to a secondary axis you may be used to in Excel.

Remove Subrelation BreakDowns
If you frequently use eVar Subrelation reports, you may find that after breaking one eVar down by another eVar, you want to go back to the report before it was subrelated. For example, let’s say you have opened aTraffic Driver report and broken it down by Offer Type as shown here:

Now let’s say you change the date range and some other report settings and then decide you want to just see Traffic Driver Type by itself again. Unfortunately, if you use the trusty “Back” button in your browser, you will have to re-do all of those customized settings. However, there are actually two ways to remove this subrelation without losing any work.

The first way to do this is to click on the “Broken Down by:” link shown in red above. Once you click on this, you will see a list of all of your variables and you can choose the bottom-most one labeled “None.” The other way is to click the green magnifying glass icon you used to create the Subrelation and do the same thing as shown here:

Double Your Searching Pleasure
Another thing I have noticed that a lot of SiteCatalyst users don’t know is that you can add search criteria to two different variables if you are using a Subrelation report. To do this, click on the Advanced Search link and then you can use the drop down boxes to choose which variable/search term combination you want:

In this case, I have chosen to filter for all Traffic Driver Types containing “SEO” and can proceed to enter my search criteria for Offer Type…

Inherit Segments
If you use DataWarehouse or ASI, you probably spend a lot of time creating Segments. If so, you may find times where you want to re-use some parts of a segment you have already created. When I first started using SiteCatalyst, I did this by printing out my segments and re-creating them manually. This is both time-consuming and prone to error, so I found the trick to do this more efficiently:

There it is! See how easily you can copy an existing segment? Do you see it? If not, would you believe me if I told you that there are actually two different ways to re-use segments on the above screen?

The first way to do this is to click the icon to the right of the Segment title. This will pop-up a new window which allows you to pick an existing segment you want your new segment to be based upon.

The second way to do this is to use the Segment Library. You can access the library by clicking its name next to the “Components” item. The Library is used to store commonly use segment building blocks. In the example below, I have created a Page View container that looks for Pages where the IP Address Geography is in the United States. By dragging this over to the Library, I can re-use this anytime I am creating a new segment.

Filter Report Suites
If you have Admin rights to your SiteCatalyst implementation and deal with a lot of report suites, the Admin Console quickly becomes one of your best friends. One of the most time consuming Admin Console tasks is finding the report suites you are looking for among all of your report suites. I frequently see people scanning up and down over and over hunting for the report suites they need. Fortunately, there is a much better way that is somewhat hidden – using the “Saved Searches” feature of the Admin Console. Using this feature you can create a filter to find report suites such that even if you add new ones, they will be added to your saved search if they meet the criteria.

Here is a real-life example. When I joined Salesforce.com, we had a lot of report suites and I began creating new report suites. When I created the new report suites, I simply added the phrase “New” to my new report suites titles. Once I did this, I clicked on the “Add” link within the Saved Searches area of the report suite manager and created a “Saved Search” rule like this:

Keep in mind that this is a very basic rule. You can actually add multiple criteria items and can build rules that take into account any of the following report suite criteria:

Lastly, if you are an Admin, be sure to read my past blog post with even more Admin Console Tips.

Final Thoughts
So there you have it. As you can see, none of these items are critical showstoppers, but I have found that knowing them can help speed up your day and give you SiteCatalyst bragging rights! Do you know of others? If so, please share them here as comments!

Analytics Strategy, Excel Tips, Presentation

Data Visualization Tips and Concepts (Monish Datta calls it "stellar")*

Columbus Web Analytics Wednesday was sponsored by Resource Interactive last week, and it was, as usual, a fun and engaging event:

Web Analytics Wednesday -- Attendees settling in

We tried a new venue — the Winking Lizard on Bethel Road — and were pretty pleased with the accommodations (private room, private bar, very reasonable prices), so I expect we’ll be back.

Relatively new dad Bryan Cristina had a child care conflict with his wife…so he brought along Isabella (who was phenomenally calm and well-behaved, and is cute as a button!):

Bryan and Isabella

I presented on a topic I’m fairly passionate about — data visualization. The presentation was well-received (Monish Datta really did tweet that it was “stellar”)  and generated a lot of good discussion. I had several requests for copies of the presentation, so I’ve modified it slightly to make it more Slideshare-friendly and posted it. If you click through on the embedded version below, you can see the notes for each slide by clicking on the “Notes on Slide X” tab underneath the slideshow, or you can download the file itself (PowerPoint 2007), which includes notes with each slide (I think you might have to create/login to a Slideshare account, which it looks like you can do quickly using Facebook Connect).

 

 

 

 

I had fun putting the presentation together, as this is definitely a topic that I’m passionate about!

* The “Monish Datta” reference in the title of this post, while accurate, is driven by my never-ending quest to dominate search rankings for searches for Monish. I’m doing okay, but not exactly dominating.

http://b.scorecardresearch.com/beacon.js?c1=7&c2=7400849&c3=1&c4=&c5=&c6=

Reporting

Department Store KPIs (an analogy)

A couple of weeks ago, I had a conversation with the newest member of the analytics team at Resource Interactive, Matt Coen. I shared with him my “Measuring digital marketing is like measuring the Mississippi River” analogy, and he, in turn, shared with me his department store analogy. I’m a big fan of using stories and analogies to get across fundamental measurement concepts, so, with his permission, I’m passing along his perspective (and, of course, in the translation from a verbal story to the written word, I’m finding that I’m taking some liberties!).

The story is a great illustration of two things:

  • How key performance indicators (KPIs) generally cannot live in isolation – driving a single KPI to a certain result is easy, but businesses operate on more than one dimension (for instance, total sales can be boosted by dropping the price well below cost…but that kills profitability)
  • Why no company can have a single set of KPIs. The appropriate KPIs depend on what and who is being measured.

Onto the Story

Let’s take a fictional department store. At this store, each department has a department manager who is responsible for all aspects of the department, including the department’s P&L. In addition, all of the departments have a KPI regarding inventory turnover – if any product sits on the shelves for too long, the store loses money. All of the departments have this KPI because, overall, the store has an inventory turnover KPI.

The office supplies department manager is seeing his inventory turnover suffer, and, by digging into the data, he realizes that pens are killing him – no one is buying them, and it’s hurting his turnover rate.

He goes to the store manager and tells him, “I’m having trouble moving pens, and that’s hurting my inventory turnover rate. You may not be seeing it at the overall store level, but it’s got to be negatively impacting that KPI. I need to move pens to the checkout line display.”

The manager scratches his head and agrees to the change – inventory turnover is one of his KPIs, the department manager is being data driven, and he’s even come to the store manager with a proposed solution! Woo-hoo! He promptly instructs his team to remove the candy from the checkout lines and replace them with pens.

Sure enough, pen sales pick up, and the department manager is thrilled.

But, the candy department manager immediately shows up in the store manager’s office and tells him, “My sales are way below target. When I developed my forecast, it was with the assumption that candy would be at the checkout lines. It’s a major impulse buy and that’s where 25% of my department sales occur!”

The store manager really didn’t need this additional headache. He was already seeing a dip in the overall store margin, and he’d realized that he might have acted too hastily when responding to the office supplies department manager’s request, because, not only is candy much more of an impulse buy – so the increase in pen sales didn’t make up for the loss in candy sales – but candy is a higher margin product.

When the store manager agreed to the change, he was making a decision based on how it would impact someone else’s KPIs. And, he focused on a single KPI – inventory turnover – rather than complementary KPIs – inventory turnover and margin.

This analogy can be applied to any number of marketing scenarios. An easy one is a web site, where the owner of a niche site section makes a case for featuring that section very prominently on the home page (the department store checkout line display) in the interest of driving more traffic to his site.

It’s a useful tale!

Analytics Strategy

Minimize Robot Traffic

Robots are cool. I like robots when they build cars, try to plug oil spills and clean carpets. The only types of robots I don’t like are the ones that hit websites repeatedly and throw off my precious web analytics data! Do you have a problem with these types of robots? Would you know how to see if you do? I find that many web analytics customers don’t even know how to see this, so in this post I will share what I do to monitor robots and hope that others out there will share other ways they deal with robots.

Why Should I Care About Robots?
This is often the first question I get. Who cares? Here are my reasons for caring about minimizing robots hitting your site:

  1. If you use Visits or Unique Visitors as part of any of your website KPI’s (i.e. Revenue/Unique Visitor), you should care because robots are inflating your denominator and dragging your conversion rates down
  2. If you are tasked with reducing Bounce Rates on your site, you should care as robots will often be seen as bounces
  3. Omniture (and other web analytics vendors) often bill you by website traffic (server calls) so you may be paying $$$ for junk data
  4. Often times web analytic KPI’s have razor-thin differences month over month and having a lot of garbage data can mean the difference between making a good and bad website business decision

Do I Have a Problem?
The first step is to identify if you have a problem with robots. Unfortunately, SiteCatalyst does not currently have an “out-of-the-box” way to alert you if you have a problem (@VaBeachKevin has added this to the Idea Exchange so please vote!), but in the meantime, here is my step-by-step approach to determining this:

  • Create a recurring DataWarehouse report that sends you Page Views and Visitors for each IP address hitting your site (If you store the Omniture Visitor ID in an sProp, I would use that in place of IP address). This can be daily, weekly or monthly depending on how much traffiic your website receives. I sometimes add the Country/City as well (you’ll see why later).

  • When you receive this report, it should look something like this:

  • Once you have the data, I create a calculation which divides Page Views by Visitors and then sort by that column (if you have a lot of data from different days/weeks, you can create a pivot table). The result should look like the report below where you will start to see which IP addresses are viewing a lot of pages on your site per visitor. Keep in mind that this doesn’t mean they are all bad. It is common for small companies or individuals to share IP addresses. The goal of this step is just to identify the IP addresses that might be issues. In the example below, you can see that the the top two IP addresses appear to be a bit different than the rest. While it may make you feel good that these unique visitors liked your website so much they viewed thousands of pages each, you might be fooling yourself!

  • Once you have this list, I like to do some research on the the top IP Address offenders. You can do this via a basic Whois IP Lookup or you can invest in a reverse IP lookup service.

What Do I Do If I Find Robots?
If after reviewing the top offending IP addresses you find that you do, in fact, have a robot hitting your site, you have a few options:

  1. Work with your IT group to exclude these IP addresses from hitting your website. This is your best option since it will be the most reliable and reduce your web analytics server call cost.
  2. Work with Omniture’s Engineering Services team to create a DB Vista Rule that will move these website hits to a new report suite so it will not pollute your data. The best part of this option is that you don’t have to engage with your IT team and you can add/remove IP addresses anytime you want via FTP. Unfortunately, you will still be hit with server call charges for this (not to mention the cost of the DB Vista Rule!), but if you also pass data to Omniture Discover, you might save money there by not passing bad data to Discover.
  3. Work with Omniture’s Engineering Services team to build a custom solution for dealing with robots…

Employee Traffic
While I don’t want to imply that your co-workers are robots, I wanted to mention employee traffic in this post as well since it is tangentially related. I find that many Omniture customers don’t exclude their own employees from their web analytics reports. This can be a huge mistake if you have a lot of employees or have employees who actively use the website. For example, at my employer (Salesforce.com), we use our website to log into our internal systems which are all run on Salesforce.com! This means that we have thousands of employees hitting our website every day to log in to our “cloud” applications and that traffic should not count towards our marketing/website goals. Therefore, we manually exclude all employee traffic from our reports by IP address to minimize the impact of employee traffic impacting our KPI’s. While we don’t consider this to be robot traffic, we address it in the same manner by passing employee traffic to its own report suite. One cool by-product of placing employee traffic in its own report suite is that you can see how often your own employees are using your website so you can show management that the dollars they give you serve multiple audiences!

Final Thoughts
As I stated in the beginning of this post, this is just one way to investigate and deal with robots. If you have other techniques, please share them here! Thanks!

Analytics Strategy

Free white paper on Tag Management Systems

This last week I was in London thanks to the good graces of our friends at Tealeaf to deliver a keynote speech at their EMEA customer conference. After the event, both reporters and conference attendees asked me “What is the most important technology trend in web analytics today?”

I have been asked this hundreds of times in my career as an analyst and consultant and the answer used to be tricky. In the past I’ve opined “multichannel integration”, “segmentation”, “application usability” and even “none, it’s about people and process, not technology.”

This time, however, my answer was clear: Tag Management Systems.

Tag management has become a nightmare for many companies.  As we outlined in our white paper with ObservePoint on the need for a “Chief Data Officer”, tagging and data collection has gotten out of control in companies of all sizes. Information Technology supports one set of tag-based tools, marketing deploys their own stuff via content management systems (CMS), advertising and other individual stakeholders drop their tags, and before you know it you have a dozen or more scripts included at various points across your site.

In a way, because of fragmentation in the marketplace this situation was inevitable. But it doesn’t have to be this way.

Emerging tag management systems (TMS) are rapidly transforming the data capture and technology deployment landscape, replacing inefficient, individual installations with a “one stop shop” able to manage any number of tag-based technologies via a single user interface. Early adopters of these systems are reporting a profound transformation of both their ability to manage data capture and their relationship with Information Technology.

From a web analytics perspective this is what we call a “win/win.”

One of these vendors is Ensighten, a company founded by a group of folks who have a long established reputation in digital measurement. Their CEO Josh Manion and I go back pretty far, and so when he told me about their platform I was immediately intrigued but somewhat skeptical.

I had already seen nearly all of the competing solutions in the market and walked away a variety of concerns. I’d even gone so far as trying to establish an “Open Tag Alliance” initiative with one vendor, which unfortunately collapsed due to time constraints.

Needless to say, I was impressed with Ensighten, so much so that I asked Josh if we could partner with him (something we rarely do with technology vendors.) He agreed, and so we are proud to announce that we are the first deployment and integration partner to sign up with Ensighten.

In support of our partnership we agreed to write a paper detailing what we see as the advantages of tag management systems. Titled “The Myth of the Universal Tag and the Future of Digital Data Collection”, this short paper outlines the need for TMS, the rationale behind deployments, and the opportunity for return on the investment. The paper is freely available now at the Ensighten web site or you can write us directly for a complimentary copy.

Readers should note that there are a handful of tag management solutions in the market today. At Analytics Demystified we believe the growth in the sector is validation of the opportunity — each of these companies have good stories to tell and an expanding customer base.

You should consider a tag management system if you are:

  • Frustrated with the “one tag, one project, one timeline” model of tag deployment;
  • Switching vendors and looking to gain leverage over future deployments;
  • Heavily invested in Flash but have long struggled to measure the technology;
  • Managing globally distributed sites but have little centralized control over tags;
  • Looking to add Q/A and workflow management to your tag deployments;
  • Concerned at all about the quality and data accuracy from your web analytics.

If any of these criteria apply to you I would strongly encourage you to give John or I a call. We’ll be more than happy to walk you through the current tag management system vendor landscape at no charge and point you towards whatever solutions seems right for you.

We welcome you to the age of tag management systems and we hope you will join us in welcoming Ensighten to the market.

Your next actions:

Analysis, Reporting

Dear Technology Vendor, Your Dashboard Sucks (and it’s not your fault)

Working in measurement and analytics at a digital marketing agency, I find myself working with a seemingly (at times) countless number of of technology platforms – most of them are measurement platforms (web analytics, social media analytics, online listening), but many of them are operational systems that, by their nature, collect data that is needs to be reported and analyzed (email platforms, marketing automation and CRM platforms, gamification systems, social media moderation systems, and so on). And, not only do I get to work with these systems in action, but one of the many fun things about my job is that I constantly get to explore new and emerging platforms as well.

During a recent presentation by one of our technology partners, I had a minor out of body experience where I saw this dopey-voiced Texan turn into something of a crotchety crank. He (I) fairly politely, and with (I hope) a healthy serving of humor poured over the exchange, lit into the CEO. I didn’t know where it came from…except I did (when I pondered the exchange afterwards).

When it comes to reporting, technology vendors fall into the age-old trap of, “When all you have is a hammer, all the world looks like a nail.” The myopia these vendors display varies considerably – some are much more aware of where they fit in the overall marketing ecosystem than others – but they consistently don blinders when it comes to their data and their dashboards.

The most important data to their customers, they assume, is the data within their system. Sure, they know that there are other systems in play that are generating some useful supplemental data, and that’s fantastic! “All” the customer needs to do is use the vendor’s (cumbersome) integration tools to bring the relevant subsets of that data into their system. “Sure, you can bring customer data from your CRM system into our web analytics environment. I’ll just start writing up a statement of work for the professional services you’ll need to do that! What? You want data from our system to be fed into your CRM system, too? I’ll get an SOW rolling for that at the same time! Did I mention that my youngest child just got into an Ivy League school? Up until five minutes ago, I was sweating how we were going to pay for it!”

The vendors – their sales teams – tout their “reporting and analytics” capabilities. They frequently lead off their demos with a view of their “dashboards” and tout how easy and intuitive the dashboard interface is! What they’re really telling their prospective customers, though, is, “You’ll have one more system you’ll have to go to to get the data you need to be an effective marketer.” <groan>

Never mind the fact that these “dashboards” are always data visualization abominations. Never mind the fact that they require new users to climb a steep learning curve. Never mind that they are fundamentally centered around the “unit of analysis” that the stem is built for (a content management system’s dashboard is content-centric, while a CRM system’s dashboard is customer-centric). They only provide access to a fraction of the data that the marketer really cares about most of the time.

Clearly, these platforms need to provide easy access to their data. I’m not really arguing that dashboard and reporting tools shouldn’t be built into these systems. What I am claiming is that vendors need to stop believing (and stop selling) that this is where their customers will glean the bulk of their marketing insights. In most cases, they won’t. Their customers are going to export the data from that system and combine it (or at least look at it side by side) with data from other systems. That’s how they’re going to really get a handle on what is happening.

The CEO with whom I had the out-of-body experience that triggered this post quickly and smartly turned my challenge back on me: “Well, what is it, ideally, that you would want?” I watched myself spout out an answer that, now 24 hours later, still holds up. Here are my requirements, and they apply to any technology vendor who offers a dashboard (including web analytics platforms, which, even though they exist purely as data capture/reporting/analysis systems…still consistently fall short when it comes to providing meaningful dashboards – partly due to lousy flexibility and data visualization, which they can control, and partly due to the lack of integration with all other relevant data sources, which they really can’t):

Within your tool, I want to be able to build a report that I can customize in four ways:

  • Define the specific dimensions in the output
  • Define the specific measures to include in the output
  • Define the time range for the data (including a “user-defined” option – more on that in a minute)
  • Define whether I want detailed data or aggregated data, and, if aggregated, the granularity of the trending of that data over time (daily, weekly, monthly, etc.)

Then, I want that report to give me a URL – an https one, ideally – onto which I can tack login credentials such that that URL will return the data I want any time I refresh it. I want to be able to drop that URL into any standalone reporting environment – my data warehouse ETL process, my MS Access database, or even my MS Excel spreadsheet – to get the data I want returned to me. I’m want to be able to pass a date range in with that request so that I can pull back the range of data I actually need.

Sure, in some situations, I’m going to want to hook into your data more efficiently than through a secure http request – if I’m looking to pull down monstrous data sets on a regular basis – but let’s cross that “API plus professional services” bridge when we get to it, okay?

I’m never going to use your dashboard. I’m going to build my own. And it’s going to have your data and data from multiple other platforms (some of them might even be your competitors), and it’s going to be organized in a way that is meaningful to my business, and it’s going to be useful.

Stop over-hyping your dashboards. You’re just setting yourselves up for frustrated customers.

It’s a fantasy, I realize, but it’s my fantasy.

Analytics Strategy, Conferences/Community, General

Web Analysts Code of Ethics …

Following up on last week’s thread about how the web analytics industry is on the cusp of becoming our own worst enemy as the tide of public opinion increasingly turns against online and behavioral analytics I wanted to make good on my offer to help the Web Analytics Association. I fully support the efforts of the Association to create a solid community for web analytics professionals around the world and have long been a contributor to their work, be it turning the Web Analytics Forum (at Yahoo! Groups) over to WAA management, opening the doors for WAA participation in Web Analytics Wednesday, and providing other “behind the scenes” support when asked.

To this end I composed a preliminary “Web Analysts Code of Ethics” that I had planned to work on here in my blog (with you all) and then turn over to the Web Analytics Association. Much to my surprise, according to my partner John Lovett (who is a Board member) the Board of Directors loved the preliminary code and asked to have it publish at the Web Analytics Association blog.

Easy enough, and so I would like to redirect all of you over to the Association blog where I and the WAA both would like to hear what you have to say about this early effort. The comments have already started over there, and of course if you’re more comfortable commenting here then by all means, I welcome  that.

As I mentioned a few times in my recent Beyond Web Analytics podcast (not live until early on September 13th), I believe that we need to start advocating on our own behalf and I see this code as one small step in the right direction. Hopefully the WAA Standards Committee, the Board, and all of you out there whether you’re in the Association or not will join me in this effort to help the wider world understand what we all do (and what we do and will not do.)

So go do two things right now:

  1. Read and comment on my “Web Analysts Code of Ethics” at the WAA Blog
  2. Listen to my interview with Adam Greco and Rudi Shumpert at Beyond Web Analytics
Adobe Analytics, Analytics Strategy, General

Internal Search Term Click-Through & Exit Rates

Recently, I was re-reading one of Avinash Kaushik’s older blog posts on tracking Internal Search Term Exit Rates and realized that I had never discussed how to report on this using Omniture SiteCatalyst. In a past Internal Search post, I covered many different things you can do to track internal search on your site, but did not cover ways to see which terms are doing well and which are not. In this post I will share how you can see this so you can determine which search terms need help…

Why Track Internal Search Term Click-Through & Exit Rates?
So why should you do this? In the era of Google, we are all slowly being trained to find things through search. Many of my past clients saw the percent of website visitors using search rise over the past few years. In addition, Internal Search and Voice of Customer tools are some of the few out there where you can see the intent of your visitors. Unfortunately, most websites have horrible Internal Search results which can lead to site exits. In my previous Internal Search post I demonstrated how to track your Search Results Page Exit Rate, but that only shows you if you have a problem or not. If you do have a high Search Results Page Exit Rate, the next logical step is to determine which search terms your users think have relevant search results and which do not. Note that this is not meant to show you which terms lead directly to website exits, but rather, which terms cause visitors to use or not use the search results you offer them after they search on a particular term.

How Do You Track Internal Search Term Click-Through & Exit Rates?
Ok, so how do you do this? Follow the following implementation steps:

  • Make sure that you are setting a Success Event when visitors conduct Internal Searches on your website. Hopefully you are already doing this so in many cases this step will be done!
  • Make sure that you are capturing the Internal Search Term the visitor searched upon in an eVar variable. Again, you should be doing this (if not, shame on you!).
  • Here is where we get into uncharted territory. The next step is to set a new Success Event when visitors click on one of the items on the search results page. Depending upon the technology you use for Internal Search, this could be hard or easy. Regardless of how you actually code it, the key here is to set the second Success Event (I call it Internal Search Results Clicks) only if visitors click on a search result item (not if they click on a second page of search results or go to another page through other navigation). It is also critically important that you only set this Search Results Clicks Success Event once per search term! Do not set it every time a visitor clicks on one of the search results after using the “Back” button. If you don’t do this correctly, your Click-Through and Exit Rates will be off. This could take a few iterations to get right, but stick with it!
  • Once you have both the Internal Searches and Internal Search Results Clicks Success Events set, you can create a Calculated Metric that divides Internal Search Results Clicks by Internal Searches to see the Internal Search Click-Through Rate as shown here:

  • From there you can create the converse metric which subtracts the Internal Search Click Through Rate by “1” to come up with the Internal Search Exit Rate as shown here:

  • After this is done, you can open the Internal Search Term eVar report and add all three metrics so you see a report like this:

In this case, it looks like the “Zune” Internal Search term might need some different search result content as it has a much higher exit rate as the others. Another cool thing you can do is to create a report which trends the Internal Search CTR % or Exit % for specific Internal Search terms so you can see if they have been good/bad over time. Also, if you use SAINT Classifications to group your Internal Search Terms into buckets, you can see the report above for groups of Internal Search terms. If you vote for my idea in the Idea Exchange, you would be able to set SiteCatalyst Alerts to be notified if your top Internal Search Terms have spikes in their Click-Through or Exit Rates. You can also segment your data to see how the Internal Search rates differ when people come from Paid Search vs. SEO, etc… and even use Test&Target to try out different promotional banners on your search results page…

Finally, don’t forget that when you create a new calculated metric like the Internal Search CTR % metric described above, you also get the bonus of seeing this metric across your entire website under the My Calc Metrics area of the SiteCatalyst toolbar. Simply find this new metric and click on it and you can see your overall Internal Search Click-Through Rate regardless of internal search term. Your report will look something like this:

For The True Web Analyst Geeks
If you were bothered when I mentioned above that you should only set the Search Results Clicks Success Event once per search term, then you are my kind of person (please apply for a job with me!)! You were probably saying to yourself: “If I only count once per search term, how will I know which search terms get visitors to click on multiple search result links?” Right you are! That could be valuable information. If you want to see that as well, all you have to do is set a second Success Event each time a visitor clicks on a search result [I call this Internal Search Result Clicks (All)]. Then you can compare how many times people click on any search result to how often click in total. Here is a sample report:

In this example, you can see that the search term “api” had one click only in either scenario, but the search term “chatter” had people click on it 100% of the time and 5 times they clicked on two search result items. If you want, you can create another Calculated Metric that divides the Internal Search Result Clicks (All) by the # of Internal Searches to see how many search result clicks each term averages. In the case of “chatter” above, it would be 2.25 search result clicks per search!

Final Thoughts
If Internal Search is important to your site, make sure you are tracking it adequately so you can improve it and increase your overall website conversion. Do you have any other cool Internal Search tracking tips I haven’t covered? If so, leave a comment here…

Analytics Strategy

Acquisitions Aplenty! comScore Buys Nedstat

We’re certainly on an acquisition hot roll here in our cozy little measurement industry. This week marked yet another buy-up of a web analytics company, Netherlands based Nedstat, was acquired by comScore. The sale price was reported at $36.7 million USD, which brings the tally of measurement buy-outs including the $1.8 billion dollar Omniture acquisition last year to nearly $2.5 billion dollars by my count. Those are some good multiples on revenue since my Forrester Web Analytics Forecast didn’t peg market spending to hit even $1 billion until sometime in 2015. Granted Omniture, Unica and to some extent Coremetrics were offering more than just web analytics in their product portfolios. But regardless, measurement technologies are all the rage these days and finally, big businesses are taking note of the value of web analytics.

Foreshadowing

Some might say that comScore and Nedstat, while serving similar industries for different purposes, were running on parallel paths and that an acquisition was a plausible outcome. But before I dive into that hypothesis, first I’ll toot my own horn by mentioning that I went on record predicting this one. The good fellas at Beyond Web Analytics interviewed me on the topic of market consolidation just after the IBM acquisition of Unica and we had a good chat about it here on the podcast. The closing question asked me to look into my crystal ball and guess who would be the next acquirer in the analytics market. While I didn’t guess that it would be comScore, I did speculate that there are some very interesting and valuable technologies that exist in Europe. I mentioned both Webtrekk in Germany and Nedstat as companies that would make appealing acquisition targets. Clearly comScore must have been listening (c’mon, I jest). But one of my clients across the pond also mentioned a couple of weeks ago that Nedstat’s CEO was quoted in a German newspaper as saying that there is no longer a place for a dedicated web analytics company in this environment. I’ve been saying this since early 2009, but coming from a chief officer of a successful technology operation…Foreshadowing indeed.

The Red Herring

So, bright and early on morning of the acquisition my friend Jodi McDermott reached out to me on the news by pointing out the press release on the deal and I owe her a big thanks for that. When we spoke later that morning along with Magid Abraham, comScore founder and CEO the first question Jodi asked me was…”Were you surprised?”. Now, the dirty little secret is that analysts can never show surprise, but heck yeah I was surprised that comScore was the buyer!?! I didn’t anticipate comScore because of their Unified Digital Measurement (UDM) solution which currently handles over 500 billion transactions per month and is growing rapidly. So, they already had their own tag based measurement solution. Additionally, just under a year ago comScore announced a strategic partnership with Omniture to deliver a newly created Media Metrix 360 solution predicated on UDM that would leverage a hybrid combination of Omniture page tags and comScore’ panel based measurement.

It was brilliant actually, and demonstrated the first significant attempt to bring together advertising measurement with site-side data. Yet, just a month after this partnership was announced, Omniture was snatched up by Adobe, and I can only speculate that the momentum on the partnership was stymied. Don’t get me wrong, Media Metrix 360 still exists, and clients like Martha Stewart and the Wall Street Journal add marquee status to the initiative. Thus, I would expect that comScore will support Media Metrix 360 by continuing the partnership with Adobe’s Omniture Business Unit as well as continue development on their own proprietary solution. Whatever they choose to do, these efforts – their own hybrid UDM tags and the Omniture relationship – created a red herring for me that had me looking elsewhere. Now the real question is… Was Nielson surprised and how will they counter? Sorry friends, my crystal ball is not that good.

The Plot Twister

I saved the best for last because here’s where the plot starts to get really interesting. comScore has stated that its acquisition interests in Nedstat are to better serve the media and publishing industries. Web analytics and site-side measurement has long been focused on the transaction and sites that don’t have traditional online transactions are left to quantify success by custom fitting solutions to meet their needs. With most web analytics solutions you’re forced to follow the conversion funnel through to a transaction (or not) and attempt tie things together or launch remarketing efforts from there. But when there’s no transaction at the end of the visit, then many traditional web metrics have very little resonance to the business.

Nedstat has long been focused on key topics like engagement and rich media measurement – metrics that matter to publishers. Now with the acquisition by comScore who has a stronghold within many media companies (not to mention a reserved line item in their budgets) they can create a very different value proposition for media companies looking to quantify metrics for their advertisers as well as optimize the experience for their visitors. I tend to agree with Magid who stated that this new paradigm for publishers is likely to create a natural segmentation in the market. With stalwart web analytics firms (albeit in their current incarnations) Omniture, Coremetrics and Unica are working towards an analytical system that feeds marketing automation. Now we’ve got the potential for something entirely different.

For these reasons I’m bullish on the acquisition. We have a new opportunity for web analytics where site-side measurement meets audience (panel based) measurement. It’s the collision course that many have been talking about. And it sets the stage for propelling measurement into next generation devices, apps and mobile platforms that don’t have transactional elements. It’s still too soon to say how this will play out, but I applaud Magid, Gian and the comScore team on their vision for creating a new measurement paradigm. And a big congrats goes out to Michael, Michiel, Fred, Ulrike and the entire Nedstat team for building a globally attractive solution. Bravo.

But these are just my thoughts…I may be way off…I may be crazy. Readers, do you agree that this new duo can impact enterprise measurement on a new level? I’d love to know what others think.