How To Tell A Story with Your Data

Posted on December 29, 2009October 25, 2019 by Eric Peterson

A few weeks ago, my business partner Eric and I attended a basketball game in Minnesota. Eric purchased the tickets a few days ahead of time and I really didn’t have any expectations going into the game except to have a great time. Much to my surprise, our seats were incredible! We were sitting immediately behind the announcer’s table in the first row. Now, keep in mind, I’m a Boston sports guy and even when the Celtics were struggling through the 90’s and the early part of this decade, you still couldn’t get a seat behind the announcer’s table or anywhere near the first row without taking out a second mortgage on your house. But, this was Minnesota and the Timberwolves are not necessarily a big market team.

Anyway, as we enjoyed the game we struck up a conversation with the woman sitting immediately in front of us who was a coordinator for the announcers. Sitting on either side of her were two official NBA scorers recording all the action into their computers and generating reports at nearly ten-minute intervals. These reports were printed and handed to the announcers, which ended up in a big pile on their desks in front of them. After a while our friendly coordinator began handing Eric and I her extra copy of these Official Scorer’s Reports. So, like any good Web Analysts would do we took a look and gave the report a critical review (see the image below).

We were astounded by how poorly constructed the reports were. Sure, they contained all the critical information on each player like minutes played, field goals, field goal attempts and total points. Yet, there were no indicators of which metrics were moving, who was playing exceptionally well, or even shooting percentages for individual players. The announcers were undoubtedly skilled at their jobs, because these reports did nothing (or at least very little) to inform them of what to say to their television audiences. Clearly the NBA could benefit from some help from @pimpmyreports.

So, here is where I get to the point about telling a story with your data. Sometime during the middle of the fourth quarter a young aspiring sportscaster came running down to the announcer’s row and handed off a stack of paper that offered some new information. Finally! His 4th-Quarternotes recap was the first written analysis we’d seen that actually placed the statistics and metrics recorded during the game into meaningful context (see image below). The 4th-Quarternotes showed that:

A win could bring the T’wolves to 3-3 in their last six games.
Al Jefferson was having a good night – approaching a career milestone for rebounds – and posting his 9th double-double of the season.
Rookie, Jonny Flynn was about to post his first double-double (which only five rookie players have accomplished), needing only one more assist.
Ryan Gomes was once again nearing a 20 point game with a 58.6% field goal percentage in the past five games.

This method of reporting used all of the same data that was contained within the Official Scorer’s Report but added historical context, which really brought the data to life. This was interesting stuff! Now T’wolves fans and casual observers alike could understand the significance of Jefferson’s 16 points and 28:27 minutes on the floor – or that Jonny Flynn needed just one more assist to achieve a significant feat. After reading this, (even as a Boston sports fan) I was invested in the game and had something to root for – Go Flynn!

So here’s the moral of the story:

If you’re going to produce generic reports with no visual cues – do not show them to anyone because they won’t use them – and make sure you hire some damn good analysts that can interpret these reports and give a play-by-play.
If you do want to distribute your reports widely – take the time to format them in a way that highlights important metrics and calls attention to what’s meaningful so that recipients can interpret them on their own.
And most importantly – place your data and metrics in context given historical knowledge; significant accomplishments; or some other method to bring the data to life. Give your executives and business stakeholders something to cheer about!

Finally, if you ever have an opportunity to sit behind the announcer’s table, make sure you befriend the coordinator so you can get a copy of the reports for yourself.

Analytics Strategy, General

The Most Important Post on Web Analytics You'll Ever Read

Posted on December 22, 2009 by Eric Peterson

When John Lovett joined Aurelie and I here at Analytics Demystified earlier this month an awful lot of people said, “Hey, nice job getting such nice guy on board,” “We love John, he’s great,” and “Man, what a great addition to your team!” Clearly John has the respect of the industry, but one thing that remained an open question in some people’s minds was “how will John make the transition from the ivory tower an analyst sits in to the ground floor where consultants actually do work?”

I admit, I wondered that too in a way, having made a slightly different transition myself years ago. It’s not easy to come away from a situation where you provide advice but are tasked with, honestly, doing very little real work. During my own tenure at JupiterResearch years ago I ensured my own connection to practical web analytics by writing my second and third books. But John had been an analyst for nearly 10 years … and so wondering how he’d hit the ground was a reasonable question.

Wonder no more.

While John has already contributed greatly to the businesses bottom line and helped out with one of our largest new retail clients, he absolutely floored me this morning when he published his post Defining a Web Analytics Strategy: A Manifesto. I asked him to elaborate on some comments he made at Emetrics where he essentially poo-pooed the use of so called “Web Analytics Maturity Models”, describing the almost religious zeal some people seem to have when talking about models and declaring himself as a “Model Atheist.”

Having written the original Web Analytics Maturity Model back in 2005, I have had first-hand experience with their failure to produce anything more than a generalized awareness that most companies simply don’t “get” web analytics, something that we more or less all know already. But honestly I was surprised when John took this position on the subject because, well, in my experience those that don’t do, teach, and models are a classic teaching tool.

I had assumed that as an analyst John was a teacher, not a do-er like I have been for years now in my capacity as a practice leader, consultant, and web analyst. Man was I wrong …

John’s “Manifesto” is perhaps the most lucid yet succinct explanation I have ever read detailing the steps required to make web analytics work for your business (as opposed to the other way around.) I almost asked him to edit the post for fear that he was opening our kimono too much, but if Social Media has taught us anything it has taught us that transparency is king. The fact that he managed to encapsulate what others have been trying to explain with long-winded speeches, tangential arguments, and downright rude behavior is a huge plus.

Some of you may read John’s manifesto and think “Gee, this seems to point to the need for outside consultants” which is a fair criticism. But before you react consider two things:

Consultants (like us) have a tendency to, you know, recommend consulting. Everyone’s perspective arises from their own personal biases, regardless of how many times they declare the contrary. We are consultants, consultants who want to feed their children. Forgive us our bias and we will forgive you yours …
Consultants in the Enterprise are like death and taxes, we are more or less inevitable. Often times an outside perspective is exactly what the business needs to actually start to act upon the message that otherwise great employees have been stating for years. Other times the business simply stops listening to their employees and won’t make a move until McKinsey, Bain, or Demystified come in and charge big money for insights that were already there. Either way, ours is the second (or is it third) oldest profession and it must be for a reason …

I would challenge you, dear reader, to spend some time reading John’s post and considering what he has to say. Think about how you could apply his ten insights to your business regardless of whether you turn to consultants for advice or not. Listen to your business partners needs, put away your models and roll up your sleeves, transcend mediocrity, establish your own waterfall and embrace change!

When I said “web analytics is hard” I meant it, I really, really did. But I wasn’t trying to box anyone in or establish myself as some kind of amazingly wonderful “guru”, I was simply telling you all the truth based on my dozen years of experience in the sector. Yes, getting started can be easy; yes, making Google Analytics do stuff can be easy; and yes, you can do an awful lot in an hour a day if you simply apply yourself to the task … but the problem is that within any business of size, complexity, or nuance — which is to say all businesses everywhere — the act of getting from raw data to valuable business insights that you can repeatedly take action upon is apparently so freaking difficult that almost nobody does it.

How is that “easy?”

You all know I love a good debate so if you disagree with my comments here please let me know. If, however, you have something to add to John’s manifesto, I would encourage you to comment on his blog post directly.

Happy Holidays, everyone.

General

Defining A Web Analytics Strategy: A Manifesto

Posted on December 22, 2009October 25, 2019 by Eric Peterson

“…Strategy is all talk unless it can be executed in a way that delivers on both the creative and business promises.“ ~Liz Gebhardt Thinking Out Loud

I’ve been thinking a great deal about strategy lately and what it means to truly build a winning strategy for Analytics. All too often strategies take shape after a plan is already in action. We’re seeing rampant instances of this type of reactionary measurement strategy in social media today because marketers simply don’t want to get left behind in the latest digital craze. Yet, they really don’t know where to focus their strategic measurement efforts, so tactics take precedence. But even with more mature disciplines, like Web Analytics, strategies are often ill formed and don’t contain the vision necessary to carry an organization to the next level. And often times it’s not their fault – getting something done requires tactical execution – but companies and marketers in it for the long haul eventually come around to a strategic approach.

In many cases, consultants are brought in to build strategic roadmaps for measurement practices, especially in the complex realm of Web Analytics. Yet, outside experts (and internal champions) are usually at a distinct disadvantage because organizations have already embarked on a process of web data analysis and somewhere along the line, those efforts failed. New strategies must then clean up in the aftermath of failure to override distrust and misguided use of digital data. However, a well defined strategy will set you on the right track. Whether you endeavor to take this strategy on by yourself or hire an external resource to guide you through, this manifesto should help. It’s one that I adhere to and offers some guidance that should make the difference between rudderless marketing efforts and well-defined programs with quantifiable measurement success.

Strategy Credo #1: Listen to your constituents. Building a sustainable strategy of Analytics requires some serious fact-finding. To advise an organization on steps toward improvement, you must first fully understand their unique situation. For internal strategists, this means soliciting feedback from your cohorts and establishing a collaborative environment – while external consultants are required to get under the corporate covers by asking the right questions and listening carefully for ambient and recurring themes. When you stop and take the time to listen you can find out some incredibly revealing things.

Strategy Credo #2: Roll up your sleeves. … And get to work. Our tight knit cottage industry that is Web Analytics is quickly outgrowing its humble origins and becoming a marketing imperative of global proportions. Yet, amid the experts and audible voices in our space there’s some derision between those who spout analytics theory and those that actually practice analytics. To become effective at delivering a strategy for Web Analytics, one must go beyond the academic exercise of offering models that propagate general best practices to proving real value through demonstrated client success. This success comes from working with clients to understand their unique issues and customizing a solution to meet specific needs.

Strategy Credo #3: Assimilate to the culture. This one is important because culture within the organization will dictate the Web Analytics strategy. If you’re on the inside working for change, you are probably already ensconced by culture so the trick will be to separate yourself from bias and your established notions of what you see – to how things really are. For outsiders, culture is something that you can actually pick up on pretty quickly. By observing who takes over a meeting immediately upon entering the room or listening to the pleas of a frustrated analyst, you get a good sense of how things operate. Culture is definitely the most difficult thing to change at an organization, so understand it in order to work with it rather than against it.

Strategy Credo #4: Mediate judiciously. Web Analytics managers and staff can typically identify 99% of their own issues – they often just need a story and an effective communicator to incite management to change. By 1) listening to your constituents, 2) demonstrating empathy and a desire to affect change by rolling up your sleeves, and by 3) understanding the culture you are working in you’ll be in a position to shape the story. However, often times, internal employees are unsuccessful at pleading for the cause. For this reason, consultants are called in to argue precisely the same points that internal advocates have been saying for years. When this is done judiciously and with conviction the light bulb goes off and management actually begins to take action.

Strategy Credo #5: Identify creative solutions. In my experience, I’ve found that when it comes to Web Analytics it’s typical that: a) resources are limited, b) budgets are constrained and c) inflexibility exists somewhere within the system. A valuable manager or crafty consultant will assess these limitations and deliver creative solutions. This means quit complaining that you don’t have enough staff or resources for Web Analysis – nobody does! Instead determine if contractors can get you over the hump – or perhaps offshore labor can free up bandwidth – or maybe just tin cupping it around your organization to get funding for that measurement project is a viable solution. Creativity requires new thinking and developing a measurement strategy that works is predicated on fresh thinking.

Strategy Credo #6: Transcend mediocrity. I’ve been saying this one for a while, but it’s something that I strongly believe in – and it’s really hard to do. Mediocre analysts spend their days producing reports and processing data that can be (but usually isn’t) consumed and put into action by others. Too often, there’s little time for actually thinking about the data and translating what it means to take insights to action. This is essentially the equivalent of opening your mouth and letting the unfiltered drivel run out before thinking it through. It’s your Does-This-Actually-Make-Sense meter that should be pinned to the red when communicating analytics data. Climbing out from this sea of mediocre analysis requires a measurement strategy that ensures everyone is working towards the same goal and that the signposts are legible and in the same language for every traveler.

Strategy Credo #7: Actually solve the problem. Strategic Web Analytics goes beyond merely pointing out problems to actually solving them. Personally, I get immensely frustrated when efforts cease upon identifying the problem and fall short of providing tangible resolutions. This classic downfall of theory is quick to point out that your Web Analytics program blows, but beyond pointing to more technology or generalized tactics, there’s no real solution. This is a result of not actually doing the work (see #2 Roll up your sleeves) and thus not really knowing what is effective. Whether initiated from the inside or from an external consultant, a Web Analytics strategy must solve the problem and establish a working solution.

Strategy Credo #8: Establish a waterfall strategy. By this I mean strategy should flow from the headwaters of the organization and align with the corporate goals set forth by the executive team. Once your measurement team is clear and united on the goals, then identify objectives as the next tier in your waterfall that supports the corporate goals (these are your business promises). The base of your waterfall strategy consists of the tactics. Tactics are the actual campaigns and programs that emerge from your marketing machine (your creative promises). Each tier within the waterfall has specific metrics that indicate success. These metrics must be clearly defined and baked into the system at all levels to ensure proper measurement. It’s also critical to recognize that neither you nor an external consultant is likely to change your corporate goals, but you can refine the way in which you get there.

Strategy Credo #9: Ensure executable recommendations. Practice a crawl → walk → run approach to implementing a measurement strategy. This involves clearly illustrating the immensely lucrative and sexy benefits of being able to run, but knocking some reality into your key stakeholders by showing that serious work is required to get there. Successful strategies are designed in bite-sized chunks that align the components necessary to perform analysis wonders. If you try to take on too much too soon, then you’ll end up falling on your face and losing the confidence of your champions. By establishing clear expectations and milestones of accomplishment you will be on your way to executing on a measurement strategy.

Strategy Credo #10: Embrace change. Change management must be factored into any Web Analytics strategy overhaul. In most cases, refining (or defining) a strategy for marketing measurement is a monumental task. To offer an analogy, it’s like asking your grandfather to step out of his Oldsmobile so you can rip out the dashboard and replace it with an entirely new one that includes foreign instruments, dials and gauges. After you go through the painful process of getting this new system in place you need to explain to your grandfather that knowing how fast he was going using the speedometer was simply not good enough. Instead he should be focused on his fuel consumption per mile in order to conserve gas (tactic) – so that he can adhere to his fixed income budget (objective) – so that there will be some money left over when he eventually passes on (goal). Apologies for the crude analogy – but your grandfather’s Oldsmobile simply won’t cut it anymore! Your measurement strategy needs to create change – communicate the benefits – and deliver value. No one said this was easy 😉

So there you have it. That’s my Web Analytics Strategy Manifesto. I really believe that there’s a profound difference between “people who think about Web Analytics” and “people who do Web Analytics”. This Manifesto is based on doing. I’m curious to know what you other “doers” think and how you’ve embarked on establishing a strategy for measurement in your organizations?

General

Four Books That Will Change the Way You Communicate

Posted on December 22, 2009October 25, 2019 by Eric Peterson

I don’t think I will ever forget the first time that I made a presentation at work. It was just over a decade ago, I was just a few months into my employment at a company where I would work for the next eight years, and I was on the hook to present a new process to a room of 20 engineers. I diligently prepared my transparencies (I’m old enough to have used an overhead projector, but not old enough to refer to the medium they supported as “foils”). I rehearsed the material again and again.

And I bombed.

The material was dry as it was, but it wasn’t, by any means, unmanageable content. I just didn’t do a good job of managing it!

Fast forward 10 years, and I found myself giving a presentation to a room of 50-60 people, and the material was set up to be just as naturally engaging — presenting on an approach to measurement and analytics to…a bunch of marketers.

The presentation went much better, judging both from the engagement level of the audience and discussions that it has prompted weeks later. I’m no Steve Jobs, but I’ve paid attention to what seems to work and what doesn’t (both in my presentations and others), read some articles here and there, and, I realized, read a few books along the way that have really helped.

So, with that — four books that all have a heavy component of “how the brain works” and that, collectively, have taught me a lot about how to present information, be it a dashboard, a report, or a presentation.

Gladwell and Gilbert

The first two books are books that I read within a few months of each other. To this day, I recall specific anecdotes with no idea which book they came from. Blink: The Power of Thinking Without Thinking made the rounds when it first came out as “another great book by Malcolm Gladwell” (following The Tipping Point: How Little Things Can Make a Big Difference). The fundamental anecdote of Blink has to do with our “adaptive unconscious” — our intuition and ability to “know” things without fully needing to process them. As he dives into example after example, Gladwell touched on various aspects of how the brain works.

Daniel Gilbert’s Stumbling on Happiness takes a more directly psychological angle, but it covers some of the same territory. One of Gilbert’s main points is that the human brain does not remember things like we think it does — pointing out that a vividly remembered, down-to-the-color-of-the-shirt-you-were-wearing memory is not really an as-recorded memory at all. Rather, the brain remembers a few specific details and then makes up / fills in the rest when the memory gets called up. It’s so good at filling in these blanks that it fools itself into not being able to tell fact from interpolation!

Both of these books made an impact on me, because they pointed out that how we take in, process, and store information doesn’t work at all like we intuitively think it does. And, both books set up the next two books by shaking the assumptional foundations I had of how we, as humans, think.

Straight-Up Business Reading

Chip and Dan Heath’s Made to Stick: Why Some Ideas Survive and Others Die is a practical manual for communicating information that you want your audience to pay attention to and retain. They boil the components into a five-letter acronym — S.U.C.C.E.S. — and go into each component in detail.

The elements are Simple, Unexpected, Concrete, Credible, Emotional, and Stories, and they provide a nice framework for critiquing how we communicate any idea. Irecognize that I regularly struggle with Simple, Concrete, and Stories as elements in my blog posts. But, every element is one that can be injected using some discipline and time to do so. I nailed all three of these elements a number of years ago when I found myself on an internal lecture circuit trying to drum up large donors for my company’s annual United Way campaign — I was heavily vested in conveying a strong message, and I wound up using an example of my grandfather’s battle with Alzheimer’s as a way to pull the audience in and ask them to find something they were passionate about and support it. I also wove in various quirky takes on how $10/week would really add up — think the sort of thing you hear again and again from your local NPR station during fundraising drives. In the case of that campaign, we blew our numbers out of the water — had a 500% increase in the number of people who gave at the “leadership level” that year. Now, a lot of things had to come together to make that happen, but, to this day, I’m sure my well-crafted, well-rehearsed, and sincere speech made to at least a dozen different groups of employees (and the fact that I was a fairly low-level employee making the case — I was asking people who were making a lot more money than I was to give at least as much as I was), played a non-trivial role.

And that was years before I read Made to Stick. But, the book helped me reflect on any number of presentations — ones that worked and ones that didn’t.

And, Finally, Wisdom from a Neuroscientist

The last book in this tetralogy is one that I just finished reading — Brain Rules: 12 Principles for Surviving and Thriving at Work, Home, and School, by John Medina. I stumbled across the book as a recommendation from Garr Reynolds of Presentation Zen, so I wasn’t surprised that it had some very practical tips, as well as the “why?” behind them, for communicating effectively. Medina’s premise is that there’s a ton of stuff we don’t yet understand about the brain. BUT, there are also a lot of things we do know about the brain, and many of those lay out pretty clearly that the way we work in business and the way our education system is set up both run counter to how the brain naturally functions.

These “things we do know” are broken down into 12 “rules” — exercise (good for the brain), survival (why and how the brain evolved…and implications), wiring (how the brain works at a highly micro level), attention (there’s NO SUCH THING as multitasking…and other goodies), short-term memory (what makes it there and how), long-term memory (what makes it there, how, and how long it takes to get there), sleep (good for the brain), stress (some kinds are good, some kinds are bad), sensory integration (the more senses involved, the better the memory), vision (the #1 sense), gender (men are from Mars…), exploration (age doesn’t really degrade our ability to learn). Medina ends each chapter (one rule per chapter) with “Ideas” — implications for the real world based on the information presented.

The book goes into very technical detail about how, when, and where electrical charges zip around in our skulls to accomplish different tasks. While that information is not directly applicable, each time he goes there it’s as a setup to more directly useful information. Throughout the book, Medina provides practical thoughts for how to communicate more effectively — helping people pay attention (getting the information you are communicating into working memory) and retain the information over both the short and the long term. Two of my absolute favorite nuggets from the book were:

p. 130 (in the chapter on long-term memory) — Medina has the reader do a little memory exercise with the following characters: “3 $ 8 ? A % 9.” The fact he drops after the exercise is interesting: “The human brain can hold about seven pieces of information for less than 30 seconds! If something does not happen in that short stretch of time, the information becomes lost.” This is about getting information on its way from working memory to long-term memory and how repetition, thinking about the information, and talking about the information all helps it on its way. As a communicator (be it through a presentation or through a dashboard of data), this seems like powerful stuff — how often have we all seen someone cut loose with slide after slide of mind-numbing information? The human brain simply cannot take all of that in and retain it without some help!
p. 239 (in the chapter on vision) — Medina has a section titled “Toss your PowerPoint presentations.” I groaned. While I get highly annoyed by the rampant misuse of PowerPoint, I’m not a Tufte acolyte to the point that I see the tool itself as evil. In the second paragraph, though, Medina clarifies by providing a two-step prescription: 1) burn your current presentations, and 2) make new ones. Medina’s beef with PowerPoint is that the default slide template is text-based with a six-level hierarchy. This entire chapter is about how a picture really is worth 1,000 words, and Medina pleads with the reader to cut wayyy back on the text in his/her presentations (he has a fascinating explanation of how, when we read, we’re really interpreting each letter as a small picture…and that’s actually not a good thing for retention of information).

There are oodles of other good information in the book, but these are two of the snippets that really resonated with me.

Better to Be Steve Jobs than Bill Gates

I do believe that some people have better communication instincts than others. I’ll never be Steve Jobs when it comes to holding an auditorium in the palm of my hand. But, between reading these books and thinking through my own evolution as a communicator (this blog notwithstanding…but I’ve always said that I write this blog to keep my e-mails shorter and to try out ideas that occur to me during the day — sorry folks…both of you…but this blog is mostly for me!), I’m convinced that effective communication is a trainable skill.

I’ve also noticed that, the more I have to communicate, and the more I work to do so effectively, the easier it seems to be getting. In another 20 years, I might just have it nailed!

Conferences/Community

Amazing response to The Analysis Exchange

Posted on December 17, 2009 by Eric Peterson

UPDATE: The compliments and willingness to help just keep coming in. Members of the Web Analytics Association Board of Directors, entire staffs from consulting groups around the globe, and too many individual students and practitioners to possibly mention. Thanks so much everyone!

Wow. Wow. Wow. I am at a total loss for words when it comes to the response from the web analytics community regarding our soft-launch for The Analysis Exchange on Tuesday. We’ve had over 250 people, largely from within the measurement community, sign up to participate and have seen the nicest emails imaginable. Clearly this is an idea that’s time has come, and clearly the lack of training opportunities in the sector was a “raw nerve” for many people.

But don’t take my word for it. The following are excerpts from emails we received over the past few days. These speak for themselves and only serve to reiterate the need for the Exchange. The first email is from Bryce in Georgia:

“I really want to do web analytics as a profession but am having a hard time breaking into the field. There’s not exactly anywhere to ‘get a degree’ doing this kind of thing and even with my business and accounting background, and decade of web development freelancing, my experience does not seem to be impressing any potential employers.

It would be awesome to get some real experience under the guide of professionals. I’ve been reading Avinash’s books, following several analytics blogs and I’ve set up numerous GA sites with funnels, goals, and KPI’s that are being tracked. The fact that we’ll be helping out non-profits and outreaches at the same time is super.

Please pick me, :)”

From Ali:

“I was reading Eric Peterson’s blog entry about the Analysis Exchange and I would like to be a mentor and help contribute to the success of a company by empowering them with the awesomeness of web analytics.”

From Emmett in Menlo Park:

“I am very grateful for a “student” opportunity you might generate for and with me as you defined in your clear video today. Through WAW, your email kindnesses, and more, you have taught me “what I don’t know,” and what I do offer is a proven data smog to actionable wealth of knowledge capital “philosophy,” and an “attitude of gratitude of How May I Help You?” These are far more than just nice words to me.”

From From Ridder in San Diego:

“This a great initiative!! Being an active student of web analytics, I encountered the same catch 22 when attempting to break into this industry. I have signed up and look forward to participating!!”

From Gerry in the UK:

“Just been through the site after receiving the email and seeing the twitter alerts and I wanted to firstly pass on my congrats for initiating such a cool idea, it’s long overdue and will be greatly needed if we are to push the industry forward, as I responded a while back on twitter – resource (and quality resource in particular) remains the biggest weakness in the industry, so anything that helps improve that is only good in my eyes.I have no idea how you intend to select mentors, but as a leading practitioner in the UK for over 10 years on both the client and vendor side if I can help in this capacity I’d be delighted to put my name forward, I promise not to be offended if you decide not to use me initially!

The application of web analytics, either in analysis or how to make it an integral part of a business is something I have done much work on over the last 2-3 years, so I’d like to think I have a lot to offer anyone new to the industry that’s trying to work out how to start pushing things forward

Anyone, good luck with exchange and hope I can help in any way.”

Finally, and this is the email that put a grin on my face for the entire day Tuesday, from Zach:

“I think this program sounds wonderful and wanted to be as early as possible in dedicating myself to this program. I am a student of analytics – I have taken courses at the University of British Columbia in Web Anlaytics, I’ve studied and will be taking the exam for Google Analytics certification soon, and I belong to the Web Analytics Association. Unfortunately, I have little “real” world application using web analytics tools and making sound recommendations using these tools.

I do have a career of experience as an advertising consultant so I am not a “traditional” student. I hope I can apply some of the experience I have and use it to develop myself and a new career in web analytics.

I am excited to be involved in this great opportunity – I feel like it was created just for me.”

Furthermore Zach reminded me that he is one of the thousands of people I have been fortunate enough to offer some small amount of advice to over my years:

“I took your advice you offered me some time ago to take the UBC classes and it was good advice, but they really lack real world application – especially for hands on learners like me. I’ve actually approached some Non-Profit organizations and boards that are willing to let me “practice” and If I could gain the experience of a mentor to work with that would be great. I know you will be sending out additional information in the coming year (right around the corner) but I am anxious to get started!”

Zach’s point is an excellent one — I have long been telling people who ask me “how do I get real-world experience” for years to reach out to their church, their kids school, their local animal hospital, or local charities. While this is good advice (sorry) it is also somewhat unpractical I suspect and falls into the category of things that are easy for me to say and hard for people to actually do.

The Analysis Exchange solves that problem. We will bring the mentors, we will bring the businesses, and we will provide the system. Our sincere hope is that by lowering the barriers as much as humanly possible we will be able to create the maximum number of opportunities for everyone — mentors, students, and causes.

The email keeps coming in so I will leave you with two thoughts:

We’d love your help spreading the word about The Analysis Exchange! If you have a blog, a Twitter account, or just a bunch of like-minded friends in Facebook, please share the news about this effort by asking people to retweet this URL: http://bit.ly/analysis-exchange
We’d love to hear from you! If you’re excited about the effort and want to pitch in or just share your thoughts please feel free to email us at exchange@analyticsdemystified.com or leave your comments below

Thanks again to everyone who has signed up and reached out so far. Just like Web Analytics Wednesday and the Web Analytics Forum, if we all work together amazing things can happen.

Analytics Strategy, Conferences/Community, General, Social Media

Announcing The Analysis Exchange

Posted on December 15, 2009April 12, 2017 by Eric Peterson

A few weeks ago I started pinging folks within the digital measurement community asking about the work we do, the challenges we face, and how we got where we are today. The responses I got were all tremendously positive and showed a true commitment to web analytics across vendor, consultant, and end-user practitioner roles. What I learned was, well, exactly what I expected given my decade-plus in the sector: “web analytics” is still a relatively immature industry, one populated by diverse opinions, experiences, and backgrounds.

Those of you who have been following my work know that I have spent a great deal of time working to create solutions for the sector. As a matter of record I was the first to create an online community for web analytics professionals and explicitly point out the need for dedicated analysis resources back in 2004, and the first to publish a web analytics maturity model and change how web analytics practitioners interact with their local community back in 2005. I’ve also written a few books, a few blog posts, and have logged a few miles in the air working with some amazing companies to improve their own use of web analytics.

I offer the preceding paragraph not to brag but rather to establish my credentials as part of setting the stage for what the rest of this post is about. Like many in web analytics — Jim Sterne, Avinash Kaushik, and Bryan Eisenberg all come to mind — I have worked tirelessly at times to evolve and improve the landscape around us. And with the following announcement I hope to have lightning strike a fourth time …

But I digress.

One of the key questions I asked in Twitter was “how did you get started [in web analytics?]” Unsurprisingly each and every respondent gave some variation on “miraculously, and without premeditation.” While people’s responses highlighted the enthusiasm we have in the sector, it also highlighted what I see as the single most significant long-term problem we face in web analytics.

We haven’t created an entry path into the system.

As a community of vendors, consultants, practitioners, evangelists, authors, bloggers, Tweeters, socializers, and thought-leaders, we have failed nearly 100% at creating a way for talented, motivated, and educated individuals who are “not us” to gain the real-world experience required to actually participate meaningfully in this wonderful thing that we have all created.

Before the comments about the Web Analytics Association UBC classes or the new certification pour in consider this: The UBC course offers little or no practical experience with real data and real-world business problems, and the certification is designed, as stated, “for individuals having at least three years of experience in the sector.” Both are incredibly valuable, but they are not the type of training the average global citizen wishing to apply their curiosity, their precision, and their individual talents to the study of web data need to actually get a good job coming from outside the sector.

And while I have little doubt people have landed jobs based on completion of the UBC course given the resource constraints we face today, as a former hiring manager and consultant to nearly a dozen companies who are constantly looking for experienced web analysts, I can assure you that book-based education is not the first requirement being looked for. Requirement number one is always, and always will be, direct, hands-on experience using digitally collected data to tell a meaningful story about the business.

Today I am incredibly happy to announce my, my partners, and some very nice people’s solution to this problem. At 6:30 PM Eastern time at the Web Analytics Wednesday event in Cambridge, Massachusetts my partner John Lovett shared the details of our newest community effort, The Analysis Exchange.

What is The Analysis Exchange?

The Analysis Exchange is exactly what it sounds like — an exchange of information and analytical outputs — and is functionally a three-partner exchange:

At one corner we have small businesses, nonprofits, and non-governmental organizations who rarely if ever make any substantial use of the web analytic data most are actively collecting thanks to the amazing wonderfulness of Google Analytics;
In the next corner we have motivated and intelligent individuals, our students, who are looking for hands-on experience with web analytics systems and data they can put on their resume during when looking for work or looking to advance in their jobs;
And at the apex of the pyramid we have our existing community of analytics experts, many of whom have already demonstrated their willingness to contribute to the larger community via Web Analytics Wednesday, the WAA, and other selfless efforts

The Analysis Exchange will bridge the introductions between these three parties using an extremely elegant work-flow. Projects will be scoped to deliver results in weeks, effort from businesses and mentors is designed to be minimal, and we’re working on an entire back-end system to seamlessly connect the dots. And have I already mentioned that it will do so without any money changing hands?

Yeah, The Analysis Exchange is totally, completely, 100 percent free.

John, Aurelie, and I decided early on, despite the fact that we are all consultants who are just as motivated by revenue as any of our peers, that the right model for The Analysis Exchange would be the most frictionless strategy possible. Given our initial target market of nonprofits and non-governmental organizations, most of whom our advisers from the sector warned were somewhat slow to invest in technology and services, “free” offered the least amount of friction possible.

Businesses bring data and questions, mentors bring focus and experience, and students bring a passion to learn. Businesses get analysis and insights, students gain experience for their resume, and mentors have a chance to shape the next wave of digital analysis resources … resources the mentor’s organizations are frequently looking to hire.

More importantly, our mentors will be teaching students and businesses how to produce true analytical insights, not how to make Google Analytics generate reports. Our world is already incredibly data rich, but the best of us are willing to admit that we are still also incredibly information poor. Students will be taught how to actually create analysis — a written document specifically addressing stated business needs — and therein lies the true, long-term value to our community.

Too many reports, not enough insights. This has been the theme of countless posts, a half-dozen great books, and nearly every one of the hundred consulting engagements I have done in the past three years. The Analysis Exchange is a concerted effort to slay the report monkeys and teach the “analysts” of the future to actually produce ANALYSIS!

A few things you might want to know about The Analysis Exchange (in addition to the FAQ we have up on the official web site):

Initially we will be limiting organizational participants to nonprofit and non-governmental entities. We are doing this because we believe this approach simultaneously provides the greatest benefit back beyond the web analytics community and provides a reasonable initial scope for our efforts. Plus, we’ve partnered with NTEN: the Nonprofit Technology Network who are an amazing organization of their own;
Initially we will be hand-selecting mentors wishing to participate in the program. Because we are taking a cautious approach towards the Exchange’s roll-out in an effort to learn as much as possible about the effort as it unfolds, we are going to limit mentor opportunities somewhat. Please do write us if you’re interested in participating, and please don’t be hurt if we put you off … at least for a month or two;
With the previous caution in mind, we are definitely open to help from the outside! If you have experience with this type of effort or just have a passion for helping other people please let us know. Just like with Web Analytics Wednesday, we know that when The Analysis Exchange gets cranking we will need lots and lots of help;

Because this post is beginning to approach the length at which I typically tune out myself I will stop here and point readers to three resources to learn more about The Analysis Exchange:

We have a basic, informational web site at http://www.analysis-exchange.com that has a nice video explaining the Exchange model in a little greater detail;
You can email us directly at exchange@analyticsdemystified.com for more information or to let us know if you’re willing to help with Exchange efforts;
You can follow Exchange efforts in Twitter by following @analysisxchange

As you can probably detect from the post I’m pretty excited about this effort. Like I did when I co-founded Web Analytics Wednesday, I have some amazing partners on this project. And like I did when I founded the Yahoo! group, I believe this effort will satisfy an incredible pent-up demand. Hopefully you will take the time to share information about The Analysis Exchange with your own network, and as always I welcome your thoughts, comments, and insights.

Learn more at http://www.analysis-exchange.com

Analysis, Analytics Strategy, Social Media

The Spectrum of Data Sources for Marketers Is Wide (& Overwhelming)

Posted on December 14, 2009October 25, 2019 by Eric Peterson

I’ve been using an anecdote of late that Malcolm Gladwell supposedly related at a SAS user conference earlier this year: over the last 30 years, the challenge we face when it comes to using data to drive actions has fundamentally shifted from a challenge of “getting the right data” to “looking at an overwhelming array of data in the right way.” To illustrate, he compared Watergate to Enron — in the former case, the challenge for Woodward and Bernstein was uncovering a relatively small bit of information that, once revealed, led to immediate insight and swift action. In the latter case, the data to show that Enron had built a house of cards was publicly available, but there was so much data that actually figuring out how to extract the underlying chicanery without knowing exactly where to look for it was next to impossible.

With that in mind, I started thinking about all of the sources of data that marketers now have available to them to drive their decisions. The challenge is that almost all of the data sources out there are good tools — while they all claim competitive advantage and differentiation from other options…I believe in the free markets to the extent that truly bad tools don’t survive (do a Google search for “SPSS Netgenesis” and the first link returned is a 404 page — the prosecution rests!). To avoid getting caught up in the shiny baubles of any given tool, it seems worth organizing the range of available data some way — put every source into a discrete bucket. It turns out that that’s a pretty tricky thing to do, but one approach would be to put each data source available to us somewhere on a broad spectrum. At one end of the spectrum is data from secondary research — data that someone else has gone out and gathered about an industry, a set of consumers, a trend, or something else. At the other end of the spectrum is the data we collect on our customers in the course of conducting some sort of transaction with them — when someone buys a widget from our web site, we know their name, how they paid, what they bought, and when they bought it!

For poops and giggles, why not try to fill in that spectrum? Starting from the secondary research end, here we go…!

Secondary Research (and Journalism…even Journalism 2.0)

This category has an unlistable number of examples. From analyst firms like Forrester Research and Gartner Group, to trade associations like the AMA or The ARF, to straight-up journalists and trade publications, and even to bloggers. Specialty news aggregators like alltop.com fall into this category as well (even if, technically, they would fit better into a “tertiary research” category, I’m going to just leave them here!).

I stumbled across iconoculture last week as one interesting company that falls in this category…although things immediately start to get a little messy, because they’ve got some level of primary research as well as some tracking/listening aspects of their offer.

Listening/Collecting

Moving along our spectrum of data sources, we get to an area that is positively exploding. These are tools that are almost always built on top of a robust database, because what they do is try to gather and organize what people — consumers — are doing/saying online. As a data source, these are still inherently “secondary” — they’re “what’s happening” and “what’s out there.” But, as our world becomes increasingly digital, this is a powerful source of information.

One group of tools here are sites like compete.com, Alexa, and even Google’s various “insights” tools: Google Trends, Google Trends for Websites, and Google Insights for Search. These tools tend to not be so much consumer-focussed as site-focussed, but they’re getting their data by collecting what consumers are doing. And they are darn handy.

“Online listening platforms” are a newer beast, and there seems to be a new player in the space every day. The Forrester Wave report by Suresh Vittal in Q1 2009 seems like it is at least five years old. An incomplete list of companies/tools offering such platforms includes (in no particular order…except Nielsen is first because they’re the source of the registration-free PDF of the Forrester Wave report I just mentioned):

Nielsen Buzzmetrics
BuzzLogic
Radian6
Alterian/Techrigy SM2
Filtrbox
Crimson Hexagon
Collective Intellect
Spiderfly

And the list goes on and on and on… (see Marshall Sponder’s post: 26 Tools for Social Media Monitoring). Each of these tools differentiates itself from their competition in some way, but none of them have truly emerged as a sustained frontrunner.

Web Analytics

I put web analytics next on the spectrum, but recognize that these tools have an internal spectrum all their own. From the “listening/collecting” side of the spectrum, web analytics tools simply “watch” activity on your web site — how many people went where and what they did when they got there. Moving towards the “1:1 transactions” end of the spectrum, web analytics tools collect data on specifically identifiable visitors to your site and provide that user-level specificity for analysis and action.

Google Analytics pretty much resides at the “watching” end of this list, as does Yahoo! Web Analytics (formerly IndexTools). But, then again, they’re free, and there’s a lot of power in effectively watching activity on your site, so that’s not a knock against them. The other major players — Omniture Sitecatalyst, Webtrends, Coremetrics, and the like — have more robust capabilities and can cover the full range of this mini-spectrum. They all are becoming increasingly open and more able to be integrated with other systems, be that with back-end CRM or marketing automation systems, or be that with the listening/collecting tools described in the prior section.

The list above covered “traditional web analytics,” but that field is expanding. A/B and multivariate testing tools fall into this category, as they “watch” with a very specific set of options for optimizing a specific aspect of the site. Optimost, Omniture Test&Target, and Google Website Optimizer all fall into this subcategory.

And, entire companies have popped up to fill specific niches with which traditional web analytics tools have struggled. My favorite example there is Clearsaleing, which uses technology very similar to all of the web analytics tools to capture data, but whose tools are built specifically to provide a meaningful view into campaign performance across multiple touchpoints and multiple channels. The niche their tool fills is improved “attribution management” — there’s even been a Forrester Wave devoted entirely to tools that try to do that (registration required to download the report from Clearsaleing’s site).

Primary Research

At this point on the spectrum, we’re talking about tools and techniques for collecting very specific data from consumers — going in with a set of questions that you are trying to get answered. Focus groups, phone surveys, and usability testing all fall in this area, as well as a plethora of online survey tools. Specifically, there are online survey tools designed to work with your web site — Foresee Results and iPerceptions 4Q are two that are solid for different reasons, but the list of tools in that space outnumbers even the list of online listening platforms.

The challenge with primary research is that you have to make the user aware that you are collecting information for the purpose of research and analysis. That drops a fly in the data ointment, because it is very easy to bias that data by not constructing the questions and the environment correctly. Even with a poorly designed survey, you will collect some powerful data — the problem is that the data may be misleading!

Transaction Data

Beyond even primary research is the terminus of the spectrum — it’s customer data that you collect every day as a byproduct of running your business and interacting with customers. Whenever a customer interacts with your call center or makes a purchase on your web site, they are generating data as an artifact. When you send an e-mail to your database, you’ve generated data as to whom you sent the message…and many e-mail tools also track who opened and clicked through on the e-mail. This data can be very useful, but, to be useful, it needs to be captured, cleansed, and stored in a way that sets it up for useful analysis. There’s an entire industry built around customer data management, and most of what the tools and processes in that industry focus on is transaction data.

What’s Missing?

As much as I would like to wrap up this post by congratulating myself on providing an all-encompassing framework…I can’t. While there are a lot of specific tools/niches that I haven’t listed here that I could fit somewhere on the spectrum of tools as I’ve described it, there are also sources of valuable data that don’t fit in this framework. One type that jumps out to me is marketing mix-type data and tools (think Analytic Partners, ThinkVine, or MarketShare Partners). I’m sure there are many other types. Nevertheless, it seems like a worthwhile framework to have when it comes to building up a portfolio of data sources. Are you getting data from across the entire spectrum (there are free or near-free tools at every point on the spectrum)? Are you getting redundant data?

What do you think? Is it possible to organize “all data sources for marketers” in a meaningful way? Is there value in doing so?

Social Media

Three Classification Genres for Measuring Twitter

Posted on December 14, 2009October 25, 2019 by Eric Peterson

I think it’s safe to say that Twitter has progressed from frivolous novelty to productivity tool for thousands of consumers, professionals and businesses alike. Projections from eMarketer have active Twitter users pegged to reach 18 million by the close of this year. I’ll admit that I was skeptical of the value of Twitter at first. I even went as far as to publish Conscientiously Objecting to Twitter, because I couldn’t see the value. But upon experiencing delirium tremors after being locked out of my Twitter account for 18 hours last week, I’m beginning to see things differently.

Frivolity leads to efficient information intake. There’s been a lot written about how and why people use Twitter for business purposes. I won’t rehash since my fellow Forrester alum, Jeremiah Owyang has published a great blog on the do’s and don’t of Twitter usage. For me, Twitter has become my go-to source for industry news and information. My evolution to this point began circa 2000 when jumping between bookmarks to troll major news sources was my common practice. A few years later, it evolved to aggregating RSS feeds (via iGoogle) in a single portal that I personalized to meet my news interests and needs. Now in 2009 I’m firing up TweetDeck to review the latest buzz; to gather news; and to quickly find information from people I feel are relevant in understanding technology, marketing and to the measurement industry. By using Twitter in this way, it creates visibility for things that I’m curious about and calls my attention to what’s new.

A new mode of discourse comes to light. It used to be that striking up a conversation with someone meant looking them in the eye and asking them what they thought about a particular topic. Twitter (and social media as a whole) has shattered the geographic limitations of conversation. This medium offers both individuals and brands the chance to pose their questions to thousands of potential listeners and receive feedback at scale. The conversation, which may have involved just a handful of people, now includes many. This offers great potential for understanding sentiment, sharing ideas and generally interacting with multiple people in an efficient way. European brands including Cadbury and Vodafone are leveraging Twitter as a new means of interacting with their customers through promotions and clever events. This exchange of information is fueling ideas, products, and adding incremental value (as well as entertainment) in an exponential way.

Twitter ushers in a new era of consumerism. The New York Times recently wrote, “America’s first Twitter Christmas got underway in earnest on Friday”. The article speaks of Black Friday shoppers using Twitter to reveal bargains, lodge complaints and even disclose parking availability at the Mall of America. Examples in the article illustrate that consumers are circumventing traditional channels and turning to social media. This poses significant threats for organizations because service issues are aired for all to see and the appropriateness of response can likely ripple extensively to shape the opinions of thousands of listeners. Best Buy is one organization that is out in front of the social networking craze and has developed “Twelpforce”, an employee driven service that responds to Twitter inquiries. According to the NYTimes article, the Twelpforce answered 25,000 questions even before Thanksgiving demonstrating substantial resolution in an efficient manner.

So what! How can a business use all this? Well, of course it all centers around measurement. As with all marketing initiatives, I recommend that businesses begin with a strategic approach to measurement by clearly understanding goals and objectives (this works for personal brands as well). It requires an introspective look at your motivations for getting involved with Twitter in the first place and then applying a matrix of key performance indicators that will indicate progress toward your goals. These KPIs should be specific to business objectives that pivot depending on the tactic and the social media channel.

The way I see it, there are three classifications of Twitter objectives that can be used by organizations and individuals alike. I’ve broken these classifications into genres because each contains myriad possibilities that will undoubtedly expand and grow as this medium matures. Yet, for individuals, marketers and the brands they represent, each Tweet falls into a genre that can be measured and evaluated with specific indicators of success. Everything else is just noise.

The Twitter genre’s are:

Visibility – This genre includes specific objectives such as building awareness, driving public relations, new customer acquisition efforts, dissemination of news and so on. Visibility is the “hey look over here” function that Twitter offers to get people to read your blog, visit your Web site, learn about your new initiative or simply turn towards that shiny new object you have to offer.

Exchange – Herein lies the catalyst to interaction between individual Twitter users, organizations and brands. The opportunity to pose questions, drive inquiries and elicit feedback within communities opens a new discourse that’s amplified through the channel. It’s the truly collaborative aspect of Twitter, where parties interact with one another in a meaningful way.

Resolution – This is the genre that provides answers. Resolution includes Twitter’s ability to provide service and support in a rapid and widespread manner. It demonstrates to the population that brands are listening to their customers and actually solving problems. It gives consumers a megaphone for expressing either satisfaction or displeasure and places them in the drivers’ seat.

I advise mapping specific measures of success (in the form of KPIs) to these genres in order to better understand the ways in which you’re providing value as an organization (or as an individual) to your following.

Categorization and measurement leads to understanding. Regardless of whether you use my genres above or develop your own, the ability to classify Tweets leads to a systemic method for measurement. The one thing that I love about social media is that there is an opportunity for measurement to transcend the mistakes we’ve collectively made while measuring web sites using Web Analytics. In most cases, Twitter doesn’t carry the baggage of legacy measures. Measurement analysts and marketers have a chance to work should-to-shoulder to establish metrics that align with the goals of their social media efforts. This requires understanding that topline metrics like follows and followers are generally meaningless without context and even with some context they aren’t actionable measures. Thus, more in depth metrics like influence, velocity and clout and the tools we use to measure these activities are required to recognize value from Twitter. For the typical digital measurement analyst this means starting from a place that’s more enlightened than visits and page views. Halleluiah!

Get on the bus and develop a measurement plan. The explosive growth of Twitter (be it healthy or unhealthy) begs the question…How are these 18 million users measuring the value of their efforts? I’ll venture a guess and say that 0.0001% is actually doing any kind of meaningful measurement on their Twitter efforts today. But, if you’re a business immersed in social media then you’d better be measuring – or – if you’re just toe-dipping into the social media arena there’s no better time to start.

But, I want to hear what you think. Are you measuring Twitter using anything like my proposed genres today? Would this method work at your organization? Could it lead to a strategy for developing your personal brand? I welcome your comments in an exchange of ideas here!

Adobe Analytics, General

Intranets – The Other Website

Posted on December 14, 2009February 6, 2020 by Eric Peterson

While most of you reading my posts are focused on your public website, in this post I am going to share how you can leverage your web analytics skills internally at your organization. Company Intranets are often times larger than the public website and using the tips I will share here, you can get some big visibility internally and become the hero of your HR team!

Why You Should Care About Your Intranet
Companies often spend a LOT of $$$ on building Intranets. Unfortunately, not everyone at the company uses the Intranet. If you can help your internal team show what is working and what is not working on the Intranet, you can help them to save a lot of money. In addition to the altruistic reasons to track what happens on the Intranet, there are the following selfish reasons:

Tagging Intranets is a great way to try new things and get better at web analysis in a safe environment
Intranets often have low traffic volume so it is a great way to help cost-justify increased budgets for web analytics (“Mr. CEO, not only does this money go towards tracking the website, it also allows us to track our entire Intranet!” – Just don’t tell them that tracking the Intranet costs all of $1,000 in server calls!)
Showing people what is happening on the Intranet does wonders for people inside your organization understanding what the heck you do for the public website!

I have seen situations where a web analytics team has killed themselves trying to get senior executives to see what is taking place on the website and what improvements could be made based upon solid web analysis, only to see the same team get promoted or more budget after spending 2-3 weeks showing what takes place on the Intranet (something that they actually use)! It sounds completely illogical, but I guess if you can’t beat them, join them!

Tracking Intranets
So what should you track on Intranets? The following are my best practices learned working with a few large clients. The one caveat to everything below is that you have to be sure to track all of this data in a different report suite than all of your other website data!

Employee ID
Depending upon the security policy of your company, ask if you are able to track down the the Employee ID level. I tend to not do this since it can be a bit creepy, but it is technically possible and you can replace the Omniture Visitor ID with your own unique employee identifier.

Non-Personally Identifiable Employee Info
On each Intranet page, I recommend that you pass Department, Region, Business Unit, Office Location, Employee Band Level (i.e. VP, Manager), etc… to variables. This will allow you to break down all Pages by these data points. I generally pass these to an sProp and an eVar (save some time setting both through this post) and also recommend you put your top five of these into a 5-item Traffic Data Correlation.

Pages & Sections
Obviously, you want to pass in a unique page name for every Intranet page like you would any other website. In addition, you should pass the Intranet section to the Site Sections (Channel) variable. As always, I recommend that you enable Pathing on the Channel sProp so you can see how employees are navigating between Intranet sections.

Internal Search
Just like a public website, Internal Search is usually important on Intranets. You should track Internal Search on the Intranet just as you would on a public website. You can apply the same principles I mentioned in this Internal Search post. This includes tracking what search terms people are looking for, but the beauty here is that you can see these by Department, Region, etc…

Timeparting
Many of my Intranet clients were keen to see when employees were accessing the Intranet, so I recommend you implement the Timeparting Plug-in. This allows you to see what day of the week and time of the day employees access the Intranet. Don’t forget to create a correlation between these sProps and your other ones so you can see when each page/section is accessed most often.

Internal Promotions
Much in the same way that I described Internal Campaigns in the past, Intranets may have promotional areas that try and entice employees to click. You can track these the same way you would a public website.

Intranet KPI’s
The following are the types of KPI’s I have seen used for Intranets:

Page Views/Visit & Average Time Spent/Visit
Depending upon whether your goal is to get employees in and out or get them to spend more time reading Intranet content, you can use this calculated metric to see how you are doing.

Page Views (Event)
As I described in this post, I would recommend that you set a Success Event on each page. Why? Well let’s say you want to see how many pages on the Intranet a specific internal e-mail led to. You can open the Campaigns report, find the e-mail and then see how many pages were viewed. You can then use an eVar Subrelation to break this down by page name (as long as you pass Pagename to an eVar) to see the exact pages viewed.

Internal Searches
As you would on a website, you should track and trend the # of Internal Searches taking place on the Intranet.

Logins
If employees have to log into your Intranet, you can capture that as a KPI to see how you are doing at getting them to access the Intranet. This can also be used for segmentation (i.e. show me all users who have not logged into the Intranet in the past 30 days…)

Custom KPI’s
Many times, Intranets are used to get employees to fill out forms, surveys, etc… Each of these key actions should be captured with a Success Event and in the case of Forms, you should capture the Form Name in an eVar so you can break it down appropriately.

Employee Profile Views
As we march down the road of internal social media, it is fun to track how often each employee’s Intranet Profile is viewed. Using new tools like Salesforce.com Chatter, we may be moving to a world where employees get “followers” so you can track how often people are looking at or following other employees. This allows you to see who your employees think are important (which may not always align to the org chart!).

Final Thoughts
As you can see, if you know what you are doing for tracking a public website, tracking an Intranet uses many of the same principles. If you are just getting started in web analytics, feel free to apply the above items on your Intranet as a testing ground before you tackle the public website. If you have some other cool things you have done to track your Intranet, please feel free to leave a comment here…

Adobe Analytics, Analytics Strategy

Data Quality – The Silent Killer…

Posted on December 7, 2009February 6, 2020 by Eric Peterson

In this post, I am going to talk about how Data Quality can kill an Omniture (or other Web Analytics) implementation. I will share some of the problems I have seen and show some ways that you can help improve Data Quality…

Sound Familiar?
So you have been managing an Omniture implementation for a while. You have your KPI’s lined up. You have been sharing some dashboards and reports with people throughout your company. People are starting to realize that they should talk to you before making website business decisions. Suddenly, you find yourself in the executive suite to answer some key website questions. Then, just as you are wrapping up your web traffic overview, an executive starts to calculate some numbers on a notepad and determines that the increase you show in Paid Search traffic doesn’t look right given other data they have seen from the SEM team. She also questions the rise in traffic data for EMEA, knowing that his VP in the region told you traffic has been down over the last few months. Suddenly, you are in a web analytics death spiral. In a split second, you have to decide, do you defend your Omniture data and risk your reputation or do you back-pedal saying you will re-check the web analytics data and live to fight another day?

Hopefully this hasn’t happened to you, but it has happened to most of us who have been around the web analytics space for long enough. Unfortunately, you only get so many chances to be wrong about data you are presenting and even if your data is right, if you aren’t confident enough to stand by it, it might as well be wrong.

Minimizing Data Quality Risk
So how do you avoid this situation? The first step is to realize that there is no way to be sure that all of your web analytics data is correct. 100% Data Quality is not only unattainable, but also not worth the time and effort it would take to achieve. Therefore, I use a philosophy of risk minimization in which I try various techniques to minimize the key things that cause data quality issues. The following will show you some of the ways to do this:

Ensure all Pages are Tagged
This is easier said than done. As we all know, IT is usually used to deploy JavaScript tags and they often have more important things to do than to guarantee that every website page has a the [correct] JavaScript tag. Fostering a good relationship with IT helps, but at the end of the day, new website pages are created all the time, and tags will be missing.

Use Technology
As you can imagine, where there is a need, there are technology vendors. The main vendors that I have worked with or heard the most about are WASP and ObservePoint. Not completely coincidental, ObservePoint was founded by John Pestana who was one of the co-founders of Omniture. In a great blog post, John Pestana talked about getting rid of asterik’s in web analytics reports. I am sure there are many other vendors out there offering similar products, but the gist of the technology is that it can spider your website and let you know which pages are missing JavaScript tags so eliminate any obvious omissions.

Blood, Sweat & Tears
Unfortunately, the main way that I have minimized web analytic data loss is by downloading data and looking for anomalies. I normally do this by taking advantage of the Omniture SiteCatalyst Excel Client and downloading key data blocks by day or week and then using formulas to compare yesterday to the same day last week or last week to the week prior. Once you have the data in Excel, you can do any type of statistical analysis you want on the data to see if anything looks “fishy.” One thing I like to do is to use Excel conditional formatting to spot data issues.

The following is a screenshot example of using Excel to spot potential data issues. In this example, I am looking at Page Views from one week prior to each day and if there is a change of more than 20%, I highlight it in red:

dq_excel2

Uh-oh… It looks like our daily data quality report indicates that we may have lost a tag on Friday for the Login page and something suspicious took place related to the Search Results page the same day. Obviously, the downsides of this approach are that it is extremely manual and that it is in arrears. As you know, once you miss a time slot of data in SiteCatalyst, there is no easy way to get that data back. While this approach can minimize the data loss to a day, it won’t help you get the Login Page data back in the example above.

Therefore, the way I employ this approach is to focus on the top items within each variable. This means, I focus on the pages with the most Page Views, the Form ID’s with the most Form Completions, the Orders for the most popular products, etc…With the Excel Client, you can download multiple data blocks at once and then use conditional formatting to easily spot the issues. Done intelligently, Data Quality for 80% of your data can be done in under a few hours each day. By doing this, you can feel more confident when your VP questions your data knowing that if something were significantly off, you would have known about it ahead of time.

Special Cases
I have found that there are a few other situations that commonly lead to missing or bad data so I quickly wanted to bring them to your attention so you can apply some additional effort to ensure they are tagged correctly:

“Lightbox” pages where a new HTML page is often not loaded. These often times are created as a window within a window and many times developers forget to put SiteCatalyst code within them.

Flash/AJAX pages where the page changes dynamically or you have an entire site/page developed in Flash. By extra careful around these as they often are missing tracking code (especially when done by an outside agency!).

Dynamically generated content, such as a page that shows historical stock price data after a user enters a ticker symbol. Often times, these dynamic pages are tagged as one single page, but might be better as unique pages from a web analytics viewpoint.

SiteCatalyst Alerts
If you have read my previous blog post on Alerts, you may figure out that you can use Alerts to help with Data Quality as well. Alerts can be used to look for changes in key metrics by Month, Week, Day (or Hour in some cases). These alerts can be handy to be notified when data is off by more than x%. However, I have found that if you want to look a more granular data (as in the example above), the current Alert functionality can be a bit limiting. You can set alerts for specific sProp and eVar values, but not as easily as you can by using Excel. Therefore, I would use Alerts as an early warning system an employ the previously mentioned techniques as your main defense against missing data.

Classification Data
Finally, when thinking about data quality/completeness, don’t forget about SAINT Classifications. If you have key reports that rely on SAINT Classifications, even if you have the source data collected perfectly, if you are missing key SAINT Classifications for that source data, your reports will be incorrect and indistinguishable from poor data quality in the eyes of your users. You will know if you are missing SAINT Classification data if your classified reports have a large “None” row. So how do you ensure your SAINT Classification data is complete? What I do is create Excel data blocks for each Classification and isolate the “None” row for key metrics.

In the screenshot below, you can see that I have created a data block that looks for “Unspecified” Site Locale Full Names (the Excel Client doesn’t use None, but it uses “Unspecified” instead for some reason). In this scenario, I store a 2-digit website country identifier in an eVar and use a SAINT Classification to provide a full name. I filter on “Unspecified” where the metric is Visits, Form Views and Form Completes.

dq_excel4

After running, you will see a succinct report that looks like this:

dq_excel3

In this case, there are no Form View or Form Complete Success Events missing a Full Site Locale SAINT Classification, but there are some Visits missing the classification. You can then easily go into SiteCatalyst or Discover, open the Full Locale Name report and break it down by its source to find out what values are left to be classified.

Finally, if you want to earn “extra credit” you can do this for all of your SAINT Classifications in one Excel workbook and make a summary screen like the one below which pulls the percentages that are unclassified into one screen so you can see how you are doing overall. What is cool about this is that you can use the “Refresh All” feature of the Excel Client to check all of your Classifications while you get coffee and when you get back, you have a fully updated view of your SAINT Classifications. In the above below, I have shaded some items in black that are OK if they aren’t fully classified, items in green that are acceptable and items in red that require attention:

dq_excel5

Final Thoughts
As you can see, Data Quality is a HUGE topic so it is hard to cover it all in one post, but hopefully some of the pointers here will get you thinking about how you can improve in this area. One last thing I will mention is that like most things related to web analytics, tools are good, but qualified people are better! Therefore, I think that any serious web analytics team will have a resource who has Data Quality as one of their primary performance objectives. Without this, Data Quality tends to fall by the wayside. Try to do whatever you can to convince your management that having a full or part-time person devoted to Data Quality will pay hefty dividends in the future…

Presentation

How Succinctly Can I Explain Why Pie Charts Are Evil?

Posted on December 2, 2009October 25, 2019 by Eric Peterson

I’m right at three months into my new gig, and, around the office, probably the most commonly known fact is, “He hates pie charts.” It’s not that I’ve exactly been standing at the elevator handing out leaflets explaining why pie charts are evil, but I have, apparently, chosen a couple of particularly public venues to make a mild statement or two. And, the quasi-preplanned visceral groan when some co-workers put up a pie chart might’ve contributed just a teensy bit.

I’ve been put on the spot since then a couple of times to do one of two things:

Explain why pie charts are evil, or
Agree that one or another particular usage of a pie chart is appropriate

After catching up on some blog reading yesterday morning and seeing an excellent example of pie chart alternatives from Jon Peltier, and then watching seven presentations yesterday, six of which used the same basic presentation template, and five of which stuck with a pie chart for the sole non-text slide in the presentation, how could I not write another post?! Let’s see how succinct I can make it (don’t hold your breath that you could read the whole thing before exhaling!).

Yes, There is ONE Thing That a Pie Chart Does Well

This kills me, because there’s one way, in a a very narrow set of circumstances, that pie charts do marginally better than alternatives. All THREE of the following criteria have to be met for this to be the case:

Exactly 2 or 3 categories that make up the “whole”
A fairly significant difference in % makeup for each of the categories
Plenty of space available to present the information

99 times out of 100 when pie charts get used, all of these criteria are not met. But, there, I’ve admitted that there is a situation where pie charts are appropriate.

Of course, mullets are an appropriate hairstyle if you are prone to both warm ears and spontaneous hair donations…but that doesn’t mean I’m going to sport one!

Of Course, We Must Start with a Before/After Example

With only the category names changed, below is one of the pie charts I saw yesterday:

Pie Chart Example

In my experience, a simple horizontal bar chart is a better option (among a variety of better options):

Bar Chart Example

Why is this a better option? Oh, let me count the ways…

1. Rainbows Are Good in Princess Tales — Not in Data Visualization

When it comes to data visualization, a chart that doesn’t rely on multiple colors always trumps a chart that does. Four reasons:

If you use subtle/muted colors, you can’t get past 4 or 5 categories before you are asking the person reading the chart to work hard to distinguish between subtle shading differences
If you use bright/high-contrast colors, you’re asking your user to put on sunglasses to keep from wincing at the visual overkill
Roughly 10% of men suffer from some form of color-blindness — it’s darn tricky to nail a palette with more than a small handful of colors that works across the various types of the condition (of course, if you’ve got a secret agenda to have women take over the world, this is one way to contribute, as color blindness is exceedingly rare in women)
Maybe you’re presenting your chart in glorious, projected color…but are you sure no one is going to try to print it in black-and-white?

These are all issues with any pie chart that has more than 3 categories. None of these are an issue with a horizontal bar chart.

2. Labels, Labels, Labels

If you’ve every constructed a pie chart in Excel, you’ve run into the challenge of trying to get all of the wedges labeled right there on the chart. Excel continues to make odd choices as to where to wrap text in pie charts, and the circular nature of the whole layout means some wedges have plenty of horizontal labeling room, while others have almost none. You’ve tried some (or all) of the following:

Using leader lines for some of the wedges so you can label the most troubling wedges somewhere more spacious
Abbreviating the category names
Strategically rotating the chart so that the labeling all happens to work (it never does)
Rearranging the underlying data so that the pie wedges occur in a different order (which also never works)

After fiddling with the above, you finally break down and yank the labels from the chart and just use a legend. This is bad, bad, BAD! Scroll back up to the pie chart example above and pretend you’re actually trying to interpret the data, but pay attention to how many times you look back and forth between the legend and the pie. This is putting a totally unnecessary strain on your brain! Take a look at the horizontal bar chart — no jumping back and forth needed!

With a horizontal bar chart, the label sits right next to the data, and it doesn’t need to be abbreviated to do so (this is one reason that I find horizontal bar charts to be better than vertical column charts in many cases — with a horizontal orientation, the labels have more width with which to work).

3. Those Pesky Near-Zero Values

Pie charts suck at the small percentages. Small percentage categories wreak havoc on the labeling issue, for sure, but they’re also nearly impossible to compare to each other. In the example above, the smallest percentage is 3%, and that’s almost manageable. But, heaven forbid you have a couple of pesky sub-one-percent categories, and you’re looking at wedges that look suspiciously like the lines between wedges.

4. Seeing Small Differences

Fundoogles & Flibbers came in at 3%, while Dracula’s Mickety Micks came in at 5%. Do the wedge sizes really look different? That’s a fundamental challenge with pie charts — we don’t do a very good job of comparing the areas of these odd sorta-triangular-but-with-one-curved-side shapes. In the case of the bar chart, all you have to compare is lengths — much easier.

5. Economy (of Space) Is a Virtue

Check out the overall size of the charts. While they have the same font size, the same text displayed, and the same width, the bar chart is 20% shorter…and it could have been shorter still! Bar charts are more efficient space-wise. With pie charts, and largely because of the other issues listed above, it’s often necessary to make the chart larger and larger to make it readable.

Of Course, This Exampel Was At Least Flat

This post would be twice as long if I went into the additional issues of using the “3D effect” version of the pie chart.

[Update] Always Room for Improvement

Of course, the danger of posting a “here’s a better way” is that you leave yourself open for suggestions as to how the better way can be improved! See Naomi’s comment below. She raises a good point — basically, that I didn’t do a great job of heeding the data-pixel ratio with my bar chart! So, below is a revised version.

bar chart example In a subsequent email exchange, Naomi made the case for keeping the x-axis and the numbers, but simply removing the “%” signs entirely and putting the word “Percent” in the axis label:

Bar Chart Example

Her main point is that numbers can be read more easily if they are not cluttered with symbols like dollar signs and percent signs. And, her case for keeping the gridlines and labeled axis is that it helps show that the bars are drawn to scale — there hasn’t been any incorrect or misleading scaling (intentional or not — in the same spate of presentations that spurred this post, there was a bar chart with an accompanying table of data…and one of the bars was clearly not accurate).

I’m partial to the version with all of the lines removed, but, at this point, the debate is at a much healthier level than “pie vs. bar,” so I’m happy!

Analytics Strategy

Welcome to our newest partner, John Lovett

Posted on December 1, 2009 by Eric Peterson

As you can see by the title of this post Analytics Demystified has some amazingly huge news — respected industry veteran and former Forrester Research senior analyst John Lovett has come on board as a Senior Partner. I have known John for years and have been one of his biggest fans all along; imagine my chagrin when he decided to leave his awesome job and help Aurelie and I build something truly great here at Analytics Demystified!

John has long blogged over at Analytics Evolution but he’s already set up shop here at http://john.analyticsdemystified.com. As you can see he’s pretty excited about joining our team as well, and you can read the official press release on our web site.

At Analytics Demystified I have worked for the past two years and Aurelie and I have worked this year to build a truly great and well-differentiated consulting firm. Lots and lots of companies do implementations, reporting, and the basic block-and-tackle work that is the foundation of our industry. But when I left Visual Sciences I wanted to fill a completely different need. In the past two years we have, I believe, successfully done that.

We count among our clients some of the best companies doing business online, some of the greatest technology firms in measurement today, and some of the nicest people working in web analytics today. Based on the early news from our trusted clients and partners John will only accelerate our growth and allow Analytics Demystified to focus on more strategic and more valuable engagements across the globe. Plus, since John is a total and complete Rock Star like Aurelie, I have a new partner that I know I can trust with the kinds of clients we work with here at Analytics Demystified.

We will have more great announcements from the firm in the coming weeks but I will leave you with these few things:

If you’re a Analytics Demystified client in any kind of retainer, you have immediate and automatic access to both John and Aurelie. Contact me directly for more information;
If you’d like to talk to John about his practice at Analytics Demystified you can email him at john@analyticsdemystified.com;
If you’d like to know more about this announcement and how Analytics Demystified can help your business, please give me a call at (503) 282-2601;
If you’re in Boston and want to congratulate John and buy the man a drink, please join us at Web Analytics Wednesday in Cambridge on TUESDAY, DECEMBER 15TH (sponsored by our friends at Unica and SiteSpect and hosted by Judah Phillips)

I hope everyone reading this blog will join me in welcoming John to the team and take the time to read his “Hello World” post titled “Let the Wild Rumpus Start”. Oh, and feel free to spread the word!

Analytics Strategy, General

Let The Wild Rumpus Start

Posted on December 1, 2009October 25, 2019 by Eric Peterson

I feel liberated. For the first time in my professional career I don’t have to answer to anyone. Sure, I still carry accountability to my new partners Eric and Aurelie, to the Analytics Demystified brand that we will continue to grow and evolve together, and most importantly to myself to produce high caliber work. Yet, there isn’t anyone telling me what to do anymore. Not that I didn’t have autonomy in many of my previous roles…I did. But somehow working in an environment where I’m calling the shots – where the upside is big and the downside threatening – where I have an opportunity to make a difference that’s entirely my own creation – it is invigorating.

Making mischief of one kind and another…
So with this newfound freedom I plan to embark on initiatives and activities that weren’t previously available to me in former roles. And I will challenge the conventional measurement dogma along the way. I mentioned earlier that I intend to be an agent for change in Web analytics. To me that means: questioning the status quo of vendor measurement practices; challenging clients to fully develop their strategic vision for measurement; cultivating talent that instills measurement as a fundamental marketing discipline; and driving the industry to collectively embark on advancement. Simply forging ahead in using analytics and optimization technologies in the way we’ve done so during the past 5 years won’t get this industry to a better understanding of customer behavior and marketing intelligence.

Sailing off through night and day…And in and out of weeks…
Just for context, I’ll let you know that I did not arrive here overnight. I began my career 15 years ago as a marketer and realized that digital mediums could offer faster, better and more effective means of reaching customers. I watched the impact of my digital efforts blossom and realized that web technologies were the way of the future. I embarked on my analyst career back in 1999 by joining two former Forrester analysts at Gomez Advisors where I immersed myself in the online experience. There I learned what it meant to have a well founded digital strategy and consulted with firms on how to formulate one. As Gomez evolved to a performance management company, I continued my consulting and delved into the technical side of what it means to offer faster and more reliable online marketing. It was during this time that I realized that measurement was the basis for truly understanding marketing efforts. This led me to conduct analytics and optimization research at several analyst firms including: the Aberdeen Group, Jupiter Research and most recently Forrester. During my time at each of these companies, my quest remained the same: to help marketers understand how consumers receive, interact and respond to digital marketing – and what to do about it.

To where the wild things are…
That was how I became acquainted with Web Analytics and the industry figures that are indeed the wild things. I’ve commented before that the people of Web Analytics are among the most inviting and hospitable bunch I’ve ever met. I can recall my first conversation with Jim Sterne where I pitched him an idea for eMetrics and his response was “more please, my antenna are tingling…”. Practitioners like Judah Phillips and Jim Hassert were willing to get on the phone and articulate their analytics frustrations along with their successes to help me create research that would resonate with the marketplace. Consultants like Jim Novo and June Li spoke freely about their experiences in education and evangelism of Web Analytics that helped me formulate a perspective on why people cared so much about this industry. And vendors spoke about their technologies with passion and excitement. Every individual that I approached regarding Web Analytics was willing to share a story, some valuable insight or their unique perspective. It was clear that the people that worked within this industry had passion for what they did and I wanted to make myself an indispensable part of that community.

The most wild thing of all…
Among all the characters I met who were involved with Web Analytics one stood out apart from the rest. Eric Peterson seemed to personify Web Analytics. His enthusiasm for analytics and his capacity to evangelize measurement somehow captivated the veterans and newly indoctrinated alike. His passion for Web Analytics was emphatic and his communication tactics resonated. I did and still do appreciate that he takes a stand on his opinions regarding Web Analytics topics, but is still willing to give audience to differing views. He’s also willing to change his opinion if he’s proven wrong. While I’ve accused him of apologizing like Larry David, he will acquiesce when he’s wrong. Most importantly, Eric has made an indelible impact on the Web Analytics industry. Thus, when Eric and Aurelie approached me about joining them as a partner at Analytics Demystified, I couldn’t refuse.

It’s still hot…
So here I am, beginning a new chapter in my career that includes: education, evangelism and inciting change for Web Analytics. I’m here because I truly believe that this space is hot and it’s one that I want to be a part of for years to come. I relish the opportunity to work side by side with Eric and Aurelie because we’re all equally invested in this industry and each offer unique perspectives on where it’s all going. This means that we won’t necessarily agree on everything, but we do share a common view of the big picture. I hope to bring balance to the partnership and the chance to offer my perspective through thought leadership, guidance and evangelism. I’m also looking forward to sharing my experience, my learnings and my viewpoint with you. This industry wouldn’t be here if not for people to move it forward. I welcome conversations about how we can collectively advance measurement technologies and encourage you to reach out and share your views.

And now, let the wild rumpus start!

Analytics Strategy

Recap: Web Analytics Wednesday with Foresee Results

Posted on November 21, 2009October 25, 2019 by Eric Peterson

Last week was our monthly Web Analytics Wednesday in Columbus. Foresee Results sponsored the event and provided a highly engaging speaker: Kevin Ertell, Foresee’s VP of Retail Strategy and the blogger behind Retail: Shaken Not Stirred.

We had a good crowd — just under 30 people — and we did our usual half-hour of networking before sitting down to order food and cover the evening’s topic.

We had attendees from a wide range of companies: Nationwide Insurance, Resource Interactive, Victoria’s Secret (including Monish Datta…which I mention here solely for quasi-inside SEO joke purposes), DSW, Diaz & Kotsev Business (Web) Consulting, WebTech Analytics, Quest Software (makers of Foglight, actually, which I didn’t realize until I was writing the rest of this post), QStart Labs, Submerged Solutions, Bizresearch, Lightbulb Interactive, JoeMetric, Express, Cardinal Solutions, and various independent consultants. By my count, 30% of the attendees were first-timers, and the remaining attendees were a pretty even split between hard-core regulars and every-few-months dabblers.

Kevin is a great speaker — one of those guys whose use of PowerPoint is primarily to provide images that back up the stories he weaves.

One of the stories was the “tree stump on the conference room table” story, which was about how we get used to having odd, not-particularly-helpful aspects of our web sites that are jarring to first-time and infrequent visitors, but that we never think to address.

Tree Stump on a Conference Room Table

You can ping Kevin on Twitter directly for a more complete explanation on that analogy, if you want. If I try to recreate it entirely, I’ll butcher it for sure! I will take a shot at summarizing the four-step process Kevin laid out for going beyond web analytics data to drive site improvement, though, which was the meat of the presentation.

Step One: Ask Your Visitors for Feedback

On-site surveys provide valuable information, because they let you ask your visitors questions directly rather than simply trying to infer what it was they are trying to do, how successful they were at doing it, and how smooth the process was based strictly on behavioral data. Web analytics = bahavioral data. Survey data = attitudinal data. Got it?

Some of the highlights on this step:

Incentives aren’t needed to get people to take a 15-30 question survey — I think Kevin said they see something like 6-10% of the people who are offered a survey actually accept the offer (not all visitors to a site get offered the survey) and they’re able to build up an adequate sample fairly quickly in most cases
The way Foresee Results offers surveys, typically, is that they offer the survey when visitors arrive on the site, but then conduct the survey on exit
The wording of the survey questions matters — there are good/valid ways to word questions and there are bad/invalid ways to word questions; there are oodles of research and expertise on that subject, and it’s worth partnering with someone (a consultant, a company) who really knows the ins and outs on that front to make sure that the data you collect is valid
The Foresee Results secret sauce is that they ask questions that fall into three broad categories: 1) questions about different aspects of the site (content, functionality, navigation, search, etc.), 2) questions to gauge customer satisfaction (very precisely worded questions that are backed up by the research behind The American Customer Satisfaction Index — ACSI), and 3) questions to gauge likely future behavior (likelihood to purchase online, likelihood to purchase offline, likelihood of returning to the site, etc.). Foresee Results then uses an analytic model to link these three elements together: the first category as a dependent variable affecting customer satisfaction, and customer satisfaction, in turn, being a dependent variable affecting the various future behaviors. It’s a pretty nifty tool that I’ve been learning more about over the past few months. Powerful stuff.

This step, done right, gives you the basic diagnostics: where the most significant opportunities for driving improvements exist with your site.

Step Two: Augment Quantitative with Qualitative

This step is to augment the quantitative survey data with more qualitative information. The quantitative data can help you slice/segment the data so that you can review the responses to open-ended questions in a more meaningful way.

Presumably, these qualitative questions are ones that you update over time as you are identifying specific areas on which you want to focus. If for instance, you found out in Step One that the navigation was an area where your site scores low and also has a significant impact on customer satisfaction, then you might want to gather some qualitative data specifically regarding navigation, and you might want to break that out between people who came to the site expecting to make a purchase, as opposed to people who came to the site simply to do comparison shopping.

This sort of analysis will give you insight into the specific friction points on the site — what types of visitors hit them and what sorts of tasks they’re trying to accomplish when they do.

Step Three: Watch Customers (in a Focussed Manner)

This is a step that Kevin pointed out companies sometimes try to put first, which makes it unnecessarily expensive and time-consuming. The key here is to use the information from the first two steps to focus what you are going to observe and how. Various options for watching customers:

Session replay — what exactly did visitors on the site do and how; in the case of Foresee Results, these replays can be tied directly to specific survey respondents (pretty slick), but Tealeaf and Foglight are tools that provide replay functionality, too
Eye-tracking — this requires getting people into a lab of some sort, so, obviously, the more focussed you can get, the better
Usability testing — this may include eye-tracking, but it certainly doesn’t have to; obviously, there are benefits of being able to focus the usability testing, whether it’s conducted in a usability lab or even in-store

Now, you should really have a good handle on specifically what’s not working. But, what if you don’t really have any good ideas as to what to do about it? Then…

Step Four: Usability Audit

Work with usability experts to assess the aspects of your site that are underperforming. Arm them with what you have learned in the first three steps!

To me, it seems like you could swap steps three and four in some cases — let a usability expert audit your site and identify likely opportunities to improve the trouble spots.

Driving Continuous Incremental Improvement

By keeping the survey running on an on-going basis — adjusting questions as needed, but keeping the core questions constant — you can monitor the results of changes to the site as you roll them out. And, of course, your web analytics data — especially on-site conversion data — is one tool for monitoring if you are driving outcomes that matter.

One point on the incremental changes front: during the Q&A, Kevin talked about how sites that roll out major redesigns invariably see a temporary dip in results while visitors get used to the new site. Incremental changes, on the other hand, can occur without that temporary drop in performance.

Interesting stuff!

Excel Tips, Presentation

An Excel Dashboard Widget

Posted on November 16, 2009October 25, 2019 by Eric Peterson

As I wrote in my last post, I’ve been spending a lot of time building out Excel-based dashboard structures and processes of late. I also wrote a few weeks ago about calculating trend indicators. A natural follow-on to both of those posts is a look at the “metric widget” that I use as a basis for much of the information that goes on a dashboard. Below is an example of part of a web site dashboard (not with real data):

Sparkline Widgets

I’ll walk through some of the components here in detail, but, first, a handful of key points:

There is no redundant information — it’s not uncommon to see dashboards (or reports in general) where there is a table of data, and that table of data gets charted, and the values for each point on the chart then get included as data labels. This is wasteful and unnecessary.
Hopefully, your eyes are drawn to the bold red elements (and these highlights should still pop out for users with the most common forms of colorblindness — I haven’t formally tested that yet, though) — this is really the practical application of the vision I laid out in my Perfect Dashboard post.
I have yet to produce a dashboard solely comprised of these widgets — there are always a few KPIs that needs to be given more prominent treatment, and there are other metrics that don’t make sense in this sparkline/trend/current format
I do mix up the specific measures on a dashboard-by-dashboard basis. In the example above, showing the past two years of trends by month, and then providing quarterly totals and comparisons, makes the most sense based on the planning cycle for the client. But, that certainly is not a structure that makes sense in all situations.

And now onto the explanation of the what and why of each element, working our way from left to right.

Metric Name

This one hardly warrants an explanation, but I’ll point out that I didn’t label that column. That was a conscious decision — the fact that these are the names of the metric is totally obvious, and Edward Tufte’s data-ink ratio dictates that, if it doesn’t add value, don’t include it!

Past 12 Months Sparkline

The sparkline is another Tufte invention, and it’s one that has really taken off in the data visualization space. That’s good, because sparklines are darn handy, and the more people get used to seeing them, the less there will need to be any “training” of dashboard users to interpret them. Google Analytics has been using sparklines for a while, even, so we’re well on our way to mass adoption!

Google Analytics Sparkline

One tweak on the sparkline front that I came up with (although I’m sure others have done something similar): I add a second, gray sparkline for either the target or the prior reporting period. I like that this gives a quick, easily interpretable view of the metric’s history over a longer period — has it been tracking to target consistently, consistently above or below the target, or bouncing back and forth? Is there inherent seasonality in the metric (signified by both the black and gray sparklines having similar spike/dip periods)?

One limitation of sparklines is that they don’t represent magnitude very well. If, for instance, a particular metric is barely fluctuating over time, then, depending on how the y-axis is set up, the sparkline can still show what looks like a wildly varying value. It’s a minor limitation, though, so I’ll live with it.

4-Month Trend Arrow

The 4-month trend is the single icon that results from a conceptually simple (but a little hairy to calculate) assessment of the most recent four data points. That was the punchline of an earlier post on calculating trend indicators. Whether the basis of the trend is months, weeks, or days can vary (not within one dashboard, generally, but as a standard for the dashboard overall), as well as whether it’s 4, 5, 6, or more data points. It’s a judgment call for both driven by the underlying business need that the dashboard supports.

I promise, promise, promise to make a simplified example of this arrow calculation and post it in a future post — check the Comments section for this post to see if a linkback exists (I’ll come back and update this entry as well once it’s done)

Current

Typically, when sparklines are used, the exact value of the last point in the sparkline is included. In the example above, I’ve done something a little different, in that I actually provide the sum of the last three data points. This is a quarterly dashboard, but the sparkline has a monthly basis to it to show intra-quarter trends. If the current value is sufficiently below the target threshold, then the value is automatically displayed as bold and red.

There are certainly situations where “Current” would actually be the last point on the sparkline. Like the trend arrow calculations, it’s a judgment call based on the business need that the dashboard supports.

YOY

In the example above, there is a comparison to the prior year. But, this could be a comparison to the target instead. Target-based comparison is even better — straight period-over-period comparisons tend to feel like something of a cop out, as prior periods really are more “benchmarks” than true “targets.” Now, setting a target as something like “15% growth over the prior year” has some validity! That would then impact both the gray sparkline, the “when does Current go bold red,” and this %-based calculation.

28 Data Points

In the version of the widget above, there are 28 unique pieces of data presented for each metric: the metric name (1), the black sparkline (12), the gray sparkline (12), the trend indicator (1), the current value (1), and the year-over-year growth percentage (1). And that’s not counting the conditional formatting that highlights values as bold and red when certain criteria are met. That’s a key aspect of the widget design. 28 sounds like a lot of data to represent for a single metric. Yet, they seem pretty digestible in this format, don’t they?

Let me know what you think. Does this work? What doesn’t work?

Adobe Analytics

Internal Search Tips

Posted on November 16, 2009February 6, 2020 by Eric Peterson

A few weeks ago, Ben Gaines (@OmnitureCare) wrote a great blog post about tracking Internal Search. In this post, I am going to add a few additional tips I have learned over the years…

Correlate Internal Search Term & Page Searched From
Knowing what people searched for on your site is certainly valuable, but knowing the exact page they searched for each term from is even more valuable. Having this allows you to see what content visitors think they should be able to find on each page. This is like gold to your content folks who can look for terms that are consistently searched for on a specific page and make a case that they need to add or improve content.

Setting up SiteCatalyst to do this is very simple. All you have to do is pass the Internal Search term to a Traffic Variable (sProp) (as Ben showed) and then set a second sProp with the previous page name value (use the Previous Value plug-in) and create a Traffic Data Correlation for these two sProps. When you are done, you will be able to see two cool things:

1) What terms are searched for on a specific page:

2) For any given term, what pages are visitors searching for that term:

Group Internal Search Terms
In Ben’s post, he discussed how to eliminate duplicate terms by taking upper/lower case out of the equation. In addition to this, there are times when you might want to group specific keywords together into buckets since they represent the same type of search. For example, if you manage a travel site, you might want to group all City internal search terms by State and Region so you can supplement your analyses. This is easily done by taking advantage of SAINT Classifications which allow you to bucket your internal Search Keywords however you would like. Here is an example of a SAINT File you could use in the preceding example:

Use Compare Feature to find differences between Dates
Once you are tracking internal search terms, you can use the Date Comparison feature in SiteCatalyst to see how the same internal search terms perform in two different time periods. You access this feature from within the SiteCatalyst Calendar window. Below is an example of looking at how the top internal search terms for September perform in October:

As you can see, by using the date comparison feature, SiteCatalyst will show you the difference between the two time periods so you can be aware of significant changes. Simply click the difference column and you can see the search terms that changed the most/least (depending upon whether you sort ascending or descending).

Use Compare Feature to find differences between Report Suites
In a similar manner, if your implementation has multiple report suites (or ASI Segments), you can use the Compare feature to see how internal search terms vary by suite/segment. For example, if you have a Customer Segment and a Non-Customer Segment, you can see what internal search terms each group is looking for:

In the above report, we can see that Non-Customers are more apt to search for careers, while Customers are more interested in detailed product information.

One cool thing you can do with this is to combine this data with Test&Target by FTP’ing the most popular search terms to a Word Cloud program and having Test&Target show the appropriate Word Cloud based upon a cookie value indicating customer status. That is a great way to proactively use your web analytics data to create a better experience for your users!

Trend Search Page Exits
One way to see how good or bad your internal search results are is to look at how often visitors exit your site on the search results page. While this isn’t a guarantee that your search results are bad, most of my clients agree that search results page exits are not normally an indicator of success! Therefore, I like to trend this and set alerts to monitor this. Here are the steps to do this:

Open your Pages report and find your Search Results page in the list
Click on its name and in the sub-menu choose Paths – Next Page report
Unfortunately, Exited Site might be one of your highest next pages, but in this case it is a good thing since you that makes trending it easier (I haven’t figured out how to trend it id it isn’t in the Top 5!). Once you are looking at your list of Next Pages, click the “Trended” link to see the top five next pages trended.
From here, I usually refine the report to only show the Exited Site and Home Page (for some reason SiteCatalyst won’t let you see just “Exited Site” so you need to have one other value – not sure why – so I normally choose Home Page)
Finally, change your date range and View by (i.e. day, week, month) and you will see a report like the one below where I am trending Exits and clicks to the Home Page by percent over time. You can now add this graph to a dashboard to monitor it over time…

Use Counter eVars!
There are two ways you can use Counter eVars with internal search. First, per my last blog post, you can use the # of Pages Counter eVar concept to track how many pages visitors view prior to doing a search to see how your website design is functioning. I showed this in my last post:

Second, you can track the # of internal searches in a counter eVar so you can see how many internal searches each visitor has done prior to completing your desired success event.

Track Recommended/Filtered Search Results
Many companies provide internal website searchers with recommended search results or filtered results based upon the search term as shown here:

You can use SiteCatalyst to track whether the visitor clicked on your organic links or the recommended/filtered links. All you need to do is add a query string to links in each distinct area and capture that in an eVar when visitors click on these links. For example, the eVar values may be “organic link click” or “filtered link click” which will show you the distribution. You can take it further by passing this to an sProp and correlating it to the search term to see which internal search terms lead to visitors clicking each type of result.

These are just a few of the fun things you can do with internal search tracking…

Analytics Strategy, General

Google Analytics Intelligence Feature is Brilliant!

Posted on November 10, 2009April 12, 2017 by Eric Peterson

Long-time blog readers are likely aware that I’m not prone to writing about individual technologies or product features unless I have the opportunity to break the news about something new and cool (or not, as the case is from time to time.) But once and awhile a single feature comes along that in my mind is so compelling and cool I need to bend my own rules; Google Analytics new “Intelligence” offering is exactly that feature.

Just in case you’ve been living under a rock for the past month and haven’t already heard about “Intelligence” have a quick watch of the following video pulled from the Google Analytics blog:

Pretty awesome, huh? What’s more, now that I’ve had a few weeks to play with the feature and think about it in the context of my published views on the Coming Revolution in Web Analytics, I think that “Intelligence” is one of the most important advances in web analytics since the JavaScript page tag.

While Google is certainly not the first vendor to apply some level of statistical and mathematical rigor to web analytics data, an honor that would likely go to Technology Leaders for their Dynamic Alert product or Yahoo for their use of confidence intervals when exposing demographic data in Yahoo Web Analytics, in my humble opinion Google has done the best possible job making statistical analysis of web analytics data accessible, useful, and valuable.

Some things I really like:

An approachable way to determine confidence intervals via their “Alert Sensitivity” slider. While the implementation doesn’t necessarily impart the level of detail some folks would like, the slider mitigates the prevalent concern that “people won’t understand confidence intervals.”
Great visual cues for alerts, especially when statistically relevant changes are not obvious based on traffic patterns. Sometimes traffic patterns just look like hills and valleys, even when something important is happening — for example, the next figure shows two alerts at the lowest threshold setting on September 16th that, upon exploration, turned out to be great news (that I might have missed otherwise.)

Good visual cues regarding the statistical relevance of the insight being communicated. This is tough since Google is trying to present moderately complex information regarding the underlying calculations and how much emphasis you should be putting on the insight. By showing a relative scale for “significance” I think Google has more or less nailed it.

Google Analytics finally starts communicating about web analytics data in terms of “expectations” instead of absolutes. All of us (present company included) have a tendency to get wrapped up in whole numbers, hard counts, and complete data sets. But we also know that Internet-based data collection just isn’t that accurate, and so any push to get us to start thinking in terms of predicted ranges and estimates is a step in the right direction. For example, I love knowing that on a given day Google Analytics “expects” between 311 and 388 people to come to my site from the UK!

Lots more, including the ability to pivot the views and look from a “metric-centric” and “dimension-centric” perspective, the ability to aggregate on day, week, and month, and the ability to add your own custom alerts based on changes in traffic patterns. Perhaps ironically this last functionality (“Custom Alerts”) is how we’ve all historically thought about “Intelligence” in reporting, and while useful seems somewhat weak compared to Google’s stats-based implementation.

While awesome in it’s first instantiation there are some obvious things that the Great GOOG could improve in the feature. Some ideas include:

More dimensions and metrics, although I believe both Nick and Avinash have commented that they are already working on adding intelligence to other data collected.
Some way to expose confidence intervals and p-values would be useful (perhaps as a mouse-over) so that the increasing number of analysts with experience in statistics could have that data in their back pocket when they went to present results.
Email alerts for the automatically generated insights, for example when “Intelligence” determines that five or more alerts have been generated it would be cool to get an email/SMS/Tweet/Wave notification.
The ability to generate alerts against defined segments, so that I could see the same analysis for different audiences that I’m tracking.

Mostly ticky-tack stuff, but again I’m pretty damn impressed with their freshman effort. I suppose I shouldn’t be surprised since evangelist Avinash has been talking about the need for statistics in web analytics for an awfully long time, but given that so many in our industry have balked at bringing more mathematical rigor to our work (including said evangelist, oh well) it’s encouraging to see Google move in this direction.

What do you think? Are you using “Intelligence”? Is it helping you make better decisions? Do you like the implementation as much as I do? I’d love to hear your thoughts and comments.

Reporting

The Perfect Dashboard: Three Pieces of Information

Posted on November 9, 2009October 25, 2019 by Eric Peterson

I’ve been spending a lot of time lately working on dashboards — different dashboards for different purposes for different clients, with a heavy emphasis on making dashboards that can be efficiently updated. I’m finding that I keep coming back to two key principles:

A dashboard, by definition, fits on a single page — this is straight out of Stephen Few’s book Information Dashboard Design: The Effective Visual Communication of Data; I was skeptical that this was really possible when I first read it, but I’ve increasingly become a believer…with the caveat that there is ancillary data that can be provided with a dashboard as backup/easy drilldowns
The dashboard must include logic to dynamically highlight the areas that require the most attention.

The second principle is the focus of this post.

Actionable Metrics

It’s become cliché to observe that data must be converted to information that drives action. I’ve got no argument with that, but, all too often, the people who make this statement would also see this statement as blasphemy:

Most metrics should drive no action most of the time

Any good performance measurement system is based on a structured set of meaningful metrics. Each of those metrics has a target set, either as a hard number, as a comparison to a prior period, as a comparison to some industry measure, or something else.

Here’s the key, though: most of those metrics will likely come in within their target range most of the time! That’s a good thing, because it is rare that a business is equipped to chase more than a handful of issues at once.

A Conceptual (If Not Realistic) Dashboard

At the end of the day, when your user looks at a dashboard, here’s what they really are hoping to get:

Conceptual Dashboard

This is as actionable as it gets:

Only the areas that are performing well outside of expectations are shown
What’s actually happening is stated in plain English
The person viewing the dashboard has a concise list of what he/she needs to start looking into immediately

Will your users ever tell you this is what they’re looking for? No! And, if asked, the reasons why not would include:

“I need to see everything that is going on — not just the stuff that is performing outside targets (…because I’m just not comfortable trusting that we designed a dashboard that is good enough to catch all the things that really matter).”
“My boss is likely to ask me about her specific pet metric…so I need to have that information at my fingertips, even if it’s not going to drive me to take new action.”
“I need to see all of the data so that I can identify patterns and correlations across different aspects of the marketing program.”

Arguing any of these points is an exercise in futility. Between the explosion of data that is available, the fact that not a day passes without a Major Marketing Mind talking about how important it is for us to leverage the wealth of data at our fingertips, and the fact that humans are inherently distrustful of automation until they have seen it working successfully for an extended period of time, all mean that a dashboard, in practice, has to include a decent chunk of information that will not drive any new action.

But the Concept Is Still Useful

I believe the conceptual dashboard above is a useful guiding vision. At the end of the day, we want to provide, and our users want to receive, information that is clear and concise, which the dashboard above certainly is. if we morph the concept above just a little bit, though, we get a dashboard that is only slightly less impactful but heads off all of the concerns listed earlier:

Conceptual Dashboard

Get the idea? The same highlights pop, but additional data is included, and it all still fits on a single page. Obviously, the real dashboard would be one step further diluted from this by presenting actual metrics — numbers, sparklines, etc. But, by working hard to keep all of the on-target data as muted as possible, some clever use of bold and color through conditional formatting can still make what’s important pop.

Parting Thoughts and Clarifications

Any dashboard, whether it’s managed through an enterprise BI tool, through Microsoft Excel, or even through PowerPoint, should be designed so that the structure of the dashboard does not change from one reporting period to the next — the same metrics appear in the same place week in and week out. BUT, within that structure, there should be a concerted effort to make sure that the metrics that are the farthest off target (usually the ones that are the farthest off target in a bad way, but if something is off the charts in a good way, that needs to be highlighted as well) are what the user’s eye is drawn to. And, furthermore, those are the metrics that warrant the first pass of drilling down to look for root causes.

Adobe Analytics

# of Pages Viewed Counter eVar

Posted on November 9, 2009February 6, 2020 by Eric Peterson

This week I will round out my Pages in Conversion trilogy by discussing a # of Pages Viewed Counter eVar. Two posts ago I discussed some of the benefits of setting a Page View Success Event and in my last post I showed some of the cool things you can do by setting a Page Name eVar. While this post will not be as “meaty,” I wanted to share a quick tip that can help you out for a few cool analyses. If you haven’t already, I suggest you read my last two posts as it might be helpful.

Counter eVar Refresher
About a year ago I blogged about what a Counter eVar was in the following Counter eVar post. If you are unfamiliar with Counter eVars, I suggest you review that post before continuing. In a nutshell, a Counter eVar allows you to increment an eVar with a numeric value (usually incrementing by one) when a specific action takes place. For example, if you would like to count how many times website visitors conduct searches on your website prior to adding an item to the shopping cart, you would use a Counter eVar to store a numeric value in each website visitor’s cookie so that when the Cart Addition Success Event takes place, you can associate that number of internal searches with that Cart Addition.

# of Pages Viewed Counter eVar
OK. So now let’s get into this week’s topic. I often like to set a Counter eVar on each page of the website so I have a running count of how many website pages the current visitor has viewed. Setting this is pretty simple as you only need to set a Counter eVar to “+1” on each Page View and if you are setting a Page View Success Event it can be done concurrently. So what does setting this # of Pages Viewed Counter eVar get you? Well, no matter what Success Events take place on your website, it may be interesting to see how many pages the active visitor has/had viewed prior to that Success Event taking place. For example, let’s say you are trying to drive website Lead Capture Form Completions and you want to know if Forms are being completed relatively quickly (after 1-3 pages) or taking more time (after 10 pages). This is not easy to do with out-of-the-box SiteCatalyst reports. You can see Average Page Depth of each Form Page, but I find that very limiting. Using this Counter eVar, you have a simple, clean way to see how many pages visitors had seen prior to completing a Form (in this example):

page_counter_1

However, you get more than just this. Since all eVars break down all Success Events, this one Counter eVar will work with all of your Success Events so you can see any Success Event broken down by # of Pages Viewed. All you have to do is add a different Success Event to the report above and you can see how many pages it took visitors to perform that action. In the following example, we can now see how many pages visitors see prior to performing an Internal Search:

page_counter_2

In this case, we can see that about 50% of all Internal Searches are taking place in the first four pages that visitors see. This could be expected, but if your goal is to improve page content and navigation, this might be an indicator of how well your changes are doing over time…

Use with Subrelations
As they used to say in the commercials: “but wait…there’s more!” The above reports only scratch the surface of what you can do with this new Counter eVar. For example, let’s imagine in the first example above that we now want to see how many pages it takes to get visitors to complete a specific website form. If you are capturing the name of the Forms on your website in another eVar and it has Full Subrelations, you can see the following:

pages_counter

Again, the same concept would apply to other Success Events so in the Internal Search example above, you can use subrelations to see how many pages website visitors had viewed prior to searching on a specific phrase.

Don’t Forget Classifications
One quick little enhancement to the # of Pages Viewed eVar is that you can use SAINT Classifications to bucket pages viewed into more manageable groupings. For example, you can see the same Form Completions report above in more concrete buckets by using SAINT to see the following:

pages_counter2

Final Thoughts
That is the quick overview of setting a # of Pages Viewed Counter eVar. Here are a few final things to keep in mind:

As with all eVars, you need to determine when it will expire. I tend to like to keep the # of Pages to an expiration that is longer than a visit so if a visitor comes back multiple times, you can see how many total Page Views they had done across more than the current visit. You can use “Never” to see all Page Views or if you have one key Success Event, you can expire the eVar at that event and then start the counter over.
You can use the same SAINT Classification file for all Counter eVars so if you create it once, be sure to re-use it.

Adobe Analytics, General

Page Name eVar

Posted on November 2, 2009February 6, 2020 by Eric Peterson

In my last post, I described some of the benefits of using a Page View Success Event. In this post I will continue along the same theme by describing the benefits/uses of a Page Name Conversion Variable (eVar). I recommend you read my last post on the Page View Success Event prior to reading this post as the two go hand-in-hand.

Setting a Page Name eVar
Setting the Page Name in an eVar, while somewhat nontraditional, can be used for many different purposes. In this post I will cover just a few, but I am sure those reading this can come up with many more. The implementation of this couldn’t be easier. Simply pass the s.pagename value to an eVar and you are done! The following sections will outline how I use this variable once it is set.

Campaign Pages
Let’s say that you are running a bunch of online marketing campaigns and you want to see how many pages on the website people coming from each Campaign Tracking Code view. In SiteCatalyst, the main way to figure this out would be to use DataWarehouse, ASI or Discover unless you read my last post and had set a Page View Success Event. But now let’s take it a step further. What if you want to see the pages that visitors from each Campaign Tracking Code viewed on your website. Easy right? Not so fast. There is really no easy way to see this in SiteCatalyst using out-of-the-box reports. One way to do this would be to use the Get&Persist Plug-in to pass the Campaign Tracking Code to a Traffic Variable (sProp) on each page of the visit and then use a Traffic Data Correlation to correlate this new sProp with the Page Name variable, but that is a lot of work! The other way is to use a Page Name eVar. By default, your Campaign Tracking Code report will store and persist the Campaign Tracking Code for multiple page views (you choose your time frame in the Admin Console) so if you begin to store Page Names in another eVar, you will have an intersection between Page Name and Campaign Tracking Code on each page. That allows you to use a Conversion Variable Subrelations report to see all Pages viewed by visitors coming from each Campaign Tracking Code You can see this by opening up the Campaign Tracking Code report, selecting the Page View (Event) metric and clicking the icon next to a specific Tracking Code to break it down by the Page Name eVar. Once you have done this, you should see a report like this:

page_evar_code

Channel Pages Tracking
If you role up your Campaigns to higher-level Marketing Channels using SAINT Classifications you can use the concept from the Page View Event post to see how many pages are viewed on your site after visitor arrive from each Marketing Channel.

page_evar_channel

You can then break this report down by the Page Name eVar to see the most popular pages for each Marketing Channel:

page_evar_channel2

While this is not as granular as viewing Pathing by Campaign (as I demonstrated in this post) , it can give you a high-level view of what pages are popular for each different marketing channel. If you are using the Unified Sources DB VISTA Rule or Channel Manager plug-in, it gets even better as you can see what pages people coming from another website or SEO are viewing on your website by breaking down a particular SEO keyword or external website link by Page Name:

page_evar_channel3

Internal Search Follow-On Pages
If you are properly tracking Internal Search on your website, you should have Internal Search Terms stored in an eVar so you can use this concept to break down Internal Search Terms by this new Page Name eVar (while using the Page View Event) to see what pages visitors view after they search on each specific Internal Search Term:

page_evar_search

What Page Does Success Take Place?
Another side-benefit of setting a Page Name eVar is that you can see on which page a Success Event takes place. For example, if you set a “File Download” Success Event and a file is available on several pages, you can subrelate each file name with the Page Name eVar to see which page is the most popular for downloading each file.

Conversion Variable QA
Finally, there is a completely different use for the Page Name eVar – Quality Assurance. Often times, you will run into situations where you have eVars that have bad data or no data at all (the dreaded “None” row!). Often times, these issues are hard to troubleshoot. However, if you have a Page name eVar, your life is much easier.

Let’s say that you have forms on your website and when visitors complete a form, they are required to enter a “Company Size” field which is stored in an eVar. However, there are many cases where you are seeing the Form Company Size eVar with no data. This might mean that IT forgot to make the field required on some of the Forms (would never happen right?). How do you figure out which forms are causing the issue? All you have to do is the following:

Open the eVar report that has data issues with a relevant Success Event metric (Form Company Size and Form Completes in this example)
Find the row that has bad data or no data (“None” row)
Click the breakdown icon to break the report down by the Page Name eVar
The resulting report (see below) will show you a list of Page Names where SiteCatalyst set the Form Complete Success Event, but did not have a corresponding Form Company Size eVar value

page_evar_qa

You can then send this report to your IT team to help them find pages where there may be tagging issues. You could even schedule this as a recurring report to you and IT so you are alerted when similar issues arise in the future, which helps with overall data quality. Keep in mind that this will only work if the eVar you are looking at has Full Subrelations or you add Full Subrelations to the Page Name eVar (see below).

Final Thoughts
As you can see, there are many different uses of this functionality. The following are some final pointers related to this topic:

As previously noted, if you plan to use the Page Name eVar extensively for testing, I would recommend that it have Full Subrelations so you can QA all eVar reports, not just those that already have Full Subrelations.
In one of the rare times I ever tell clients to do this, I would recommend that you set the Page Name eVar to expire at the Page View in the Admin Console. Expiration beyond that will probably add little value and only slow down reporting. There are some special things you need to do here if you use Custom Links so I would advice you speak to Omniture Consulting about this.
Consider Classifying the Page Name eVar by Page Type, Page Product Category, etc… to increase the value you get from this eVar.

Reporting

Measurement Strategies: Balancing Outcomes and Outputs

Posted on October 26, 2009October 25, 2019 by Eric Peterson

I’m finding myself in a lot of conversations where I’m explaining the difference between “outputs” and “outcomes.” It’s a distinction that can go a long way when it comes to laying out a measurement strategy. It’s also a distinction that can seem incredibly academic and incredibly boring. To the unenlightened!

Outputs are simply things that happened as the result of some sort of tactic. For instance, the number of impressions for a banner ad campaign is an output of the campaign. Even the number of clickthroughs is an output — in and of itself, there is no business value of a clickthrough, but it is something that is a direct result of the campaign.

An outcome is direct business impact. “Revenue” is a classic outcome measure (as is ROI, but this post isn’t going to reiterate my views on that topic), but outcomes don’t have to be directly tied to financial results. Growing brand awareness is an outcome measure, as is growing your database of marketable contacts. Increasing the number of people who are talking about your brand in a positive manner in the blogosphere is an outcome. Visits to your web site is an outcome, although if you wanted to argue with me that it is really just an aggregated output measure — the sum of outputs of all of the tactics that drive traffic to your site — I wouldn’t put up much of a fight.

Why Does the Distinction Matter?

The distinction between outputs and outcomes matters for two reasons:

At the end of the day, what really matters to a business are outcomes — if you’re only measuring outputs, then you are doing yourself a disservice
Measuring outputs and outcomes can help you determine whether your best opportunities for improvement lie with adjusting your strategy or with improving your tactics

Your CEO, CFO, CMO, COO, and even C-3PO (kidding!) — the people whose tushes are most visibly on the line when it comes to overall company performance — care that their Marketing department is delivering results (outcomes) and is doing so efficiently through the effective execution of tactics (outputs).

Campaign Success vs. Brand Success

Avinash Kaushik wrote a post a couple of weeks ago about the myriad ways to measure the results of a “brand campaign.” Avinash’s main point is that “this is a brand campaign, so it can’t be measured” is a cop-out. If you read the post through an “outcomes vs. outputs” lens, you’ll see that measuring “brand” tends to be more outcome-weighted than output-weighted. And (I didn’t realize this until I went back to look at the post as I was writing this one), the entire structure of the post is based on the outcomes you want for your brand — attracting new prospects, sharing your business value proposition more broadly, impressing people about your greatness, driving offline action, etc.

Avinash’s post focuses on “brand campaigns.” I would argue that all campaigns are brand campaigns — while they may have short-term, tactical goals, they’re ultimately intended to strengthen your overall brand in some fashion. You have a strategy for your brand, and that strategy is put into action through a variety of tactics — direct marketing campaigns, your web site, a Facebook page, press releases, search engine marketing, banner ads, TV advertising, and the like. Many tactics are in play at once, and they all act on your brand in varying degrees:

Tactics vs. Brand

And, of course, you also have happenstance working on your brand — a super-celebrity makes a passing comment about how much he/she likes your product (or, on the other hand, a celebrity who endorses your product checks into rehab), you have to issue a product recall, the economy goes in the tank, or any of these happen to one of your competitors. You get the idea. The picture above doesn’t illustrate the true messiness of managing your brand and all of the other arrows that are acting on it.

Oh, and did I mention that those arrows are actually fuzzy and squiggly? It’s a messy and fickle world we marketers live in! But, here’s where outcomes and outputs actually come in handy:

In a perfect world, you would measure only outcomes for your tactics…which would mostly mean you would actually measure at some point after the arrows enter the brand box above, but…
You don’t live in a perfect world, so, instead, you find the places where you can measure the brand outcomes of your tactics, but, more often than not, you measure the outputs of your tactics (measuring closer to the left side of the arrows above), which means…
You actually measure a mix of outcomes and outputs, which is okay!

Tactics are what’s going on on the front lines. Their outputs tend to be easily measurable. For instance, you send an e-mail to 25,000 people in your database. You can measure how many people never received it (output — bouncebacks), how many people opened it (output), how many people clicked through on it (output), and how many people ultimately made a purchase (outcome). Except the outcome…is probably something you wildly under count, because it can be darn tough to actually track all of the people for whom the e-mail played some role in influencing their ultimate decision to buy from your company. The outputs can also be measured very soon after the tactic is executed (open rate is a highly noisy metric, I realize, but it is still useful, especially if you measure it over time for all of your outbound e-mail marketing), whereas outcomes often take a while to play out.

At the same time, if you ignored measuring the tactics and, instead, focussed solely on measuring your brand, you would find that you were measuring almost exclusively outcomes (see Avinash’s post and think of typical corporate KPIs like revenue, profitability, customer satisfaction, etc.)…but you would also find that your measurements have limited actionability, because they reflect a complex amalgamation of tactics.

So, What’s the Point?

Measure your brand. Measure each of your tactics. Accept that measurement of the tactics is heavily output-biased and measurable on a short cycle, while measurement of your brand is heavily outcome-biased and is a much messier and sluggish beast to affect.

Watch what happens:

If your brand is performing poorly (outcomes), but your tactics are all performing great (outputs), then reconsider your strategy — you chose tactics that are not effective
If your brand is performing poorly (outcomes) and your tactics are performing poorly (outputs), then scrutinize your execution
If your brand is performing well…cut out early and play some golf! Really, though, if your tactics are performing poorly, then you may still want to scrutinize your strategy, as you’re succeeding in spite of yourself!

The key is that tactics are short-term, and driving improvement in how they are executed — through process improvements, innovative execution, or just sheer opportunism — is an entirely different exercise (operating on a different — shorter — time horizon) than your strategy for your brand. Measure them both!

Adobe Analytics

Page View Success Event

Posted on October 26, 2009February 6, 2020 by Eric Peterson

Those who are familiar with Omniture SiteCatalyst come to learn that there is a big difference between Traffic Variables (sProps) and Conversion Variables (Events & eVars). This separation can get in the way of many analyses unless you have access to Omniture Discover. In this post I will describe one of the methods you can use to blur the lines between Traffic and Conversion variables by describing the use of the Page View Success Event.

Why a Page View Event?
In my experience as an Omniture Consultant, I found that most non-Media clients did not utilize a Page View Success Event. However, there are great reasons why all Omniture clients can benefit from a Page View Success Event. So what exactly is a Page View Success Event? It is basically nothing more than setting a Success Event on every page of your site (kind of obvious huh?)! Setting a Page View Success Event is very easy and can be done with minimal updates to your JavaScript file. Once set, you will have a Page Views metric in your list of Success Events and this metric should, for the most part, be the same as the Page View metric you would see in the Traffic reports area.

OK. So now you have the same metric you already had. Doesn’t sound very exciting does it? Despite its simplicity, it actually can be very beneficial. Here are a few examples of what you can now do with this newly created Page View Event:

Basic Visitor Engagement
Let’s say that you are running a bunch of marketing campaigns and in addition to seeing how many purchases or leads each campaign generates, you also want to see how many pages on your site those arriving from each Campaign viewed. Sounds easy right? But how would you do this? In the Traffic report area, there is no easy way to do this without using DataWarehouse, ASI or the Get&Persist Plug-in. However, now that you have a Page View Success Event, you can open up your Campaigns report and in addition to your other site Success Metrics, you can add Page Views to see the number of Page Views visitors viewed on your site. Having this metric is a [very] rudimentary way to view Visitor Engagement for Campaigns.

pv_engage

If you use the Unified Sources Vista Rule (or the JavaScript version known as the Channel Manager) to roll-up your traffic sources, you can use the Page View Event to see which Marketing Channel leads to the highest Number of Page Views on your site:

pv_channel

In addition, you can see how different Campaign Tracking Codes performed against each other with respect to Page Views by opening the same report at the Tracking Code level. For example, you can use this concept to see how many Page Views a particular Google Paid Search Keyword leads to on your site by finding that particular keyword in the Tracking Code report:

pv_code

Internal Campaign Page Views
So earlier I mentioned that most Media clients set a Page View Event by default. The reason they do this is that for most Media clients, Page Views represent how they make money! They usually get paid for every Page View (through online advertising) so Page Views is one of their most important success metrics. However, even if you aren’t a Media client you can use the same concept on your site. Though you may not sell advertising on your site, you are most likely showing display ads for web promotions or to drive people to other areas of your site. If you set an Internal Campaign code each time someone clicks on one of your on site promotions, you can use the Page View Event to see how many Page Views on your site each internal display ad leads to.

Basic Success Event Efficiency
Every once in a while, a client will ask me to do some analysis on how many Page Views it takes for visitors to complete website Success Events. I have seen many clients try to answer this question by opening the Calculated Metric window and trying to create a metric that divides a particular Success Event by Page Views only to be puzzled why Page Views is not an option in the metric selector! Why isn’t Page Views there? Because Page Views is a Traffic metric and your Success Events are conversion metrics and you can’t mix those in SiteCatalyst. However, if you have a Page View Success Event, you can easily create a Calculated Metric to see this. Simply divide any existing Success Event metric by the Page View Success Event and you will be able to see a trended ratio that compares the two. I call this Success Event Efficiency (at a high level, how many pages do visitors need to see for each Success Event) and it can provide an alternative view of how your site is performing.

pv_efficiency3

Internal Search Term Influence
Ever wonder how many pages on your site people view after searching on a particular term using your site’s internal search? How about seeing this for each internal search term? While this sounds easy, it isn’t with an out-of-the-box implementation. But if you have a Page View Success Event and are already storing internal search terms in an eVar, all you have to do is open the Internal Search Term eVar report and add the Page Views Success Event to the report. This metric will show you how many pages were viewed on your site after each internal search term was passed to the eVar.

pv_int_search2

Sort by Page Views and you can see which internal search term led to the most site Page Views (keep in mind that the length of time you select to to expire the internal search term eVar will impact how many page views are shown, but most people expire this at the end of the Visit). Another thing I have done is divide the number of Pages Viewed by the number of Internal Searches [Success Event] to see how many pages visitors tend to view after searching on each specific term (see 3rd column above).

Final Thoughts
As you can see, adding a Page View Success Event has many different uses and I have only touched upon a few here. I encourage you to consider if you have a use for this for your SiteCatalyst implementation. If you decide to do this, I would recommend the following:

Be sure to name the Success Event appropriately so you don’t confuse it with the Traffic Page View metric. I tend to name the Page View Success Event as “Page Views (Event).”
Keep in mind that when you set other Success Events you need to have both in the s.events string separated by a comma (i.e. s.events=event1,event10). You want to be sure you don’t lose your key website Success Events when you implement this!
Keep in mind that setting multiple Success Events can have an impact on latency so be sure to talk to your Account Manager if your site has a lot of traffic.
It is not recommended that you set a Page View event when using Custom Links.

Adobe Analytics

Internal Campaigns

Posted on October 19, 2009February 6, 2020 by Eric Peterson

By default, most Omniture SiteCatalyst clients are tracking their external Marketing Campaigns using Campaign Tracking. These reports allow you to see how many Success Events take place on your site for each type of Campaign you run (i.e. E-mail, Paid Search, etc…). However, I am surprised how rare it is that Omniture clients are tracking their Internal Campaigns (also referred to as Internal Promotions) to the same extent. Most websites promote products or content on their site through the use of display ads, buttons or links. These Internal Campaigns should be tracked in the same way as external campaigns. While I have touched upon this concept a bit in the past in the Conversion Variable post and the Products Variable post, in this post, I will provide the basics on Internal Campaign tracking.

Why Track Internal Campaigns?
So why should you track Internal Campaigns? At most organizations, there is constant debate about which website promotions perform better than others. This is especially the case for high-profile pages like the Home Page. For example, the screen shot below shows four distinct Internal Campaign Promos:

While you can try to see how often visitors are clicking on each promotion item by looking at Pathing reports (look how many people went from Page A to Page B where you had a promotion on Page A), this takes a lot of time and won’t help you if you have multiple links to this same destination page on the same page. You can try to use the ClickMap feature of SiteCatalyst, but in my experience, ClickMap data is not wholly accurate. If you have a tool like Test&Target then you can easily test and promote content that is proven to be the best in each content area, but if you don’t, you can use Internal Campaign tracking to provide some basic information.

How to Track Internal Campaigns?
Tracking Internal Campaigns is done through an eVar. As I have pointed out in the past, the s.campaigns variable in SiteCatalyst is really nothing more than a predefined eVar with Full Subrelations. Therefore, you can track Internal Campaigns in the same way. I tend to do this using the getQueryParameter plug-in which captures a code placed in the URL and passes it to the Internal Campaigns eVar. These codes can be whatever you like, but the parameter identifier should be different from what is used for external campaigns. In the fictitious example shown here, a user has clicked on a website banner and the destination URL has a “pid” parameter which passes the code “home_hero_112” to the Internal Campaigns eVar:

As you can imagine, the hardest part of Internal Campaign Tracking is adding tracking codes to each promotion link on your site. However, this can be built into the process of banner/promo creation and done on a going forward basis if needed. All you need to do is to come up with a logical naming convention or if you want, you can even just use numeric codes and use SAINT Classifications to add meta-data later. When using SAINT for Internal Campaigns I tend to use the following Classifications:

Page on which the promo banner was shown
Location on page of promo banner
Format (i.e. GIF vs. Flash)
Creative Copy (i.e. $50 off vs. 10% Discount)
Owner of the Promo

How to Use Internal Campaigns?
Once you are passing Internal Campaign codes to an eVar, it is time to use the data for analysis. The most basic way to do this is to open the Internal Campaigns eVar report and look to see how many of your website Success Events take place after a visitor clicks on one of your Internal Campaign elements. You can see an example of this in the following report:

internalcamp_3

In this example, I have set an additional “Internal Campaign Clicks” Success Event to track each time a visitor clicks on an Internal Campaign promo item. You could rely on the “Instances” metric, but as I have stated in this post, I am not a big fan of this. This new “Internal Campaign Clicks” metric is an internal equivalent to the Clicks metric set by default for External Campaigns.

However, there is one difference between Internal and External Campaigns to keep in mind. Unlike External Campaigns that usually have one value per visit, visitors can click on multiple Internal Campaigns within one session. Therefore it is important that you understand the principles of eVar Allocation so you understand which Internal Campaign element will get credit for website Success Events. If you want to go really deep with Internal Campaigns, you can even set multiple eVars such that you have the following:

One eVar to store the first Internal Campaign clicked in a visit (First Value)
One eVar to store the last Internal Campaign clicked in a visit (Most Recent)
One eVar to store all Internal Campaigns clicked in a visit (Linear) [remember that Linear Allocation is only Visit-based!]
One eVar to store all Internal Campaigns clicked across multiple visits using Cross-Visit Participation

One of my favorite reports to run is one in which I look for synergistic effects between External and Internal Campaigns. Since the External Campaigns eVar comes with Full Subrelations, you can automatically break it down by the Internal Campaigns variable. Doing this allows you to see which combinations of External campaigns and Internal Campaigns lead to success. For example, it may be the case that a particular Paid Search Keyword, when combined with a specific Internal Campaign promo converts above the average for the site. These hidden gems can help you boost overall conversion and are found by simply opening a Subrelation report between the two variables as shown here:

internalcamp_4

Finally, another benefit of tracking Internal Campaigns is that it enables you to improve your building of DataWarehouse Segments to include visitors who have/haven’t seen a particular Internal Promo. This information can be valuable to re-marketing efforts in general.

Analytics Strategy, General

Are You Ready for the Coming Revolution?

Posted on October 18, 2009 by Eric Peterson

Few would argue that the past few years in web analytics have been, well, intense. The emergence of Yahoo Web Analytics, multiple management shake-ups at WebTrends, Adobe’s acquisition of Omniture following Omniture’s acquisition of Visual Sciences, WebSideStory, Offermatica, Instadia, and TouchClarity, and the continued push into the Enterprise from Google Analytics. From where I sit we have seen more changes in the last 24 months than we had in the entire 12 years previous (my tenure in the sector) combined.

When I think about these changes, I find myself coming to the undeniable conclusion that our industry is undergoing a radical transformation. More companies than ever are paying attention to digital measurement, and despite my disbelief in Forrester’s numbers, an increasing number of these companies are forging a smart, focused digital measurement strategy. At the X Change, at Emetrics, and at Web Analytics Wednesday events around the world there is more and more evidence that this wonderful sector I call “home” is really starting to grow up.

And we’re just getting started.

If you pay close attention to the marketing you see from Omniture, WebTrends, Unica, Coremetrics, and the other “for fee” vendors you’ve surely noticed a dramatic change recently. Nobody is talking about web analytics anymore; the entire focus has become one of systems integration, multichannel data analysis, and cross-channel analytics.

All the sudden web analytics is starting to sound like, gasp, business and customer intelligence.

Eek.

Since it’s late and since this post will be over-shadowed by the hype around Google Analytics releasing more “stuff” on Tuesday I’ll cut right to the chase: I believe that we are (finally) on the cusp of a profound revolution in web analytics and that the availability of third-generation web analytics technologies will finally get digital measurement the seat at the table we’ve been fighting to get for years.

Statistics, people … statistics and modeling, predictive analytics based on web data, true forecasting, and true analytical competition for the online channel. Yahoo’s use of confidence intervals when presenting demographic data and the application of statistical models in Google’s new “Analytics Intelligence” feature are just the beginning. As an industry it’s time to stop fearing math and embrace analytical sciences that have been around for longer than many of us have been alive. It’s time to stop grousing about how bad the data is and actually do something about it.

Do I have your attention? Good.

Thanks to the generosity of the kind folks at SAS I have a nicely formatted white paper that is now available for download titled “The Coming Revolution in Web Analytics.” Just so you can see if you might be interested here is the Executive Summary from the document:

“Forrester Research estimates the market for web analytics will be roughly US $431 million in the U.S. in 2009, growing at a rate of 17% between now and 2014. Gartner reports that the global market for analytics applications, performance management, and business intelligence solutions was US $8.7 billion in 2008—roughly 20 times the global investment in web analytics. Among their three top corporate initiatives, most companies are focusing their efforts online, expanding their digital efforts Internet to increase the organization’s presence in the least expensive, fastest growing channel.

Today, a majority of companies are dramatically under-invested in analyzing data flowing from digital channels. Even when business managers have committed money to measurement technology, they usually fail to apply commensurate resources and effort to make the technology work for their business. Instead, most organizations focus too much on generating reports and too little on producing true insights and recommendations, opting for what is easy, not for what is valuable to the business.

Analytics Demystified believes this situation is exacerbated by the inherent limitations found in first- and second-generation digital measurement and optimization solutions. Provided by a host of companies primarily focused on short-term gains in the digital realm, not long-term opportunities for the whole business and their customers. Historically these companies worked to differentiate themselves from traditional business and customer intelligence, focusing on the needs of digital marketers. Unfortunately, as the need for whole business analysis increases, many of these vendors are playing catch-up and forced to bolt-on data collection and processing technology as an afterthought.

The current state of digital analytics is untenable over time, and Analytics Demystified believes that companies that persist in treating online and offline as “separate and different” will begin to cede ground to competitors who are willing to invest in the creation and use of a strategic, whole-business data asset. These organizations are using third-generation digital analytics tools to effectively blur the lines between online and offline data—tools that bridge the gap between historical direct marketing and market research techniques and Internet generated data, affording their users unprecedented visibility into insights and opportunities.

This white paper describes the impending revolution in digital analytics, one that has the potential to change both the web analytics and business intelligence fields forever. We make the case for a new approach towards customer intelligence that leverages all available data, not just that data which is most convenient given the available tools. We make this case not because we believe there is anything wrong with today’s tools when used appropriately, but because we believe digital analytics should take a greater role in business decision making in the future.”

Since I pride myself on the quality of my readership I sincerely hope that each of you will download this document and take the time to read it. More importantly I’d love you to share it with your co-workers, friends, and followers on Twitter. I believe we are at a critical juncture in our practice’s history where the skills that have served us all along are not going to serve us for much longer, but I am always willing to admit that I’m wrong and more than anything I love a spirited debate.

Are you ready for the revolution?

Analytics Strategy

SEO Tips and Thoughts at Web Analytics Wednesday

Posted on October 18, 2009October 25, 2019 by Eric Peterson

Last week’s Columbus Web Analytics Wednesday had something of an odd vibe, but it was also one of the most tactically informative ones that we’ve had to date! The crowd was smaller than usual — 18 attendees — due to a confluence of factors ranging from the influenza virus (not H1N1, as far as I know, but appropriate precautionary non-attendance by several people), to business travel to residential water line leaks, to touching-if-inconveniently-timed spousal romantic gestures! The silver lining is that, to a person, there was genuine regret about not being able to attend the event, which is a strong indication that our informal community of local analysts really has solidified. (Monish Datta was in attendance, so I am able to gratuitously make a reference to him — ask him or me at the next WAW what that is all about, if you don’t already know!)

As for the event itself, we welcomed a new sponsor — Resource Interactive. The topic for the event was search engine optimization (SEO) with a little bit of search engine marketing (SEM). It wasn’t the first time that we relied on Dave Culbertson of Lightbulb Interactive to present, and it likely will not be the last, as his knowledge and enthusiasm about SEO, SEM, and web analytics is both entertaining and informative!

Dave attended SMX East in New York the week before WAW, and he agreed to pull together the highlights of the sessions that he attended. One of my favorite tweets from Dave while he was at the conference was this one:

“Ended up leading a lunchtime discussion on web analytics at #smxeast. Web analytics and SEO – like peanut butter and chocolate!”

Partly because Dave is one of the organizers of Columbus Web Analytics Wednesday, and partly because, well, SEO/SEM and web analytics really should be integrated, “search” is a frequent cornerstone of our WAW topics. Dave’s presentation was titled SMX East 2009: The Spinal Tap Wrap-up. At least half of us (myself included) didn’t get the reference, while a solid quarter of the attendees immediately got it and thought it was quite clever and amusing. There were 11 slides in the deck, so:

The presentation focussed primarily on SEO tips, although there was some SEM here and there. An incomplete list of the nuggets/surprises that jumped out the most to me included:

PageRank sculpting — this is when you try to gently influence the Google PageRank for pages you control by making subtle, behind-the-scenes tweaks to both that page and other pages that you control that link to that page. Apparently, a somewhat common way to do this has been through the use of the NoFollow tag. While this may have worked at one point, Google now pretty much ignores the tag when it comes to assessing PageRank
rel=”canonical” — this is a biggie, especially when it comes to web analytics and campaign tracking; this is a tag that can be added to a page to specify the exact “preferred” URL for the page. It’s important because many pages get linked to or arrived at with one or many extraneous parameters tacked on to the end of the URL: campaign tracking parameters for the web analytics tool, link tracking information for the e-mail engine from which a user may access the page, session ID or user ID information for the application that is rendering the page to enable it to make subtle tweaks in the content, etc. The full adoption of this tag by Google, Yahoo! Search, and Bing should go a long way towards removing the tension that exists between the SEO person pushing for the removal of these parameters in links (to avoid link dilution) and the web analyst who pushes to add them (to improve tracking capabilities). Google put together a nice write-up and video on the canonical tag after SMX West.
keywords — this is “keywords for SEO,” rather than the SEM usage of the term. A lot of information was presented about studies as to where the appearance of a keyword had the most/least impact. Having the keyword in the domain name itself was great, but, of course, you’re not going to be able to do that for too many keywords! (I couldn’t help but thinking of Clearsaleing’s http://www.attributionmanagement.com/ site, though!) Even better is to have the keyword in the domain and in the directory path (i.e., http://www.keyword.com/keyword). Having the keyword in a subdomain (http://keyword.company.com) is apparently not very effective (there was a quick side discussion about an online shoe retailer — and I can’t remember which one it was and, ironically, can’t seem to put together the right Google search to figure it out — that tried creating a subdomain for very type of shoe they sold…which then helped trigger Google to make this not effective; I’m fuzzy on the specifics, obviously!) Another point here is that there is both the “what the search engine algorithm puts weight on keyword-wise” and the “how user behavior — which links users follow — is affected by keywords showing up in subdomains, domains, query parameters, etc.” factor — it’s hard to tease out which is which, so the studies have focussed more on “what actually happens” rather than “why it happens.”

At the end of the day, search engine optimization still comes down to providing great content in a way that users can easily navigate to it and consume it. Google’s algorithms are geared around making the same recommendations that a human being with an infinite knowledge of what content was where on the web would recommend in response to a question from another human being. SEO efforts need to focus on helping that theoretical human out — not trying to fool him/her!

I also distributed copies of the deck that Laura Thieme of Bizresearch presented at SMX East. That presentation was primarily SEM-focussed, but it also had some great nuggets in it. Unfortunately, Laura wasn’t able to attend WAW (see the first paragraph of this post!) this month. Laura presented at WAW back in July and really knows her way around SEM, so we missed having her there!

All in all, it was a good event!

Analytics Strategy

An Apology of Sorts …

Posted on October 15, 2009 by Eric Peterson

Now that Omniture’s Q3 earnings are public that I sort of felt like I needed to apologize to the company or at least recognize that they did a good job last quarter leading into their sale to Adobe Systems. Despite what I had heard from multiple sources their earnings announcement was right in line with guidance. Congratulations to the entire Omniture and Adobe team!

It still leaves me scratching my head about the deal since the synergies are less obvious to me than they clearly are to Adobe and Omniture’s management and shareholders, but hey, with the sheer number of changes occurring in the industry right now who knows what might actually work. Hell, based on what I’m hearing about the Google Analytics announcement next Tuesday, it’s going to look like a great time to be focusing on something other than competing with Google Analytics …

I’m going to get to spend time with many of their largest customers next week so I suspect I’ll hear a great deal more about how this sale is being met by HBX customers, Visual Sciences customers, and those folks who have a tremendous amount invested in the SiteCatalyst line of products. If you’re an Omniture customer going to Emetrics next week and have an opinion you’d like to share please reach out to me directly and we’ll arrange some time to chat.

Again, congratulations to Josh James and all of the OMTR shareholders on what is increasingly looking like a great deal for all involved.

Adobe Analytics

Feature Request: Classifications & Menu Customizer

Posted on October 12, 2009February 6, 2020 by Eric Peterson

A few weeks ago, fellow Omniture SiteCatalyst blogger Jason Egan described some ways to take advantage of the Menu Customization feature in SiteCatalyst. I wrote about how to use the Menu Customizer in this post, but recently I have been wanting to do something new with the feature that doesn’t appear to be possible. In this post I will describe what I am trying to do in hopes that someone else out there knows of a way to do it or if nothing else to get it on the radar of the SiteCatalyst product management team…

Why Classifications Mess up the Menu Customizer
For those of you who are savvy enough to understand SAINT Classifications, you will quickly realize how powerful they can be in a SiteCatalyst implementation. In addition to the normal function of rolling up data into groups, SAINT Classifications are great in cases where the root source of the data is not as “clean” as you would like. For example, imagine a scenario where you are trying to capture two character text strings for US States into a Traffic Variable. Unfortunately, your developer has passed data in using both upper and lower case (i.e. CA and ca). While this can be fixed going forward, there may be reports you need to show that group these together. Therefore, you would create a SAINT Classification and lump both of these values into a value of “California” in the classified report.

So far so good. But what if you wanted to show only the Classification report, but not the source report (the one with the “CA” and “ca”)? Believe it or not, this is actually possible by 1) creating a custom report for the Classification version of the variable and then 2) using the Menu Customizer to put this custom report in the right spot and 3) hiding the original report that was classified. However, there is one major drawback of this approach. If you ever need to have a user perform a break down by that classification, for say a Traffic Correlation, you are completely out of luck since hiding the source report disables the ability to break it down. So in this case, if you had an Internal Search Term report and wanted to break Internal Search Term instances down by state (either the “CA” version or the “California” version), you cannot do it once you have hidden the source report. The same limitation applies to classified Conversion Variables (eVars) and Conversion Variable Subrelations.

Proposed Solution
While the example above is somewhat basic, there are many cases where you might want to show a classified version of a report, but not show the source report of the same variable. Unfortunately, since you cannot even see Classifications in the Menu Customizer yet, I am not optimistic that this will be addressed anytime soon, but in an ideal world, it would be possible to do the following:

See SAINT Classifications in the Menu Customizer as they appear in the regular SiteCatalyst menus and have the ability to hide/move any of them or the source of the Classification. Currently, most of my Classification data appears in a 3rd level fly-out from the menus and it would be great if I could move these reports anywhere I’d like as I can with non-classification reports.
In cases where the source of the Classification is hidden, still allow users to breakdown Traffic and Conversion reports by the classified versions of the source variable
If a Custom Report is created using a variable that has Classifications, allow users to have breakdowns of that report in the same way they would if they were looking at the regular version of that report (i.e. don’t punish users for using the Custom Report functionality!)

If anyone out there has come up with a work-around for this, please leave a comment here…Thanks!

Analytics Strategy, General

New Data on the Strategic Use of Web Analytics

Posted on October 11, 2009April 11, 2017 by Eric Peterson

Recently Google published the results of a Forrester Research study they had commissioned (PDF) to help the broader market understand the use and adoption of free web analytics solution. Google should be applauded for commissioning Forrester to conduct this work, especially given the quality of the research and the level of insights provided. Without a doubt, free solutions like Google Analytics and Yahoo Web Analytics are having an impact on our industry and driving change in ways few of us ever imagined.

I really did enjoy the Forrester report, primarily because the author (John Lovett) managed to surface totally new data. When he first told me that over half of Enterprise businesses were using free solutions I have to admit I didn’t believe him. In a way I still don’t, but perhaps that’s only because I work with a slightly different sample than he presents. Regardless, John’s report paints a picture of an increasingly challenging market for companies selling web analytics and a new sophistication among end users.

Speaking of sophistication, there are a few points in the report that I question, and since I have pretty good luck getting feedback from readers on big picture stories I figured I’d bring them up here in the blog. Before I do I want to emphasize that I am not questioning Forrester or John’s work—I am merely trying to explore some data that I find contrary to my own experience in this public forum. To this end I pose a handful of questions that I would love to discuss either openly in comments or via email.

The first point I question is the observation in Figure 3 that 70% of companies report having a “well-defined analytics strategy.” Two years ago my own research found that fewer than 10% of companies worldwide had a well-defined strategy for web analytics. Last year Econsultancy reported that only 18% of the companies in their sample had a strategy for analytics. To jump from these low numbers to the majority of Enterprises just doesn’t square with my general experience in the industry.

Remember, the implication of this data point is that 70% of all companies having more than 1,000 employees have a “well-defined analytics strategy.” According to a 2004 report from the U.S. Census Bureau there were just over 12,000 companies in the U.S. with more than 1,000 employees. Without assuming any growth between 2004 and 2009, Forrester’s 70% figure would result in over 8,500 companies in the U.S. that have a “well-defined” strategy for web analytics. Does that sound right to you?

Consider that the combined customer count for Omniture, WebTrends, Coremetrics, and Unica combined in the U.S. doesn’t even add up to 8,500 companies. Even if you use the more conservative 13% who “strongly agree” with Forrester’s statement you end up with over 1,500 U.S. companies. I may suffer from sample bias, but personally I can barely think of 150 companies that I would identify as having any strategy for web analytics, much less a “well-defined” one.

Most companies I talk to have the beginnings of an over-arching strategy—they’ve realized the need for people and are beginning to reduce their general reliance on click-stream data alone. But given that I think about this topic from time to time, I think a “well-defined” strategy for web analytics takes into account multiple integrated technologies, appropriate staffing, and well thought-out business and knowledge processes for putting their technology and staff to work. What does the phrase “well-defined strategy” imply to you?

Similarly, if 60% of companies truly believed that “investments in Web analytics people are more valuable than investments in Web analytics technology” there would be THOUSANDS of practitioners employed in the U.S. alone. But again, every conference, every meeting, every conference call, and every other data point suggests that the need for people in web analytics is still an emerging need. Hell, Emetrics in San Jose earlier this year barely drew 200 actual practitioners by my count. How many web analytics practitioners do you think there are in the United States?

Same problem with the rest of the responses to Figure 3 on web analytics as a “technology we cannot do without” (75%) and the significance of the role web analytics plays in driving decisions (71%). Perhaps I’m talking to entirely the wrong people, perhaps I’m interpreting these data wrong, and perhaps I’ve gone flat-out crazy, but these responses just don’t match my personal understanding and experience in the web analytics industry.

This issue of data that simply does not make sense, while not universally manifest in the report, manifests elsewhere as well. For example, Figure 8 reports on the percentage of application used segmented by fee and free tools:

When I look at these responses and see that 63 percent of respondents using fee-based tools and 50 percent of respondents using free tools claim to be effectively using more than half the available functionality, again I find myself scratching my head. As this data appears to speak to the general sophistication of use of analytics I went back and looked at Dennis Mortensen’s quantitative study of how IndexTools was being used around the world.

Dennis reports that fewer than 10% of his customers were using even the most basic “advanced” features in web analytics (report customization) and that fewer that 4% of his customers (on average) are making any “advanced” use of the IndexTools application. While this dataset is somewhat biased towards European companies who I believe, on average, to be somewhat behind their U.S. counterparts it does provide an objective view in how web analytics are used that seems to directly contradict the self-reported responses in Forrester’s figure 8.

Clearly there is a gap between the responses John collected and the current state of the web analytics market. Since John is a very smart guy I know part of his rebuttal will include the observation that he surveyed people directly responsible for web analytics (see Forrester’s methodology) and that people in general have a tendency towards positivism. Trust me, my son is the most handsome little boy ever born and my daughter’s beauty is only matched by that of Aphrodite … same for your kids, right?

Given the difficulty associated with gathering truly objective data regarding the use of web analytics, this type of self-reported data is usually what we have to go on. While Omniture, WebTrends, Coremetrics, and Unica all have the fundamental capability to report data similar to that provided by Mr. Mortensen, it may not be in their best interests to expose underwhelming adoption and unsophisticated use (if that is what the analysis uncovered.) Ultimately we’re forced to accept these self-reported responses and then reconcile them against our own views, which is why I’m asking my readers what they think about the data Forrester is reporting!

Regarding these self-reported attitudinal responses on how web analytics is used strategically, perhaps the truth is found in the companies who “strongly agree” with John’s statements. If we apply this lens, as opposed to the more optimistic view, we get the following:

17% of companies recognize that web analytics is a technology they cannot live without;
Web analytics plays a significant role in driving decisions at 12% of companies;
13% of companies have a well-defined web analytics strategy;
9% of companies recognize that investments in people are more valuable than investments in technology

These numbers start to make a lot more sense to me. Likely the truth, as with so much in our industry, lies somewhere in between, but I would love to hear what you think about these adjusted numbers. Do the lower numbers make more sense to you, or do you agree with John’s more optimistic assessment?

Unfortunately if the lower numbers are correct the implication is that despite the incredibly hard work that companies, consultants, and industry thought-leaders around the world have done for years we still have an incredibly long way to go before web analytics is recognized as the valuable business practice that you all know it can be!

Regardless I want to state that I do not disagree at all with the fundamental thesis in this report, that “free” is creating a whole new level of interest in web analytics and that, given proper consideration, free is an excellent alternative to paid solutions. Lacking clear strategy and resources, too many companies have wasted too much money on paid solutions for free to not be compelling. Thanks to the dedication of the Google and Yahoo teams, the world now has access to great applications that are in some regards more compelling than fee-based alternatives.

While I may not have said this a few years ago, today I honestly do believe that “free” is a viable and appropriate alternative to fee-based solutions. While not appropriate in every situation, it is irresponsible to suggest that any company not willing to fully engage in web analytics should pay for ongoing services and support. Given advances from Google and the availability of Yahoo Web Analytics, any motivated company large or small now has access to a wealth of data that can be translated into information, insights, and recommendations.

Conversely I agree with John (and Jim, and almost ever thought leader I respect) who states that you need to “prioritize your business needs and culture for analytics first and then evaluate the tools.” This goes back to the fundamental value proposition at Analytics Demystified: It’s not the tools you use but how you use them. If you’re not invested in developing and executing a clearly defined strategy for digital measurement, you may as well be grepping your log files.

I would love your feedback on this post, either directly in comments or via email. Thanks again to the folks at Google for making this awesome research freely available and to John Lovett for shedding light on this incredibly important aspect of our sector. Remember: we are analysts—our jobs are to ask hard questions and then ask even harder ones!

Presentation, Reporting

Calculating Trend Indicators

Posted on October 5, 2009October 25, 2019 by Eric Peterson

Put this down as one of my more tactical posts, brought on by a fit of lingering annoyance with the use (and by “use” I mean “grotesque misuse”) of trend indicators on reports and dashboards. The trouble is that trends are a trickier business than they seem at first blush, and, at the same time, there are a number of quick and easy ways to calculate them…that are all problematic.

With the well-warranted increasing use of sparklines, which are inherently trend-y representations of data, I like to be able to put a meaningful trend indicator that complements the sparkline. Throughout this post, I will illustrate trendlines, but I’m really focussed on trend indicators, which are a symbol that indicates whether the trend in the data is upward, downward, or flat. Although there are a few minor tweaks I’d love to make once Excel 2010 is released and allows the customization of icon sets, I’m reasonably happy with their 5-arrow set of trend indicators:

They’re clean and clear, and they work in both color and in black and white. And, with conditional formatting, they can be automatically updated as new data gets added to a dashboard or report. While I won’t show these indicators again in this post, the trendlines I do show are the behind-the-scenes constructs that would manifest themselves as the appropriate indicator next to a sparkline or numerically reported measure.

I’ll use a simple 12-period data set throughout this post to illustrate some thoughts (not as a sparkline, but the principles all still apply):

Sample Data

Trends are slippery beasts for several reasons:

Noise, noise, noise — all data is noisy, which means it’s easy to over-read into the data and spot a trend that is not really there
The aircraft carrier vs. the speedboat conundrum — the more data points you use, the more stable your trend, but the longer it takes to collect enough data to identify a trend, or, worse, to determine if you’ve truly impacted the trend going forward

Let’s start this exploration by walking through some of the common ways that “trend” judgments get made and point out why they’re troubling. I will then show an alternative that, while only marginally more complex to implement, works better when it comes to specifying trend-age.

Trending Approaches of which I’m Leery

Trending Based on the Change Over the Previous Period

The most common way I see trends reported is on a “change since the previous period” basis.

Prior Period

In this example, the trend would be an “up” because the data went up from the prior period to the current period. The problem with this is that, if you look at the longer pattern of data, you see that the data is pretty noisy, and it’s entirely possible that this “trend” is entirely a case of noise masking the true signal.

Trending Over an Extended Period

Another way to trend your data, which Excel makes very simple, is to add a trendline using Excel’s built-in trending capabilities (converting this trendline to an indicator would require some use of a couple of Excel functions that I’ll go into a bit in my recommended approach later in the post).

Trendline Example

With this method, the trend would be indicated as “slightly up.” While this may be a valid representation of the overall trend…it seldom seems quite right to use it. The trend gets impacted heavily by any sort of big spikes (or dips) in the data. These keep the same upward or downward trend for a very long period of time. I had a blog post during March Madness one year that wound up driving a big spike in traffic to my site. While it was legitimate for that spike to show an upward trend when I looked at my traffic that week or month, that spike has now wreaked havoc on the macro trend indicator that Google Analytics has shown ever since — for several months that spike kept my overall trend up, and, then, once that spike passed the fulcrum of the tool’s trend calculation, it caused the reporting of a downward trend for severals subsequent months. Through the whole period, I had to mentally discount what the trend indicator showed.

Year-Over-Year Trending

Because seasonality wreaks havoc with trendlines, it’s not uncommon to see trend indicators based on year-over-year results — if the current reporting period is a higher number than the same period a year ago, then the trend is up. For trending purposes, this combines the worst of the two prior examples — it takes a very small number of data points (subjecting the assessment to noise) and it uses ancient history data in the equation.

This isn’t to say that comparisons to the same period in the prior year (or even the same period in the prior quarter, since many companies see an intra-quarter pattern) are bad. But, the question those comparisons answer differs from a trend: a trend should be an indication of “where we are heading of late such that, if we continue on the current course, we can estimate whether we will be doing better or worse next week/next month,” while a year-over-year comparison is more a measure of “did we move positively from where we were last year at this time?”

Trending Approaches I Feel Better About

I’ve spent an embarrassing amount of time thinking about trending over the past four or five years, but I’ve finally settled on an approach that meets all of these criteria:

It balances the number of data points available for the trend with the sluggishness/timeliness of the results
It’s reasonably intuitive to explain
It passes the “sniff test” — while a trend indicator may initially be a little surprising, on closer inspection, the user will realize it’s legit

The last bullet point is really a combination/result of the first two.

My Failed Exploration: Single Point Moving Range (mR)

Because of criteria above, I’ve discarded what I thought was my most promising approach — using the single point moving range (mR). A light bulb went off last spring when I took an intermediate stats class, and, although the professor glossed over the moving range formulas, I thought it was going to be the answer that would allow me to solve my trendline quandary — it would look at the “change over previous period” and determine if that change was sufficiently large to warrant reporting a measurable trend. After noodling with it quite a bit… I don’t think that it works for the purposes of trend indicators. For chuckles, a moving range chart for the example in this post looks like the following:

Moving Range

If you want to read more about moving ranges, the best explanation I found was on the Quality Magazine web site. I’ll just stop there, though. We’ve already lost on the “reasonably intuitive” front, and I haven’t even calculated the control limits yet!

And Another Failed Exploration: the Moving Average

There’s also the “moving average” approach, which smooths things out quite a bit:

Moving Average

I always feel like the moving average is some sort of narcotic applied to the data — it makes things fuzzy by having a single data point factored into multiple points represented on the chart. But, I’ll grudgingly admit that it does have its merits in some cases.

My Approach to Trending (At Last!!!)

There are two key elements to my trending approach, and neither is particularly earth-shattering:

Break the data into smaller components than the reporting cycle
Trend only over recent data, rather than over the entire reported timeframe

Going back to the original example here, let’s say that I update a dashboard once a month, and that the dashboard primarily looks at data for the prior 3 months. In that case, the 12 data points each represent (roughly) one week. IF I simply reported the data on a monthly basis, then the chart would look like this:

Trending Example

That shows a clear upward trend, regardless of whether I look at the last month or the last two months of data. It would be hard not to put an upward trend indicator on this plot. But, we’re relying on all of three data points, and we’re going back three full reporting periods to draw that conclusion. Both of these are a bit concerning. Invariably, we’d want to go back farther in time to get more data points to see if this trend was real…and then we’re falling into the aircraft carrier dilemma.

Instead, though, I can keep the granularity of the reporting at a week, but only trend over the last four periods:

Trendline Proposed Approach

I don’t actually plot the trendline shown in the chart above. Rather, I calculate the formula for the line using the SLOPE and INTERCEPT Excel functions. I then calculate the value of the 4-weeks-ago endpoint of the line and the most-recent-week endpoint of the line and look at the percentage change from one to the other. I actually set some named cells in my workbook to specify how many periods I report over (so I can vary from 4 to 6 or something else universally) as well as what the different thresholds are for a strong up, weak up, no change, weak down, or strong down trend.

In the example in this post, the change is a 16% drop, which usually would garner a “strong down” trend — very different from all the upward trends in the early examples! And, even somewhat counter-intuitive, as the most recent change was actually an “up.” If the entire range has been trending upward, as shown by the 3-point plot as well as by a close inspection of the raw basic data (think of it as a sparkline), then you already have that information available as the longer term trend, but, of late, the trend seems to be somewhat downward.

A Note of Caution

This post has gone through what works for me as a general rule. As I read back over it, I realize I’m setting myself up for a case of, “Yeah, you CAN make the data say whatever you want.”

I’m less concerned about prescribing a universally-effective approach to trend calculation as I am about putting out a cautionary tone on the various “obvious” ways to calculate a trend. The sniff test is important — does the trend work for your specific situation when you actually apply it? Or, have you adopted a simplistic, formulaic approach that can actually provide a very clear misrepresentation of the data?

And…a Nod to Efficiency and Automation

The prospect of introducing SLOPE and INTERCEPT functions may seem a little intimidating from a maintenance and updating perspective, but it really doesn’t need to be. By using built-in Excel functionality, these can be set up once and then dynamically updated as new data comes in. I like to build spreadsheets with a data selector so that the dashboard is a poor man’s BI tool that allows exploring how the data has changed over time. The key is to use some of Excel’s most powerful, yet under-adopted, features:

Conditional formatting — especially in Excel 2007 where conditional formatting can make use of customized icon sets
Named cells and named ranges — these are handy for establishing constants used throughout the workbook (thresholds, for instance) that you may want to adjust
Data validation — using a cell as your “date range selector” that references a named range of the column that lists the dates for which you record the data
VLOOKUP — because you used data validation, you can then use VLOOKUP to find the current data based on what is selected by the user
Dynamic charts — these actually aren’t a “feature” of Excel so much as the clever combination of several different features; Jon Peltier has an excellent write-up of how to do this

If set up properly, a little investment up front can make for an easily updated report delivery tool…with meaningful trend indicators!

Adobe Analytics, Analytics Strategy, Conferences/Community, General

Analytics Demystified European Tour

Posted on October 3, 2009April 12, 2017 by Eric Peterson

Those of you who live in Europe are likely already aware that my good friend Aurelie Pols has joined me as a partner in Analytics Demystified. Over the next two weeks she and I will be making a series of presentations and announcements at events across Northern Europe. We will be at:

The Online Performance Management seminars, hosted by Creuna, in Copenhagen on Thursday, October 8th and in Oslo, Norway on Friday, October 9th. More information about our hosts and registration is available from Creuna.
While we’re in Copenhagen we will be having a Web Analytics Wednesday on Wednesday, October 7th. I will be giving a short presentation on testing and if you’re in Copenhagen please join us at this FREE EVENT sponsored by IIH Nordic and Webtrekk
Over the weekend Aurelie and I will be hanging out in Stockholm, Sweden. If you’re in Stockholm and want to meet-up please either shoot me an email or Twitter me and we’ll make plans!
On Monday, October 12th and Tuesday, October 13th Aurelie and I will be joining the excellent Emetrics crew at Emetrics Stockholm. I will be giving the keynote on Tuesday morning and Aurelie and I will both be participating on a series of panels and shorter presentations. Those of you keeping score will note that I have attended EVERY SINGLE Emetrics ever held in the United States but this is my FIRST EVER event in Europe. Yahoo!
On Wednesday, October 14th, I will be hanging out in Amsterdam with the Nedstat crew but have a fair amount of downtime during the day. I’m staying near Vondelpark and if you’d like to meet and get a cup of coffee (seriously, I mean coffee, I’m too old for the other stuff) Twitter me and we’ll make plans!

Since I usually do three European cities in three or four days this trip is a lazy walkabout for me (four cities, seven days) but Aurelie and I have planning to do and, of course, we’ll spend a little time enjoying the local culture.

If you live in any of these cities, or if you plan to come to Emetrics, please join us and come say hello!

Adobe Analytics

Extracting Unique Visitor IDs

Posted on September 22, 2009February 6, 2020 by Eric Peterson

In this post, I am going to delve into an advanced topic that very few of my past Omniture customers had dealt with – Extracting Unique Visitor ID’s for re-marketing purposes. Unless you have done a few Genesis integrations, this is most likely functionality that is new so I will do my best to keep it simple and explain why it is useful.

Why Extract Unique Visitor ID’s?
The easiest way to explain this topic is through an example. Let’s imagine that your business sends lots of marketing e-mails to customers and prospects. Each of these e-mail recipients, has a unique ID in your e-mail system (we’ll use Responsys for this example). If you are a good web analyst, you should have set-up your e-mails so that when an e-mail recipient receives an e-mail and clicks on one of its links, they arrive at your website with both a campaign ID and a unique e-mail ID. For example, your e-mail link may resolve to:

http://www.test.com?cid=springmailblast&mid=bd69c458909

Setting these in Conversion Variables (eVars) will allow you to see how each e-mail performed (through the campaign ID) and how often each e-mail recipient visited the site (through the e-mail ID). If you aren’t doing this already, I suggest you start there.

Now let’s say you are capturing these ID’s. What most intelligent marketers want to do is to segment their website behavior and then see if they can re-market to those who meet certain criteria. For example, let’s say that you want to send a re-marketing e-mail to all e-mail recipients who came to your site from the e-mail and then filled out a specific offer form. To do this, you would need to find a way to identify the e-mail ID’s of those completing the specific offer form you care about. For an advanced SiteCatalyst user, this would be pretty easy since they would know that they could simply use a Conversion Variable Subrelation report to breakdown the Offer ID eVar value by the E-mail User ID eVar, but you would need to pay for Subrelations on one of these eVars. However, there is a pre-built feature of SiteCatalyst that is available to do this using Data Warehouse. In fact, SiteCatalyst has had the ability to extract Unique Visitor ID’s for many versions, but it is rarely used. This feature allows you to tell SiteCatalyst which eVar stores your Unique User IDs (e-mail ID in this case) and will allow you to easily extract those ID’s using Data Warehouse. By using this feature, you can automatically create a Data Warehouse Segment that pulls the Unique Visitor ID’s you are looking for and then simply tell SiteCatalyst what data points you want as you would in a normal Data Warehouse request. As I mentioned, this is a bit more advanced, but pretty cool (even though it is sufficiently well hidden!).

How It Works
So how does it work? The first step is to tell SiteCatalyst which eVar you are going to use to store your Unique Visitor IDs. Please note that this is not the same as replacing Omniture’s Visitor ID in your js file. To learn more about that, see this post. Adding this value in the Admin Console will not affect your unique visitor counts in any way. In this example, we will use e-mail ID’s which I have labeled “Responsys ID” and to do this we go to the Admin Console. In the Admin Console, you select the report suite(s) and under the conversion area, select “Unique Visitor Variable” as shown here:

uniqueid_admin

On the next screen, you simply choose the eVar that stores your Unique Visitor ID (Responsys ID in this example) as shown here:

uniqueid_admin2

Believe it or not, you are done! But what does doing this actually do? Now if you go to reports in this report suite, you will see a very subtle difference. Per the example above, we want to identify all of the E-mail ID’s of people who looked at a particular website offer. To do this, we will go to the eVar report that stores all of our website Offer ID’s which might look like this:

uniqueid_offer1

Normally, if you click on an eVar value in a report it will take you to a what is known as an “Item-Specific Summary” report which details how often and what percentage that eVar value was involved in each website success event. I find that very few people actually use that report and often go to it once, panic and then hit the back button (I do encourage you to explore that report, but in the interest of staying on topic, I will continue)! However, once you have enabled your Unique Visitor ID variable in the Admin Console, clicking this row will not take you to the Item Specific Summary report, but rather, will present you with a magical new option shown here:

uniqueid_offer2

If you then click on the new row that you see “Extract visitor IDs for event…” you can select the success event that you want (in this example we are looking for Form Completes). Doing this will pop open a new screen that outlines what you are looking for like this:

uniqueid_extract

From this screen, you choose the “Request” button at the bottom and you will be e-mailed a list of the ID’s that match your criteria.

In addition, you will also be taken to the Data Warehouse request manager screen (assuming you have security access to Data Warehouse) and you will see a new segment (see screen shot below) created that matches the criteria you are looking for (in this case, all Responsys ID’s that had a Form Complete and the form matched the specific Offer ID we selected from the eVar report). While this screen is optional, I believe it is presented in case you want to further refine the segment or add additional data fields to your Data Warehouse report:

uniqueid_extract2

At first, I was skeptical and didn’t believe that my click in an eVar report had actually led to a full blown Data Warehouse segment having been created, so if you have any doubts, you can click the Edit Segment button and see the actual segment definition:

uniqueid_segment

Now all you have to do is to add any data points you want to see related to the report area and use this newly created segment. Obviously, you would want to include the Responsys ID’s in this example (I wish this would be pre-selected in the new Data Warehouse screen, but what can you do?), but you can add any others that you wish.

Additional Information
As you can see, while powerful, this functionality can get pretty involved! If you have implemented Genesis integrations, you will find that much of this functionality comes bundled with Genesis so the Unique ID’s you need are automatically extracted and sent to partners for you through API’s. However, I think it is useful to understand how this User ID extraction works, especially if you plan to do advanced customer segmentation.

Finally, keep in mind that there are many other ways to use this functionality beyond the simple e-mail example here. One of the most powerful uses of this feature applies to sites where users login to the website. In these cases, you can store the user’s ID (or a hashed version of it using DB Vista) and perform some amazing analysis and re-marketing to registered users.

Analytics Strategy, Social Media

Web Analytics Wednesday: A Segmentation Experiment

Posted on September 17, 2009October 25, 2019 by Eric Peterson

Last night was another great Web Analytics Wednesday in Columbus, courtesy of the Web Analytics Wednesday Global Sponsors (Analytics Demystified, SiteSpect, Coremetrics, and IQ Workforce). We had a respectable turnout of ~25 people (not including children) and a great time! And, all the better, I got to blind people with the flash on my new camera. A few of the highlights on the picture front:

Bryan Huber from huber+co. interactive and Jen Wells from TeamBuilder Search

Todd Ehlinger from Nationwide, Mike Amer from DSW, and Elaine F.

The Erics — Goldsmith from AOL and Diaz from Diaz & Kotsev Business Consulting (not shown: the THIRD Eric — Eric Moretti from Quest Software)

The picture that didn’t come out well was the one of Laura Thieme of BizResearch with her daughter, Melina — hanging out on her mom’s shoulder…and ‘nary a peep the whole evening (why couldn’t I have had one of those kids?!)! And (cliche warning) cute as a button! As it turned out, Melina wasn’t the only kid who made an appearance — Dave Culbertson’s sons were in attendance on the periphery for the first part of the evening as well.

Rather than a formal presentation, we did an interactive, get-to-know-each-other, have-a-chuckle activity — conceived of and coordinated by Dave Culbertson from Lightbulb Interactive. Unlike my attempts to photograph Melina and Laura — where I only took one shot and then figured the flash was just cruel — I kept clicking the shutter at Dave until he struck a sufficiently expressive pose:

What Dave walked us through was a segmentation exercise: he had a list of questions, each with four possible answers, and we had to segment / re-segment ourselves after each question by going to the area of the room designated for how we would answer that question. An incomplete list of the questions:

Where did you go for your undergraduate degree? a) Ohio State, b) not Ohio State, but another school in Ohio, c) not in Ohio, but in the U.S., or d) outside the U.S.
Which of the following most describes your opinion of social media? a) revolutionary, b) evolutionary, c) nothing new, or d) what’s social media?
If you were going to read only one book this month, what kind of book would it be? a) non-fiction business, b) non-fiction non-business, c) fiction non-science fiction, d) science fiction (or something like that)
If you took only one vacation this year, where would you most like to go? a) the beach, b) the mountains, c) a large city, d) Disneyland
What kind of car do you drive? a) American, b) European, c) Japanese, d) Korean

After we’d segment ourselves, Dave would ask a few follow-up questions of the group. It really did turn out to be a lot of fun (and, if you’re reading this post and recommended a book on that question, please leave a comment with the book you recommended! There sounded like some excellent reads there, and I wasn’t taking notes!)

For my part, I enjoyed getting folks’ take on the Omniture acquisition by Adobe. And, Bryan Huber mentioned what sounds like a pretty slick tool for <social media buzzword>online listening</social media buzzword> that factors in the influence of the person who commented about your brand as well as what they said — another part of the evening where I wasn’t taking notes (but, come on, the pictures ARE fabulous, right?).

So, that’s the hasty recap of the evening. By the time this post publishes, I’ll be on my descent into Boston for a lonnnnng weekend with Mrs. Gilligan:

(And, for you, Eric G., none of the photos used in this post were subjected to post-processing other than cropping. There’s no way I’m going to be able to stick with that, though!)

Analytics Strategy

More color on Adobe + Omniture

Posted on September 16, 2009 by Eric Peterson

Wow, everyone seems to have an opinion about this acquisition. Some people think Microsoft will ride in at the 11th hour and out-bid Adobe because Microsoft and Adobe compete, and because Google has Google Analytics. On this point I am inclined to agree with Joe Davis, CEO of Omniture competitor Coremetrics, who comments that Omniture has been shopping the company around for some time and it is unlikely that Redmond hasn’t already had the opportunity to play (given the significant investment Microsoft has in Omniture.)

Other folks appear to be worried that Adobe will be integrating Omniture into Flash and this raises privacy concerns. While certainly folks have concerns about tracking and the possibility of embedding tracking into Flash Local Shared Objects (LSO) I just have to believe that management at Adobe is smart enough not to risk Flash’s dominance by subjecting the technology to the scrutiny, navel-gazing, and paranoia of the “privacy police.”

Their customers, at least the ones I am talking to, are more or less 2 to 1 against the acquisition at this point citing a variety of concerns (transition, failure to execute on stated product plans, talent flight, Adobe is not adept at services, etc.) Far be it from me to tell anyone’s customers they are wrong when expressing concerns, especially since this is an out-of-sector acquisition and Omniture is now more or less a medium-sized cog in a very big machine. Arguments for include loving Adobe (I love Adobe!), being relieved that Adobe is a big, grown-up company, and hopes that Adobe will focus on fundamentals like customer support, product execution, and global expansion.

Another customer complaint is that Omniture is now losing the (thin, pasty) veneer of third-party objectivity and that some companies may not actually want Adobe to have access to their site’s data. I think this may be the same boondoggle that Omniture (and others) have used to explain why “the Enterprise wouldn’t use Google Analytics” — except there is more and more (and more) evidence that the Enterprise does use Google Analytics — but it will be interesting to see how the “free-standing” analytics vendors work to make Omniture eat their own words now that they too are part of something larger.

The comment that has me most concerned is one best detailed by Carter Malloy from Stephens, Inc. Research Analyst who I have known for years and who I know to be pretty level headed regarding the sector. Carter sent me this, which I am simply repeating with his permission:

“I don’t understand the strategic rationale on adobe’s part. Different end market buyers. Very different products. No real cost savings or integration between the two products. OMTR is very capital intensive vs. adobe not much at all. Seems like Adobe is buying growth with hopes for cross sells. I would be surprised to find out that OMTR did not shop the business around before accepting the bid from Adobe – we should find out soon in public filings required by the SEC. Omniture will still have to report 3Q09 earnings in October, but I think the deal will get closed before Q4 in Jan/Feb. I also think Adobe will show Omniture’s revenue performance on an informal basis going forward. It will be <10% of Adobe’s total revs, but I still think they will give analysts at least some idea of what growth looks like.”

This was in response to my comment detailing a thesis that I have heard from several of Carter’s peers: that Omniture was about to blow Q3 earnings and that the result would be a dramatic dip in OMTR share price as investors head for the exit. The rationale is, apparently, that the company has over-promised and under-delivered for too long, both to investors and customers, and the economy has been the “last straw” for many who have opted to look elsewhere for web analytics technology. This, combined with slower-than-hoped adoption of non-core solutions (data warehouse, Test & Target, Search Center, Survey, etc.) resulted in a “company who’s greatest days are behind them” (direct quote, and I begged to attribute but was told “no” due to company policy.)

Don’t get me wrong: This is not my thesis, at least not yet.

While I have seen evidence of larger Omniture customers switching, increasingly to Unica, I have not seen enough evidence of the kind of massive shift away from SiteCatalyst that would warrant a sudden exit. The good news is that Carter’s thesis can easily be tested: Either Omniture will make expectations for Q3 or they won’t. I’m sure this will make for an interesting Q3 call, at least for those investors who are taking a bath on the acquisition price.

My concern is this: If the investment banker thesis is correct, if Omniture was about to report a second quarter of, um, disappointing results, then what does that mean for the larger industry? Is Adobe really evidence that the larger market is taking an interest in digital analytics? Or was the company thrashing about looking for something new to cover for recent declines and this really isn’t about Omniture or web analytics at all?

Again, I don’t know, at least not yet, and I don’t think any of us do. But given the very mixed reviews about the acquisition I think we as an industry should take a step back and consider the larger ramifications. Personally I don’t think web analytics is going ANYWHERE — hell, I’m recruiting at Analytics Demystified — but we can all admit we collectively haven’t done the best job explaining what we do and what the data we live and die by means.

This interesting acquisition will certainly get more interesting as the days pass. Congrats again to all involved.

General

The Acquisition…

Posted on September 15, 2009February 6, 2020 by Eric Peterson

So by now, you have heard the news about the Omniture acquisition by Adobe. Some out there have pinged me for my thoughts on the matter. Since my blog is reserved for education vs. opinion, I am inclined not to comment much on the matter, but given the magnitude of the transaction, I thought I might provide a few random thoughts…

10 Things I Think About the Acquisition

I think that Omniture has some great products and even better people. I wish them the best and hope that this acquisition doesn’t impact them in a negative way.
I think that the two companies are a strange match. I understand the potential synergies and know that both companies will do their best to portray the acquisition as having a synergistic effect, but I am a bit skeptical. When you spend years preaching about optimizing websites and conversion, I don’t see how that jives with a company that makes design related products. Sure you can track Flash components and Flash microsites better, but you could do that without the need to acquire the company that does the tracking.
I think acquisitions are hard and fail more often than they succeed. Integrating two companies is simply hard work. Many years ago, I was a Lotus Notes expert. Lotus had a thriving e-mail and collaboration tool. People like me ran consultancies around their products. Then IBM bought them and the product died. If you are still using Lotus Notes today, you are one of the few (and maybe proud?). Lotus Notes became an after-thought to IBM as it was a small part of their overall business. I fear that the same thing could happen to Omniture at Adobe.
I think that Omniture acquired too many companies too fast and this may have led to a loss of focus. The Omniture leadership team often spoke about the goal of becoming a company that generated a billion dollars in revenue/year. I think that all of the companies that Omniture bought and the difficulties in integrating them together may have made it more difficult for the company to achieve its goal. I think they had the right vision of creating a cohesive online marketing suite (minus the sorely needed e-mail provider acquisition), but I think a more methodical approach and more up-front integration plans could have made a world of difference.
I think a good question to ask is why Omniture chose to sell now? While they are getting a decent premium, I am sure they could have stayed independent for a while and continued to grow the company. Did the management team feel they had taken it as far as they could? Did they find the prospects of future growth too daunting?
I think that Google Analytics played a silent, but big part in this transaction (the elephant in the room!). I also think that the long term winners of this deal could be Unica and/or Eloqua.
I think that Adobe should focus on three of Omniture products: SiteCatalyst, Test&Target (formerly Offermatica) and Insight (formerly Visual Sciences). Without trying to offend anyone, I think these three products are the most valuable and unique to Omniture. If I were in charge, I would focus all Omniture development resources on those products…
I think that five years from now, there is a chance that we may all be viewing websites and display ads that are much more Flash-intensive and interactive and that there will be people running those sites/ads using Omniture data and targeting to get more and more of our money! Those people will look back on this date as the day that things changed for the better. If Google Analytics can drive more advertising revenue for AdWords, maybe Omniture Analytics can drive more product revenue for Adobe…
I think that Adobe would be wise to keep Omniture as a standalone brand since it is very well known in the web analytics space (not to mention that I would have to change my Twitter Name!). I don’t care if they want to cross-sell products, but the last thing Omniture customers need right now is rebranding, bundling, new contract/payment terms, etc… (unless they want to go down the free model which I would be supportive of!). Trim the fat, re-focus the company on a few core products, retain the good folks and I think they will see a profitable subsidiary.
I think that Adobe could do the following to help Omniture customers and be seen as heroes:
- Use their size and $$$ to find a way to make SiteCatalyst servers more robust, reliable and speedy
- Find a way to simplify Omniture products so they are on par with newer analytics tools like GA (there is a cool product named Flash with which they could build a state-of-the-art SiteCatalyst interface!)
- Deliver on the product-to-product integrations for which Omniture customers have been anxiously waiting
- Find a way to provide more Omniture resources to help customers through Support and/or Account Management (and give Ben a raise!)

I wish both companies the best and hope to continue to be an advocate and champion of the Omniture products…

Analytics Strategy

Thoughts on Adobe + Omniture

Posted on September 15, 2009April 12, 2017 by Eric Peterson

Wow, I have to admit that I was surprised mid-day today at a new client meeting in Chicago when, at the same moment, my phone, my SMS, and my email all went off at the same time. When we got to a break I quickly glanced down and the SMS message said “Adobe buys OMTR for $1.8B!!!!!!”

Wow.

I didn’t get to talk to the press (John got the honors, congrats) and am just not getting a chance to cogitate a little on what Adobe’s entrance to the web analytics market means after non-stop phone calls for the past five hours. A lot of interesting comments have already been published so I will try and reference the stuff I think is insightful in an effort to avoid repetition.

In general, the more I think about the deal the more it makes sense, at least for Omniture. Given increasing pressure from lower-cost (and free) solutions, the economy, and a customer base that is more and more prone to complain about service issues and the high cost of doing business with the company, exiting now makes good sense. Why fight the sea change in the analytics market when you can saddle someone else with the responsibility?
Like others, I don’t really see the synergy in the deal, but I admit that I love Adobe and so I’m willing to be surprised. I think of Adobe as a software company for creative types; Omniture sells software-as-a-service to analytical types; these are different business models and very different customers. The idea that somehow this acquisition bolsters Adobe’s position in content management or as a global delivery platform just doesn’t resonate with me.
Similarly, I don’t see this acquisition as creating anything new regarding measurement being embedded into rich media applications. Thanks, perhaps ironically, to Macromedia (owned by Adobe) we have been embedding tracking codes into Flash, Flex, Silverlight, AJAX, etc. for years … and while the integration is botched as often as not, I don’t see how adding a “Click here to Omniture-ize” button into Dreamweaver and Adobe’s RIA development suite will solve that problem.
I do agree with Alex Yoder’s general thesis that this acquisition increases the overall visibility of the sector and that this is a good thing. I also agree that this acquisition is likely not the last — both WebTrends and Coremetrics are owned by investors and you know how those guys are. His citation of Microsoft and Oracle is interesting given both companies historical interest in the sector (although neither has had the chutzpa to actually pull the trigger — at least in a substantial way.)
I also agree with Gary who is somewhat skeptical about acquisitions, especially out-of-sector ones like this (anyone remember NetIQ? How about you Deepmetrics customers?) and since the Instadia, HBX, and Visual Sciences acquisitions that Omniture made didn’t really generate the benefits promised. However, where Gary favored IBM (who I didn’t realize wanted back into the sector after selling SurfAid to Coremetrics) I liked the idea of WPP increasing their $25M investment by, well, I guess about $1775M or so. Given my position on how companies will deploy web analytics in the future, WPP adding a premium measurement brand to their analytics tools and giving them the ability to pass world-class analytics along to their best customers made sense to me. Oh well.

Regarding Omniture customers … I am getting feedback from across the spectrum. Some customers are encouraged by the news, largely because they believe that Adobe will bring a new level of rigor to product development, integration, and customer support. Others (including those customers still on HBX) are somewhat discouraged by the news, given that they’ve been hearing a lot of promises lately and they’re not sure what a new owner will mean. Still others have expressed that they really liked what the company had been doing this past year and so are bummed that things might slow down while the deal and integration are completed.

Prospects are a different question. Since I am working with a number of companies currently evaluating Omniture products … the best guidance I can give is “wait and see.” Again, I think Adobe is an awesome company and every interaction I have ever had with them has been positive. Hopefully this acquisition will be mostly painless and largely transparent to outsiders. We’ll know soon enough if Omniture’s recent aggressive pricing and willingness to cut deals to close business differs from Adobe’s business practices. And while competitors will almost certainly claim “Omniture is out of the game,” I am personally encouraging my clients to think carefully about what Omniture and Adobe have been able to do independently before writing the combined company off.

At the end of the day I’m really happy for the bright folks I know who have been plugging away at Omniture all these years in a variety of their companies. The teams at Omniture, HBX and Visual Sciences, Offermatica, and hell even Matt Belkin (remember that guy!) who hopefully get to participate in the largess that Omniture has created should all get credit for the thousands of hours they spent on the road, fighting for the big green machine, never willing to concede until they’d finally lost (and sometimes even after they’d been asked to go home!)

Congratulations to Josh, Chris, Brett, John, Kristi, and the entire senior management team at Omniture! Also, best of luck to the management team at Adobe with your new acquisition; your new customers are among the best in the business and will look to you to make a good thing even better.

Analysis, Analytics Strategy, Reporting, Social Media

The Most Meaningful Insights Will Not Come from Web Analytics Alone

Posted on September 14, 2009October 25, 2019 by Eric Peterson

Judah Phillips wrote a post last week laying out why the answer to the question, “Is web analytics hard or easy?” is a resounding “it depends.” It depends, he wrote, on what tools are being used, on how the site being analyzed is built, on the company’s requirements/expectations for analytics, on the skillset of the team doing the analytics, and, finally, on the robustness of the data management processes in place.

One of the comments on the blog came from John Grono of GAP Research, who, while agreeing with the post, pointed out:

You refer to this as “web analytics”. I also know that this is what the common parlance is, but truth be known it is actually “website analytics”. “web” is a truncation of “world wide web” which is the aggregation of billions of websites. These tools do not analyse the “web”, but merely individual nominated “websites” that collectively make up the “web”. I know this is semantics … but we as an industry should get it right.

It’s a valid point. Traditionally, “web analytics” has referred to the analysis of activity that occurs on a company’s web site, rather than on the web as a whole. Increasingly, though, companies are realizing that this is an unduly narrow view:

Search engine marketers (SEO and SEM) have, for years, used various keyword research tools to try to determine what words their target customers are using explicitly off-site in a search engine (although the goal of this research has been to use that information to bring these potential customers onto the company’s site)
Integration with a company’s CRM and/or marketing automation system — to combine information about a customer’s on-site activity with information about their offline interactions with the company — has been kicked around as a must-do for several years; the major web analytics vendors have made substantial headway in this area over the past few years
Of late, analysts and vendors have started looking into the impact of social media and how actions that customers and prospects take online, but not on the company’s web site, play a role in the buying process and generate analyzable data in the process

The “traditional” web analytics vendors (Omniture, Webtrends, and the like) were, I think, a little late realizing that social media monitoring and measurement was going to turn into a big deal. To their credit, they were just getting to the point where their platforms were opening up enough that CRM and data warehouse integration was practical. I don’t have inside information, but my speculation is that they viewed social media monitoring more as an extension of traditional marketing and media research companies that as an adjacency to their core business that they should consider exploring themselves. In some sense, they were right, as Nielsen, J.D. Power and Associates (through acquisition), Dow Jones, and TNS Media Group all rolled out social media monitoring platforms or services fairly early on. But, the door was also opened for a number of upstarts: Biz360, Radian6, Alterian/Techrigy/SM2, Crimson Hexagon, and others whom I’m sure I’ve left off this quick list. The traditional web analytics vendors have since come to the party through partnerships — leveraging the same integration APIs and capabilities that they developed to integrate with their customers’ internal systems to integrate with these so-called listening platforms.

Somewhat fortuitously, a minor hashtag snafu hit Twitter in late July when #wa, which had settled in as the hashtag of choice for web analytics tweets was overrun by a spate of tweets about Washington state. Eric Peterson started a thread to kick around alternatives, and the community settled on #measure, which Eric documented on his blog. I like the change for two reasons (notwithstanding those five precious characters that were lost in the process):

As Eric pointed out, measurement is the foundation of analysis — I agree!
“Web analytics,” which really means “website analytics,” is too narrow for what analysts need to be doing

I had a brief chat with a co-worker on the subject last week, and he told me that he has increasingly been thinking of his work as “digital analytics” rather than “web analytics,” which I liked as well.

It occurred to me that we’re really now facing two fundamental dimensions when it comes to where our customers (and potential customers) are interacting with our brand:

Online or offline — our website, our competitors’ websites, Facebook, blogs, and Twitter are all examples of where relevant digital (online) activities occur, while phone calls, tradeshows, user conferences, and peer discussions are all examples of analog (offline) activities
On-site or off-site — this is a bit of a misnomer, but I haven’t figured out the right words yet. But, it really means that customers can interact with the company directly, or, they can have interactions with the company’s brand through non-company channels

Pictorially, it looks something like this:

I’ve filled in the boxes with broad descriptions of what sort of tools/systems actually collect the data from interactions that happen in each space. My claim is that any analyst who is expecting to deliver meaningful insight for his company needs to understand all four of these quadrants and know how to detect relevant signals that are occuring in them.

What do you think?

Adobe Analytics, General

Segment Bounce Rates

Posted on September 9, 2009February 6, 2020 by Eric Peterson

In my last post, I discussed a topic which I called Segment Pathing, which allows you to see how Pathing on your site differs by Visitor Type or Campaign Tracking Code. In this post I will build upon this concept with one of the most popular topics in the Web Analytics field: Bounce Rates. While I am not as enthusiastic about Bounce Rates as many others in the field, I do understand their importance and why people like them them. However, one of my gripes with the Bounce Rate metric (which I have always defined as Single Access/Entries) is that there is not an easy way in SiteCatalyst to see Bounce Rates for different types of visitors or Campaigns. Unless they have Omniture Discover or are experts at ASI Segments, most of the Omniture clients I worked with were primarily looking at Bounce Rates for the entire population. While this is OK, I think we can do better than that. In this post I will show you how I create Segment Bounce Rates. However, to get the most out of this post, I strongly encourage you to read my prior post on Bounce Rates and my previous post on Segment Pathing before reading this post.

Segment Bounce Rates
As I just described, my goal when looking at Bounce Rates is to be able to tell my peers how visitors are bouncing off key pages based upon both the page and the segment. In my previous post, I highlighted two segments that I commonly use: 1) Visitor Type (i.e. Customer vs. Non-Customer) and 2) Campaign Tracking Code (i.e. visitors from Google keyword A vs. Yahoo keyword B). If I can dissect how each segment bounces off pages, I can determine if I need to create different versions of pages for each Visitor Type or Campaign Code or I can use this information to build future A/B Tests using a tool like Test&Target. As I mentioned in my last post, this is a moot point if your organization already has Omniture Discover, but as is always the case in my blogs, my goal is to show you how to do things if you only have access to SiteCatalyst.

Implementing Segment Bounce Rates
The good news is that if you have already followed my instructions from my previous post on Segment Pathing, you are 95% of the way to being done with implementing Segment Bounce Rates! As a quick recap, in my last post I described a process in which you concatenate the Page Name with another Traffic Variable (sProp) that contains a segmentation that you care about (i.e. Visitor Type). Once you have these values concatenated on every page, you enable Pathing so you can see paths or pages by segment. However, when you enable Pathing on this new sProp, you immediately gain access to the two metrics that you need to calculate Bounce Rate: Single Access & Entries. Therefore, without even knowing it, by implementing Segment Pathing, you have also implemented Segment Bounce Rates! All you need to do is to create the Bounce Rate Calculated Metric (which hopefully you already have as a Global Calculated Metric) and you are done.

So how do you see the results of your work? All you need to do is to open the new concatenated sProp and add the Bounce Rate metric to the report. In the example shown below, I will use the Campaign Pathing sProp which shows Campaign Tracking Codes concatenated with Page Names. I will add Visits, Single Access, Entries and Bounce Rate to the report:

SegmentBounce_1

As you can see, the Bounce Rate for each Tracking Code/Page Name combination is displayed and you can sort by any metric you wish.

As a best practice, I like to conduct a text search filter to isolate one Page Name so I can see how the Bounce Rates differ for the same page with different Campaign Tracking Codes. In the following example, I filtered on the phrase “:Home Page” and limited my results to see only Home Page Entries and the associated Bounce rates of each Campaign Tracking Code:

SegmentBounce_2

Keep in mind that I am only showing a few simple examples here and that this functionality can be extended to any segment of your choosing. If you want to get really advanced, you could even concatenate multiple items together, such as Visitor Type + Campaign Tracking Code + Page Name. This would allow you to see how different Visitor Types, coming from specific Campaign Tracking Codes, landing on specific Pages, navigate your site or Bounce off pages (i.e. Customer:ggl_1:Home Page). Just don’t go too crazy since there are character limits on sProps and you don’t want to exceed the 500,000 monthly unique limits on sProps.

Final Thoughts
As you can see, you get a “two for the price of one” deal if you do all of the steps in this post and the previous post. If you don’t have access to Omniture Discover and want to see how people navigate through your site or bounce off your site pages by specific segment, I suggest you give this a try and see if it helps you.

Conferences/Community

X Change 2009: Sold Out!

Posted on September 5, 2009April 12, 2017 by Eric Peterson

You may have already noticed this when you went to the registration page if you’re still considering the X Change next week but we officially put a cap on things last Thursday. While I’m disappointed that more people won’t be able to join us, it is incredibly gratifying to know that in the midst of the worst business economy in decades that smart people are still able to get management approval for continuing education, networking, and professional development.

I am certainly excited about the group we have coming next week: some great vendors, some awesome consultants, an incredible keynote event, and many of the best and brightest practitioners in the digital measurement industry. Excellent!

Also, as Gary pointed out in his blog post today, if you really, really, really need to join us in San Francisco and have already gone to bat for the budget, just let me know. We can always squeeze one more in, but we’ll probably make you attend Gary’s Think Tank training session just so he feels a little better (see his blog post for the back-story …)

If you can’t make it and you’re on Twitter please watch for conversation and insights on the #xchange hashtag in Twitter. Like Gary, I’m not foolish enough to promise to blog from the conference (hell, I barely blog as is anymore … too busy with work I guess!) but I will definitely try and push up 140 characters here and there, or slightly less when I co-tag with #xchange and #measure LOL!

If you’re coming to San Francisco, I look forward to seeing you next week!

P.S. piggybacking on Gary’s comments about WebTrends … I’m with Gary and Phil on the whole “9” release. I’m encouraged by the company making a move in the right direction, but I feel that the release was dramatically over-marketed and set a new, all-time high for “hype over substance” in this industry making even the great green machine look conservative. When your own staff are forced to admit the release is not “the new UI” despite marketing’s claim that “[9’s] clean, professional interface lets you creatively explore your data like never before,” well something has gone wrong somewhere.

That said, it’s great to see Alex committing to the product and, at the end of the day, it’s not what Gary, Phil, or I think … it’s what their customers and prospects think that counts. I know a handful will be at the X Change, along with someone from WebTrends marketing, so that interplay will be … ummm … interesting to watch!

Analytics Strategy

The Inertia of the Status Quo

Posted on September 4, 2009October 25, 2019 by Eric Peterson

Some definitions (courtesy of Wiktionary):

status quo — the way things are, as opposed to the way they could be
inertia — The property of a body that resists any change to its uniform motion
cognitive dissonance — a conflict or anxiety resulting from inconsistencies between one’s beliefs and one’s actions or other beliefs

OlofS_boulders The first two of these can be applied to any sort of technology or process change being introduced to an organization — entire careers and companies are built around trying to figure out how to effectively drive change within organizations. In the case of data management, the third defintion — cognitive dissonance — comes into play as well.

As a brilliant and phenomenally handsome man* once said, “Customers are people, and people are messy.” Customer data is inherently incomplete and imperfect. Any process or system that captures and stores customer data stores flawed data as soon as it rolls out for two reasons:

It is not reasonable to add to any process all of the overhead required to rigorously capture and validate all attributes of a customer — it’s a balancing act between the efficiency of the process and the quality of the data captured
Customer data decays, and it decays a lot more quickly than we like to admit; customer data maintenance tends to be an afterthought that gets addressed only after time has degraded the data to the point that it starts causing the company real problems

Once we hit the point where we really need to tackle our customer data management head on, we have two options, of which one option is completely inviable:

Throw out all of our customer data, customer data processes, and customer data systems and start over, but “do it right this time”
Identify the most broken parts of our processes and start fixing them — going after the lowest cost and highest benefit ones first and then working our way down the list until we hit a satisfactory point (which is, typically, never)

Clearly, the first option is not an option. No company would survive if they tossed out their customer base, barred their doors, and conducted no business for a year or two while they rebuilt their process and technology infrastructure.

That leaves us with the second option (technically, “do nothing” is an option as well, but that’s only an option if the pain hasn’t reach the point where it’s not an option!), and, thus, we reach a cognitive dissonance conundrum:

We know our customer data is dirty — customer service reps complain about the number of duplicate records in their systems, sales reps complain of the incomplete pictures they have of their customers (which hinders their ability to prep for and conduct customer visits), marketing complains that they can’t effectively segment and target their database because the customer data is bad, customers complain because the company keeps screwing things up in one way or another…

BUT

…as we start to explore and design replacement processes, we realize that these processes are going to be inherently imperfect, too. We may accept that the new process will be better (even significantly so), but we obsess about the flaws.

We don’t want to repeat the mistakes of the past and roll out something that is not bulletproof — a chink in the data management armor is a chink, no matter how small. So we obsess about the chinks. We propose process changes to accomodate the identified gaps. Even for the gaps that are purely theoretical (“yes, I see, but what if the poles reversed at the exact same point that pigs learned to fly — the process would break!”) We’re trying to do the right thing. We’re aiming for perfection — for a flawless process.

But we’re talking about customer data, and customers are people, and people are messy.

We find ourselves (and/or the people who will ultimately need to adopt the process) paralyzed, caught in an endless cycle of Visio vetting and process rework, perpetually getting halfway to the “perfect” process, but never actually getting there. At some point, due to impatience or frustration, someone stands up and yells, “Enough! Just build what you’ve got!”

And then we realize we’ve designed a process that is so complex and unwieldy that the cost to implement it would wipe out any hope of the company having a profitable year for the ensuing decade.

Of course you’d like a more tangible example:

Let’s say we’re trying to clean up our customer’s mailing addresses (which, thankfully, is now an exercise from my past, but that’s more a digression for a discussion over drinks than for a blog post!). Let’s say that, for any 1,000 customer addresses, we have conclusively demonstrated that at least 50 of them are bad — the postal service is going to struggle to deliver mail sent to them, and the postal service is going to fail more often than not. Now, let’s also say that we’ve demonstrated that, by introducing some automated cleansing processes, we can: 1) identify those 50 addresses, 2) “fix” 30 of them, and 3) flag the remaining 20 as being known problems that need some sort of manual intervention. Let’s say that, rather than 1,000 records, we’re talking about 10 million.

“Hurray!”

“Sounds great!”

“Awesome!”

“Gimme some of that!”

Ah…BUT…

…we have also determined that, as part of those automated cleansing processes, we might actually take 1 of the 950 addresses that were already good…and make it worse.

Logically, the project should still be a go. We’re making 30 addresses better and only might be making a single address worse!

bobster855_unhappyman

Ohhhhh…that single address. That molehill that eats its Wheaties, regularly applies cream provided by a shady character, and injects itself in the buttocks with a substance its cousin purchased over the counter in the Dominican Republic. The molehill grows. It grows quickly. Suspiciously quickly…yet no one seems to notice. It becomes a hill, and then a big hill, and then a mountain! The project manager is left scratching his head and wondering how a theoretical aside in a meeting three weeks ago has now become a virtually insurmountable issue that has put the entire project at risk of ever being implemented!

Cognitive dissonance — simultaneously recognizing that things are bad and must be fixed, but also accepting that the status quo is “right.”

The answer? I’d like to say it’s just a matter of putting the dissonant perspectives side by side and forcing objectors to reconcile them. That should work, right?

Alas!

As it happens, the current debate about healthcare reform in the U.S. prompted James Surowiecki to right a column on Status-Quo Anxiety in The New Yorker a couple of weeks ago. Surowiecki discusses the “endowment effect:”

“…the mere fact that you own something leads you to overvalue it. A simple demonstration of this was an experiment in which some students in a class were given coffee mugs emblazoned with their school’s logo and asked how much they would demand to sell them, while others in the class were asked how much they would pay to buy them. Instead of valuing the mugs similarly, the new owners of the mugs demanded more than twice as much as the buyers were willing to pay.”

Surowiecki goes on to relate this effect to the healthcare debate: “What this suggests about health care is that, if people have insurance, most will value it highly, no matter how flawed the current system.”

The same applies to customer data management all too often — we know we have a flawed system, but it’s the system we have, gosh darn it, and I don’t want your new system if I can find any imperfections in it!

This really has been a farewell post of sorts. Rambling, yes. Academic, yes. Lacking any prescriptive solution. But, hopefully at least a little entertaining, and maybe even with an insight or two that may come in handy to you. Look for a topical shift to measuring digital media going forward.

So long, and thanks for the fish!

* Dramatic license — I said that in this post, and “brilliant and phenomenally handsome” is perhaps a bit of an overstatement.

Photos courtesy of Olof S and bobster855

General

Am I Ever BeHIND on Posting…

Posted on September 2, 2009October 25, 2019 by Eric Peterson

August was a little crazy for me:

I changed jobs — left Nationwide to become Director, Measurement and Analytics at Resource Interactive — which is 1000% the “right” move, but meant for a hectic/stressful month
Back-to-school time, which was more than just getting our kids ready — my wife ran our two sons’ elementary school’s entire supply sale…and my “I’ll show you a few tricks in Excel to help you stay organized” offer morphed into a full-blown custom ERP system built in MS Access; August was the month when all the supplies arrived (think almost 10,000 no. 2 pencils…) and had to be divvied up; I did no divvying, but there were a number of late-breaking report requests; at last count, the database had over 20 tables (it’s almost a fully denormalized database), over 40 queries, 12 forms, and 20+ reports; AND…it’s now been extended to also handle the production of the school’s student directory; gotta love MS Access!
Company, company, company — two visits from friends in Texas, two visits from my parents, a visit from my in-laws, and my mother-in-law moved in for six weeks to convalesce from surgery…all in a 3-week period in August

I’ve got one more good customer data management post in me that needs to get written, at which point I expect to be shifting over to more web analytics-y, social media measurement-y posts going forward.

And…as I played around with Drupal for a couple of projects over the past couple of months, I realized that the theme that I settled on after weeks of experimentation on this blog…is one that was built for WordPress to mimic one of the Drupal default themes! How embarrassing!

Please be patient! My life will settle back down soon (I hope). In the meantime, if you’re going to be in Columbus in the middle of September, consider stopping by this month’s Web Analytics Wednesday on September 16th!

Adobe Analytics

Segment Pathing

Posted on August 31, 2009February 6, 2020 by Eric Peterson

In past blog posts I have discussed SiteCatalyst Pathing Analysis in general and some specific examples (i.e. Success Event Pathing). In this post, I will share a more advanced technique I call Segment Pathing which is often used to extend the capabilities of Pathing Analysis. While this technique can be used in many different ways, I will use Visitor Type Pathing as the primary example and way to explain the concept.

What Is Segment Pathing?
Most of you are probably familiar with the idea of Pathing and that SiteCatalyst Pathing Analysis tracks the order in which a visitor looks at pages, sections or anything else on your site. As such, it is normal to pass a page name or section name value to a Traffic Variable (sProp) so you can then enable Pathing. However, there are often cases where you want to see how different segments of your visitors navigate through your site. For example, what pages do New Yorkers look at first vs. those from Chicago? Are there Pathing differences between younger vs. older visitors?

In order to see how these different segments navigate your site, you have the following options:

Create an ASI Segment for the population you care about and look at Pathing reports there
Utilize Omniture Discover (assuming you have paid for that), create a Segment and view Pathing reports

But what if you don’t have Discover and you don’t want to burn up an ASI segment perpetually for this Pathing Analysis? The answer is to use Segment Pathing which I will demonstrate here.

An Example: Visitor Type Pathing
In this example, let’s assume that your organization has a cookie that stores (to the best of its ability) the current visitors customer status. Often times companies assume that a visitor with no cookie value is a “Non-Customer” and those who have logged in or purchased something are “Customers” (obviously this is subject to cookie deletion). Now let’s assume this this Visitor Type is passed to a SiteCatalyst Traffic Variable on every page. Obviously, the name of each page is passed on each page and should be set to the s.pagename Traffic Variable. Therefore, you have Page Name and Visitor Type, but no way to see pages by Visitor Type. All you have to do is to set a new Traffic Variable (sProp) that concatenates these two values together in a format like this:

[VISITOR TYPE]:[PAGE NAME] or “Customer:Home Page”

If you do this on every page of the site and then have your Account Manager enable Pathing on this new sProp, you now have an intersection between Visitor Type and Page Name on each page and can use any of the many Pathing reports (including Fallout and Pathfinder) for this new variable. SiteCatalyst experts long ago realized how simply concatenating values together into one SiteCatalyst variable could yield powerful results. By using this technique, you can now select the appropriate “Visitor Type” concatenated value in the Next Page Flow report to see what “Customers” do on your Home Page:

Customer_Path

as compared to “Non-Customers” viewing the same page:

NonCustomer_Path

As you can see here, Non-Customers have a much higher exit rate from the Home Page than Customers do, but without the use of this Visitor Type Pathing, it might be difficult to spot this since you are looking at Pathing for all segments lumped together.

Keep in mind, however, that this is just one example of how you can do Segment Pathing. One of my favorite uses of this technique is to concatenate Campaign Names or Campaign Tracking Codes and Page Name so you can see how visitors from different Campaigns navigated through your site. In the more advanced version of this shown below, you can see a Pathing Flow for visitors who arrived at a website from a Tracking Code “ggl_1” and landed on the Video Games page. By concatenating these two values, we can see how visitors arriving from the “ggl_1” Campaign Tracking Code navigated the site as compared to those arriving from a different Campaign Tracking Code. In fact, we can also see how people coming from the same Campaign Tracking Code (i.e. “ggl_1” navigated the site differently when they arrived on a different page (i.e. a page different than the “Video Games” page).

Note that in the example below, the Campaign Tracking Code is not concatenated with the Page Name on every page, but rather just on the first page. In this case, this was done because of the massive number of potential Campaign Tracking Code & Page Name combinations, which could lead to a “uniques” issue in SiteCatalyst. However, the good news is that since Pathing reports only show values that took place after the element before it, by simply selecting the value of “ggl_1:Video Games,” we are guaranteed that all path views after it had to be preceded by the selected value.

CampaignPathing

Final Thoughts
As you can see the implementation of this through the use of variable concatenation is not terribly difficult. However, before you run out and concatenate all of your Traffic Variables together, keep in mind the following:

You do not want to enable Pathing on too many sProps since it will cost you $$$ and could result in report suite latency
While powerful, this technique is more of a “hack” so if you are going to be doing a lot of segmentation, I encourage you to invest in Omniture Discover which is a much easier way to do Segment Pathing

Adobe Analytics

SiteCatalyst Quiz Answers!

Posted on August 24, 2009February 6, 2020 by Eric Peterson

Thanks to all of you who took the time to complete my SiteCatalyst quiz. I hope it was a fun way to put your knowledge to the test.

So for the rest of this post, I will show how people answered the survey and point out what answers I was looking for. When looked at as an entire population, if I include anytime someone got the correct answer, the majority of people got 10 correct answers out of 15 (66.67%). However if I just look at just those responses where the exact right answer was given (no incorrect answers included where you could check off multiple boxes), the average score went down to about 6 out of 15. However, please bear in mind that I am not an educator so if you interpreted a question differently than I did and gave a different answer, it is probably my fault not yours so don’t lose any sleep over it! On the bright side, one (anonymous) individual in Europe got 14/15 correct (I am resisting the urge to find you by IP address and hire you!). Either way, I strongly encourage you to look at your answers and see which ones you missed and read the linked posts below so you can become a SiteCatalyst Ninja!!

Question #1 (Correct Answer=Traffic Variables (sProps))
This first question was intended to be an easy one. Think of it as a way to build engagement and not scare you off. Most of you got this answer correct, but I was surprised to see that 33% of you thought that you could enable Pathing on more than just Traffic Variables (sProps). Keep in mind that one of the main reasons to use sProps is to enable Pathing. If you need a refresher, please check out my past posts on Traffic Variables (sProps) or on Pathing.

quiz1

Question #2 (Correct Answer=True)
For many of these True/False questions, it is hard for me to tell if you got the right answer based upon knowledge or luck, but I am going to give you the benefit of the doubt! In this case 75% of you were correct in saying that it is possible to share a segment with other users in your company. I show how to do this in my past post about the Admin Console. Keep in mind that you can only share a segment within one report suite so if you have multiple report suites you are out of luck. If you really need to share segments across multiple report suites, the only way I know to do this is to create them under a shared Omniture User ID and give that ID to multiple users so they can see the segments owned by that ID.

quiz2

Question #3 (Correct Answer = ZERO)
This question is admittedly a difficult one. To get this one right, you would have had to really been in the trenches with SAINT Classifications. Those who have ever tried to classify a variable that has a value of “0” in the Key column have probably learned this the hard way. While you can classify a value of “1” or “43,” there is no way to classify a Key value of “0” in SiteCatalyst. Therefore, you need to pass in a text value for “0” so you can classify it later on. Therefore, the best answer to this question is the 3rd answer below “ZERO.”

quiz3

Question #4 (Correct Answer=When a Success Event takes place or after a specified Time Period)
You guys knocked it out of the park on this one. The correct answer here is that an eVar can be expired when a Success Event takes place or based upon a time period. This happens to be one of my pet peeves since I really wish you could expire an eVar based upon a Success Event or a time period (whichever comes first). There are many cases where having this ability would have saved me a lot of time. Maybe in a future release (or all of you can help me by requesting this as a feature request!).

quiz4

Question #5 (Correct Answer=True)
Most of you got this one right as well. One of the cool things about classifying Conversion Variables (eVars) is that if you have paid for full subrelations on the eVar it is based off of, you get full subrelations on all of the Classifications. This can save you time and money!

quiz5

Question #6 (Correct Answer= All but Conversion Variables (eVars))
This question was a hard one and another one of my pet peeves. The correct answer is the second one “Conversion Variables (eVars).” The security features in the Groups area of the Admin Console are very good and a much better way to hide reports from select groups of users than the Menu Customizer. However, for some unknown reason, you can hide pretty much everything in SiteCatalyst except Conversion Variables (eVars), which are some of the most critical reports! I am not sure why this one thing was omitted and I have been asking for this for some time. Hopefully it is on the product roadmap.

quiz6

Question #7 (Correct Answer=None of the Above)
This question probably caused some confusion due to the wording, but the correct answer here is “None of the Above” since I was looking for the best way to assign credit across multiple visits. Most of you fell for the trap I set here and chose “Linear Allocation.” Many people I talk to think that Linear Allocation of an eVar works across multiple visits, but it does not. Linear Allocation only works within a visit (for the most part, but the details are a bit confusing!). Therefore, the real best answer for this question was Cross-Visit Participation which I covered a while ago. Cross-Visit Participation is the only real way to assign credit to an eVar across multiple visits. If you are not familiar with Cross-Visit Participation, please review my previous post.

quiz7

Question #8 (Correct Answer=None of the Above)
Importing offline or external data via Data Sources is a more advanced topic, but many of you look like you are familiar with it. The majority of you got this one correct since none of the options provided here will allow you to back out data sources metrics. For this reason you have to be extremely careful when importing Data Sources data since there is really no going back if you make a mistake!

quiz8

Question #9 (Correct Answer=False)
This is one of those questions you used to get from your teacher and absolutely hate them afterwards when they told you the answer, so I will apologize in advance. The key phrase here is “the only difference” so the correct answer here is “False.” While the difference cited here is correct, there is one really big difference between Correlations and Subrelations that you need to know. That difference is that you can correlate two sProps, five sProps or twenty sProps with each other, but with Subrelations it is an all or nothing proposition. It would be great if you could Subrelate just two eVars together, but that is not currently possible like it is for sProp Correlations. This is a key thing that every SiteCatalyst Ninja must know!

quiz9

Question #10 (Correct Answer=Unique Visitor Counts an Pathing)
Most of you got this one correct. The key disadvantages of Roll-ups are that they don’t de-dup uniques and you cannot do Pathing analysis. But hey, they don’t cost a lot! Personally, I tend to not use Roll-ups since I can duplicate a lot of the info they provide using the ExcelClient and I like Pathing and de-duped Unique Visitors so I tend to favor Multi-Suite tagged sites.

quiz10

Question #11 (Correct Answer=Calculated Metrics)
Great job on this one as most of you got this one correct! In my post about Conversion Funnels I explained all of the ways they can be used and highlighted what, in my opinion, is an oversight of the functionality that you cannot add Calculated Metrics to them. I hope this ability will be available at some point in the future, but in the meantime, you should keep this in mind when determining whether you should pass in a metric organically or rely on a calculated metric.

quiz11

Question #12 (Correct Answer=Page Views)
In this question, I allowed you to choose from the following metrics. While most of you got this correct that the Page Views metric is available in Traffic Variable Correlations those of you who also said that you could add Visits, DUV’s and MUV’s were not correct. Please keep in mind that only Page Views are available when using Correlations.

quiz12

Question #13 (Correct Answer=Classifications cannot be used in DataWarehouse Segments)
I had a hard time figuring out how to word this question, but if you really understand SAINT Classifications , you should have been able to get this one right by the process of elimination. As you can see, most people had a hard time with this one, but the correct answer did emerge in the end as the only true statement below is that Classifications cannot be used in DataWarehouse Segments. We can deduce this by understanding that 1) Classifications can be used in correlations/subrelations, 2) Classifications can be used in Omniture Discover and 3) Classifications cannot have pathing enabled in SiteCatalyst (You can, however, apply Pathing to classifications in Omniture Discover).

quiz13

Question #14 (Correct Answer=False)
While I have been using Advanced Segment Insight (ASI) segments for many years, I only recently figured out that you can change the ASI type from recurring to time slice and vice versa if you know what you are doing so the correct answer below is actually False (the whole double-negative thing!). If you have a static ASI that has run for say last month, you can use the “Add Data” link to bring up the ASI segment set-up screen, change the type to Daily Recurring and make the start date the day after your ASI last ran. Just be sure to uncheck the box that asks if you want to remove existing data or you will lose your past ASI data. If the ASI you already have is a Daily Recurring, simply wait until it has finished its daily processing and click the “Cancel” link. Once you have cancelled it, you can click the “Add Data” link, make the type “Time Slice,” select your dates and set it to run. Again you have to be sure to uncheck the remove existing data checkbox. I am not sure if this is “officially” supported, but I have done lots of testing on it and so far it has worked fine…

quiz14

Question #15 (Correct Answer=>!)
This last question is one of those “inside secrets” that only true SiteCatalyst Ninjas know. Unfortunately, only 32% of you got this one right as the correct answer is “>!” which is a way to tell if a value exists or not in a specific variable when using the segment builder. I covered this in my Tips & Tricks are of the Segment Builder post so if you didn’t get this one correct, check out that post which has lots of goodies in it.

quiz15

Well, there you have it. Hopefully this was a fun way for you to see some of the things that SiteCatalyst Ninjas know. If you keep reading my blog posts, I can (almost) guarantee you will learn everything that there is to know…

Analytics Strategy

Type I vs. Type II Errors in Customer Data Management

Posted on August 18, 2009October 25, 2019 by Eric Peterson

[Update: See the comments at the end of the post. I might have my definitions backwards, but I need to do some digging to make sure I get the corrections right. The underlying point of the post — that, in customer data management, you’ve got to recognize that false positives and false negatives are not the same thing and situationally err on the side of one or the other — still holds. But, it’s not good to have the terminology reversed!]

Last week, I ended a post promising a future post on Type I vs. Type II errors when it comes to customer data management. I’ve found myself running into confusion on the distinction, with all customer data errors being treated as the same type, when they are not.

Let’s Start with Definitions

Wikipedia has a nice write-up that goes into much deeper detail, but here are my quick and crude definitions:

Type I Error = α (Alpha) Error = False Positive = you mistakenly judge something to be true that, in fact, is false
Type II Error = β (Beta) Error = False Negative = you mistakenly judge something to be false that, in fact, is true

Type I vs. Type II Errors

It’s easy to get mired in statistics-speak and make a snap judgment that this distinction between types of data errors are solely of interest and use to statisticians. That’s not the case. Depending on the situation, one type can be critical while the other can be barely consequential.

Customer Data Example: Automated Customer Merges

Most companies of any size battle duplicates in their customer data. I have yet to work at or with a company where the sales force and call center staff don’t complain about the number of duplicates that exist in internal systems. In this situation, it’s pretty common to try to automate a deduplication routine of some sort (some would say that this is the biggest benefit of a customer data integration, or CDI, initiative).

Unfortunately, customer data is messy. For every “identical match” in the system, there are generally 5 to 10 “probable matches.” After all, if the data was truly identical, then the likelihood that a duplicate would have gotten created in the first place would have been greatly diminished! That means that the automated matching system has to apply various business rules to determine which matches are true matches and which ones are not — the system makes an educated guess. Many CDI systems apply a scoring system — the higher the score, the more likely the two records are a true match. The lower the score, the less likely. It’s then up to the user to establish the threshold above which the records will automatically be merged (and, possibly, a lower threshold that will trigger a manual review of the records by a human being).

Where that threshold gets set is purely a Type I vs. Type II error decision:

The higher the threshold gets set, the greater the likelihood of a Type II error — a false negative where two records could have been merged because they were, in fact, duplicates, but they were not identified as such
The lower the threshold gets set, the greater the likelihood of a Type I error — a false positive where two records get merged, even though, in reality, they represent two different customers

It would be wonderful if the matching logic was such that a threshold existed that eliminated all errors, but no such threshold will ever exist. The question then becomes: which type of error is “more” acceptable? That will influence where you set your threshold. It’s a situational question! For example:

If your customer data includes extremely sensitive information (think medical records, social security numbers, credit card numbers), then you will want to err on the side of Type II / False Negative errors — deal with the messiness of more duplicates and put the burden on your call center / sales force to trigger merges if and only if they have confirmed the merge needs to happen through a manual inspection and/or an interaction directly with the customer
If, on the other hand, you are only using the post-merged customer data to send marketing promotions to the customers, then the added cost of the direct mail may push you to err on the side of Type I / False Positive errors — some of your customers may miss out on some marketing promotions, but very few of them will receive multiple copies of the same direct mail piece, and your overall postage costs will be lower

This is a very real example — a single company may use the former mentality as its core matching logic, but then maintain a separate data store with more aggressive matching for purposes where Type I errors are not critical.

Type I and Type II in the News

A friend and former colleague shared a recent story in The Washington Post about the Social Security Administration running into trouble when trying to deny social security benefits to certain felons. It sounds like there were some technical gaffs, for sure — when people with minor offenses on their record got caught up in the denial of benefits. Overall, this reads like it was a case of an unacceptable number of Type I errors — people being denied benefits because they were mis-identified as “fleeing felons” (the article reads like some of the stories of people being banned from flying because they had the same name as someone on the terrorist watch list).

Both types of errors will occur. As important as being clear on which type are “better” given the situation (and developing your processes accordingly), ensuring that you have processes in place to fix the errors when they do occur and are identified is just as critical — something that the federal government was apparently missing in this case! The federal government? Lacking in customer service? Shocker!

So, there you have it — more thought than you ever wanted to see on Type I and Type II errors. I also promised in my earlier post that a future post would cover my observations on cognitive dissonance and the status quo when it comes to customer data management, but that’s going to have to wait for another day!

Analytics Strategy, Conferences/Community

White Paper: Testing Secrets of Success

Posted on August 18, 2009April 12, 2017 by Eric Peterson

If you’ve been reading my blog for any amount of time you’ve inevitably heard me comment that I think “web analytics is hard” — not complex, not mysterious … just plain difficult. It’s hard to select vendors, hard to install code, hard to train users, hard to get the “right” reports, hard to get management’s attention, hard to make the case for change … the list goes on and on (and on and on and on!)

Hard, but not impossible.

In the past few years we have definitely started seeing an increase in the number of companies that “get it” — so much so that we’re able to program an entire conference built around the superstars of web analytics. More and more I am talking to, working with, and hearing about companies who have leveraged the “web analytics is hard” mindset to properly set expectations regarding their use of technology, their deployment of people, and their use of business process to really excel at web data analysis and turn this analysis into tangible improvement for both the business and their customers.

As I collect more information about these web analytical competitors one thing that nearly always emerges as a hallmark of their success is some type of structured testing program. Of course this makes perfect sense because without testing the analyst is never really sure about the impact of their recommendations, so much so that I have often said “if you’re not testing, you’re not really doing web analytics!”

But the increase in testing has raised an interesting corollary to my “web analytics is hard” manifesto … testing is hard too!

Fortunately we’re all a little bit older and a little bit wiser this time around and we recognize that testing requires more than just throwing code on the page and clicking the “Optimize” button. Testing is a process that requires people and technology … sound familiar?

I’m bringing this up for two reasons:

At the X Change conference in San Francisco on September 9, 10, and 11 Matthew Wright from HP will be leading a conversation titled “Testing, Testing, Testing: Building Consensus and Evaluating Results” to discuss the nuances behind testing, things like getting stakeholder approval, planning, and clearly defining measures of success;
This morning the nice folks at SiteSpect published a white paper I wrote that details ten “best practices” for testing that I think a lot of folks new to testing often forget. Titled Successful Web Site Testing Strategies: Ten Best Practices for Building a World-Class Testing and Optimization Program, this white paper is freely available (requires registration) and covers nuances like testing teams, stakeholder involvement, test plans, timelines, and, of course, measurement.

If you’re working to become an analytical competitor and join the ranks of the kinds of companies who get invited to lead a conversation at X Change I highly recommend either grabbing the testing white paper, coming to the X Change, or BOTH! Especially if you’re serious about testing I think you’ll find this free white paper useful when you work to set expectations, and trust me, testing is as much about expectations as it is execution!

Register to download the free white paper on successful multivariate testing strategies!

Adobe Analytics

Which Pages on Your Site Matter?

Posted on August 18, 2009February 6, 2020 by Eric Peterson

Did you ever go through your clothes closet one day and figure out that your never wear half of the stuff in there? It seems like it is always much easier to buy new clothes than it is to discard old clothes. Well, the same thing hold true for websites. Most clients I worked with had thousands of web pages on their website, but in reality, only a fraction of them had an impact on their website success. Having too many pages on your website costs your business money for maintenance, translation (if your company is international) and makes the design and navigation more complex. Often times, these extra pages on your website make it more difficult for your visitors to do the small number of things you actually want them to do. In this post, I will demonstrate how you can help “trim the fat” from your website.

Finding the Pages that Matter
So how do you determine which pages matter and which pages don’t? The first step is to determine the website Success Events for which you want to optimize. If you care about multiple Success Events, this analysis becomes more complex, but the concept is similar. Therefore, in this post, we will assume a scenario where one website Success Event, Website Registrations, is the primary objective. The first thing we need to do is to ensure that a SiteCatalyst Success Event is being set for every successful Website Registration. Once this is in place, you will want to talk to your Account Manager or ClientCare and tell them to enable Participation for the Website Registration Success Event. As I covered previously in the Participation blog post, when Participation is enabled for a Success Event, Omniture will track every page in the flow leading to that Success Event and give each page “credit” for the success. Over time, the pages that are the most often in the flow, or participating, in the eventual Website Registration will have high Participation scores and those that do not, will have low Participation scores. Once Participation is enabled and has run for a while, you will see a report that looks like this:

Participation_SC

While this is optional, from here, I like to download this data to Microsoft Excel or pull into into Excel using the ExcelClient so I can re-sort the data and create any totals I need. In this case, I look and see that by the time I get to the 32nd page on my site, each page is participating in fewer than 1,000 of my 45,560 total Website Registrations taking place in this time period. Now it is important to keep in mind that many of the pages below the 31st page may have been in the flow of the top pages that led to success, but the data suggests that they were critical less often than other pages (for this particular Success Event).

Participation_Excel

If this website had 15,000 total pages, you could inform your web team that 31 website pages (.2%) accounted for the majority of your 45,560 Website Registrations. This begs the question as to what purpose the other 14,969 pages are doing!! However, I would not suggest you use this data to immediately start cutting pages from your website since there may be other purposes served by many of these pages, but rather, I do think it is reasonable to have an intelligent conversation about which pages should stay and which should go. My philosophy is that a website is comprised of a set of KPI’s and pages help achieve those KPI’s, so if you can show that a page is not “Participating” in any of the top KPI’s, then it may just be taking up space (like that 80’s t-shirt that no longer fits!). You may even find that this type of analysis leads to a smaller, simpler version of your website, which in turn makes the lives of your web developers much easier and allows them to spend more time on the pages that do matter!

Analytics Strategy

Monish Datta Stays on the Vegetarian Wagon (Sort of) at WAW

Posted on August 16, 2009October 25, 2019 by Eric Peterson

I’ve officially dropped off the first page of results for a Google search for Monish Datta. Further proof that SEO is an on-going process! Monish made a crack last year that this blog was going to start dominating search results for his name. I took his wisecrack and ran with it! “Dominate” has never really happened, but I did briefly climb into the top 5 of organic results a few months ago.

From Monish’s tweets, a handful of us knew he had gone vegetarian a few weeks ago, and no one (Monish included) knew how he would fare at a barbecue joint. As it turned out, he ordered fish, which he said was good enough to count as staying on the veggie wagon. Laura Thieme asked if Monish had actually tweeted about what he was eating. Indeed, he has!

Although I wasn’t cognizant of it as I was reading his tweets, Monish has provided fodder to Twitter critics who equate “tweeting about what you eat” to public navel-gazing. The question is: will he now become self-conscious about it, or, rather, will he go to the other extreme and provide detail at every meal? I’m sorta’ hoping for the latter.

Yes, There Was Actually a Topic Beyond Monish’s Diet and Twitter Usage

This month’s event was sponsored by IQ Workforce. Corry Prohens made the trip to central Ohio to present on what he is seeing on the jobs/careers front for web analysts and search marketers.

Highlights of the presentation included:

The Good News: internet retailers and online advertising have both continued to grow throughout the current economic downturn
The Bad News: there are more people chasing fewer full-time web analytics jobs, and there are fewer remote/virtual office positions and less willingness/need on the part of companies to relocate candidates to fill positions
Many people find their way into web analytics as a complement to another role: SEM, SEO, digital media analytics, offline marketing analytics, research/qualitative analysis, BI, etc. Having this complementary skillset clearly identified and articulated can be useful in a job search
Contracting has its pros (high demand, usually more money, more flexibility, less politics) and its cons (the need to always be selling, limited “depth” with any project/company, travel often required, administrative headaches, and it can be hard to go back to a non-contracting role)
When trying to hire a web analyst, work directly with the recruiter (don’t use an HR intermediary), write the job description yourself, be clear as to “needs” vs. “wants,” and choose a recruiter with expertise in the area you are hiring

A handful of resources that Corry provided for anyone who is looking to more effectively manage their career in web analytics:

Job board consolidators such as Indeed and Simply Hired
IQ Blast — the excellent e-newsletter that IQ Workforce produces
The Web Analytics Association
Analytics Demystified (Eric Peterson’s company)
Newsletters (besides IQ Blast):
- Internet Retailer
- WebGuild Searchnomics (Dave Culbertson was floored that he and Corry were the only two people in attendance who were subscribers to this one)
- ClickZ

My one addition would be WebSight, Stratigent’s monthly newsletter.

Attendees from Far and Wide

As I wandered around the room chatting with attendees, I realized we probably had our widest geographic reach of any of our Columbus WAWs to date:

Portland, Oregon — Noé Garcia from Webtrends (a Columbus WAW sponsor several times over)
Cincinnati, Ohio — Cindy Rabkin from Web Tech Analytics
Philadelphia, Pennsylvania — Steve Colon from Webtrends
Long Island, New York — Corry Prohens from IQ Workforce
Durham, North Carolina — several attendees from McKinney

Locally, we had had attendees from Acappella, AOL, Bizresearch, Cardinal Solutions, Cisco, Highlights for Children, JPMorgan Chase, Lightbulb Interactive, Nationwide, Ohio Historical Society, Real Estates’s SEO, Resource Interactive, Victoria’s Secret, and a few others that I probably missed (note to self: a sign-in sheet would really be helpful!)

We had a good set of mingling/mixing before and after:

Cheng Deng and Todd Greene

Dave Culbertson and Steve Krause

Monish Datta, Karen Schneider, Bill Carey, Jen Elliott, Andrew Blank, and Steve Colon

Noé Garcia and Eric Moretti

Franklin Gbenah and Carol Fleming

What’s Next?

Our next WAW is tentatively slated for September 16, 2009, but I have yet to secure a sponsor. Let me know if you can help on that front — I’ll pursue any lead!

Adobe Analytics, General, Reporting

Custom Search Success Events

Posted on August 13, 2009February 6, 2020 by Eric Peterson

I know many Omniture clients that spend much of their time using SiteCatalyst for SEO and SEM tracking. If you are one of these clients, the following will show you a fun little trick that you can use to improve your Search reporting by setting custom Search Success Events.

That Darn Instances Metric!
As a Search marketer, you tend to spend a lot of your time in the various Paid and Natural Search Engine reports within SiteCatalyst. While in those reports, you would normally use the out-of-the-box “Searches” metric for most of your reporting. If you stay in the Search reports, life is good, as you can use the Searches metric and any other Success Event to see what success takes place after visitors arrive from a particular Search Engine or Search Keyword. For example, here is a report that shows Searches and Form Completions coming from various Search Engines:

customsearch_1

However, as I blogged about a while back in my Instances post, the Searches metric is really just a renaming of the dreaded SiteCatalyst “Instances” metric. Why is that bad? It means that if you need to see Searches in any other Conversion Variable (eVars) report, you are out of luck. For example, let’s say that your boss wants to see a report that shows Searches and Form Completes (and possibly a Calculated Metric that divides the two) by Site Locale (each country in which you do business). To do this, you would open the Site Locale eVar report and add Form Completes, but guess what…there is no “Searches” metric to add to the report since it only exists in the Search Engine reports! Rats!

Let’s say you are an eternal optimist and you say, darn it, I can solve this! After pouring over past blogs, you finally arrive at the perfect answer! I can use Conversion Subrelations to break the Search Engine report down by Site Locale while the Searches metric is in the report! So you go back to the Searches report shown above and realize that all you have to do is use the green magnifying glass icon to and break the report down by the Site Locale eVar (which BTW will only work if Site Locale has Full Subrelations enabled). I’m a genius, you think to yourself! Then you wait for the report to load…brimming with anticipation only to see this…

customsearch_2

Yuck! What’s up with all of the “n/a” values? Foiled again by the darn Instances metric!

Don’t Panic!
Don’t be so hard on yourself since if you got that far, you are ok in my book! Just consider this a well earned lesson on why you have to be careful around any Instances metric (don’t fall for the same thing with Product Views!). As always, I don’t like to just present problems since the Omni Man is all about solutions! To solve this enigma, we have to find a way to get around the Instances metric. At a high level, the solution is to set custom Success Events when visitors arrive at your site from a Search Engine. I usually set a Natural Search, Paid Search and Paid + Natural Search metrics. This can be done in several ways, but the easiest way is through the Unified Sources Vista Rule or the JavaScript equivalent known as the Channel Manager Plug-in. Regardless of how you implement it, once you have true custom success events set when visitors arrive from a search engine, you can use these success event anywhere within Omniture SiteCatalyst which means that you can now create the report you were looking for above like shown here:

customsearch_3

The following are some other advantages of using a custom success events for Searches:

You can use these metrics in Calculated Metrics (i.e. Shopping Cart Additions/External Natural Search) without having to rely upon the ExcelClient
You can create Alerts on Paid or Natural Search metrics
You can add some cool SiteCatalyst Plug-ins or advanced features to the new Custom Search success events that make them even better than the out-of-the-box Searches metric (i.e. Avoid back button duplicate counting by using the getValOnce plug-in or Event Serialization).
You have an easy way to create a metric report for Searches (see below) and add it to a SiteCatalyst Dashboard

customsearch_4

The only caveat I will give you is that the new custom Search metrics will probably never tie exactly with the out-of-the-box metrics, but in many cases you can make them more accurate and useful. If SEO/SEM is something that is important to your organization, I suggest you talk to Omniture Consulting and give it a whirl… Let me know if you come up with any other cool uses for this functionality…

Analytics Strategy

Rare x Rare x Rare in Customer Data Management

Posted on August 10, 2009October 25, 2019 by Eric Peterson

I once had an operations management professor who asked the class how often we would expect a product to be defective if it was made of 10 components, each of which had a 1% defect rate, if a single component failure would result in the entire product not working.

The math is pretty simple:

99% x 99% x 99% x 99% x 99% x 99% x 99% x 99% x 99% x 99% = 90.4%

Only 90.4% of the finished products would work? That doesn’t seem good at all! Considering that there are very few manufactured products — especially electronic ones — that have only ten critical parts, it was an eye-opening insight (albeit obvious in hindsight).

The point the professor was making was that there are many cases where “99% perfect” really isn’t good enough when that one part is considered in a larger context.

Of late, I’ve had a few run-ins with the opposite insight. Stick with me — it’ll be fun!

For starters, customer data management processes are not automated manufacturing processes. Customers are people, and people are messy!

In a manufacturing environment, a key way to drive quality is to remove as much variability as possible by strictly controlling the environment. Customers (people) are none too keen about being “strictly controlled.” From a pure (read: manufacturing) customer data perspective, what we’d like is:

To have every human assigned a unique ID
To have every human log into a system once a week and update all sorts of meta data about themselves:
- Who they are related to and how (using those people’s unique IDs)
- What products they own
- How old they are
- How much they weigh
- What their favorite flavor or ice cream is
- What political party they support
- …and so on
To enter all of this information from drop-down lists so that all of the data is structured
To have them be very, very careful when they update this information, maybe even swearing that the data is perfectly accurate, under penalty of severe consequences

Obviously, that ain’t gonna happen.

Our processes to manage customer data are very different from manufacturing processes for one simple reason: the data does not have to be perfect. It has to be good enough for us to effectively interact with our customers.

Here is where a similar example to the one that started this post comes into play. When working on processes that deal with customer data — creating or maintaining it — we all develop use cases and scenarios to ensure that we are keeping the data as accurate as possible. It is exceedingly easy (and awfully tempting) to start working with scenarios that are theoretically ossible but not very probable. If we’re not wearing our Hat of Practicality, we will find ourselves developing processes that are so inordinately complex that one of two things happen:

We never get the new process implemented because it collapses under its own developmental weight, or
We implement it, but it is so complex that it collides with itself and starts generating bad customer data!

Is this sounding theoretical? I’ll illustrate with an example.

A couple of weeks ago, I ran into an issue that had to do with a new third-party data cleansing process that we are introducing that involved sending customer name and address data to a third party service (all over obscenely secure channels and with no more personal information than could easily be found in a phone book or through Yahoo! People). During testing, we came across some unexpected behavior as to how the third party vendor handled hyphenated last names. The initial proposal was to throw out responses for any customer who had a hyphenated last name. Something seemed amiss with that approach.

I thought up the most plausible scenario I could where the returned data would actually be incorrect, and it looked like this (I’ll spare you the details as to why this scenario was the most plausible — just trust me):

John Smith marries Mary Jones and they both keep their original surnames
They have a son named John Smith-Jones
When John Smith-Jones is a teenager, he becomes a customer of the same company of which his dad is a customer
When John Smith-Jones graduates from high school, he moves out of the house, while both he and his father remain customers of that same company

In this scenario…the process would be a little broken — in a way that the customer (the father, in this case) would probably understand and would definitely be able to easily correct.

So, here comes the math. Without doing any research beyond my own gut-based estimates from 37 years of experience on planet Earth, I made conservative estimates for all of the variables involved:

The percent of all married couples in the U.S. where both parties have kept their original surnames: 1%
The percent of all kids in the U.S. with hyphenated surnames: 0.5%
The percent of all kids in the U.S. who share the same name as their mother or father: 2%
How often a kid in the U.S. is a separate, distinct customer of the same company that his parents are (in the particular space this company is in) at the point that he/she leaves home: 75%

Then comes the math. It works just the opposite from the original equation, in that it is an “AND” situation rather than an “OR” situation — all of these factors had to be met in order for the process to make an erroneous customer data update (as opposed to any one of the components having to be defective in order for the final product to be defective):

1% x 0.5% x 2% x 75% = 0.000075% (!!!)

If my estimates were accurate, which they almost assuredly were not, then we would make this customer data error roughly once for every one million customers. If you think about it, you realize that the absolute accuracy of the small percentages just doesn’t really matter once those small percentages start multiplying. Let’s say I was off by a factor of four on my estimate of the percent of kids with hyphenated last names, so the formula above should have 2% where it had 0.5%. That ups the likelihood of this data error occurring to 3 times in a million rather than less than one — given the highly non-catastrophic nature of the error, this is still an “almost never” when it comes to looking at the types of other, more critical customer data errors that occur day in and day out.

In this case, there was another factor that I could have applied, and that was, for those one million customers, how many would be affected in any given year? 13% is a fair estimate of how many people move each year in the U.S., which means we would need to apply that percentage to the original result…and we’re back to “effectively never” for our likelihood of occurence.

There are a couple of caveats here, and they’re important:

I came up with one scenario. If there were four other plausible scenarios that were all equally likely to occur, then I would need to multiply the final result by five. In this case, we’re still talking a very small number, but there may be cases where a particular process gap could cause problems in a long tail’s worth of scenarios and may need to be viewed differently
It’s worth vetting the estimates somewhat — not through extensive research, necessarily, but at least by running them by a couple of sharp people to see if they pass the sniff test

In this example, we were deep into testing — well past the point where code updates could be made without introducing risk to the overall implementation. To me, it was a no-brainer — proceed as planned!

The pushback I’ve received in other, similar situations, has been: “Well, yeah, that’s only one person in a million. But…what about that one person?!” THAT gets us to my next post, which will be about Type 1 vs. Type 2 errors and cognitive dissonance when it comes to both knowing that the status quo is bad but also assuming the status quo is right. More on that next time!

Photo by Bill Burris

Adobe Analytics, Reporting

Classifying Out-of-the-Box Reports

Posted on August 10, 2009February 6, 2020 by Eric Peterson

While there are many great out-of-the-box reports in Omniture SiteCatalyst, there is one key limitation to them that can cause problems from time to time. This limitation is that you cannot apply SAINT Classifications to out-of-the-box reports. In this post, I will demonstrate why this can cause issues and how I get around this limitation.

What’s The Big Deal?
So you cannot classify some out-of-the-box reports. What’s the dig deal? Let me show you a real-life example of where this limitation comes into play. Let’s imagine that your boss tells you that he needs to see a weekly report of the top 25 Natural Search Keywords leading to Site Registrations. No problem! Simply open the Natural Search keywords report, add the Site Registrations Success Event and schedule the report for delivery (easy enough!). However, the life of a web analyst is never that easy. Next your boss says that he needs to see the same weekly report, but broken out by Branded vs. Non-Branded Natural Search Keywords. Uh oh! Now you have a problem. Your first thought is to use the ExcelClient to download the Natural Search Keywords report and then use a pivot table to group each Keyword into Branded vs. Non-Branded buckets. However, you soon realize that this will soon become a maintenance nightmare as you will have to manually do this each week and there isn’t an easy way to distribute the report to all Omniture users like you can through a SiteCatalyst Dashboard. So next, you recall reading a [brilliant] blog post about Classifications and realize that the easiest thing to do would be to classify the top 200-300 Natural Search Keywords and then add the Branded vs. Non-Branded Classification version of the report to a SiteCatalyst Dashboard. This would only require a one-time work effort and barely any maintenance. Problem solved! However, when you go to the Admin Console to add a Classification to the Natural Search Keywords report, you soon discover, that there is no way to do this (why, Omniture why?). The inability to classify this report can have a real negative impact on end-user adoption, which is why at times, this can be a big deal.

But this is not the only place where this limitation can haunt you. Another common example, is the Visit Number report. It is pretty cool that you can look at the Visit Number report and add a Success Event metric and see what percentage of success takes place within the first visit, second visit, etc… But if your site has a “long tail” it may take many visits for success to take place. How would you like to present your boss with a report about Internal Searches that looks like this:

Custom_OOB_VisitNum

While not the worst thing in the world, this report does not provide an easy way to perform analysis, nor does it “tell a story” at an executive level due to its level of granularity. However, if you could classify the Visit Number report, you could create a more functional report like this:

Custom_OOB_VisitNum2

Here we can more easily see that the bulk of Onsite Searches are being conducted by first timers and those who have been on the site many times which can lead to follow-on questions.

The following are some of the places where I have run into this limitation:

Search Keywords
Search Engines
Visit Number
Referrers/Referring Domains
GeoSegmentation Country, Region, City, etc…

The Workaround
So if this limitation has affected you or you could see how it might in the future, how do you get around it? Thankfully, the solution is very easy if you know what you are doing. To get around this problem, all you need to do is to use JavaScript (or in some cases a VISTA Rule)to copy the values stored in these out-of-the-box reports into regular Traffic Variables (sProps) and Conversion Variables (eVars). By duplicating this data into custom variables, which can be classified, you can use the Menu Customizer to steer your users to the custom versions of each report (which contain the Classification) instead of the out-of-the-box versions. I have seen this quick/easy solution help clients turn otherwise unused reports into versions that are popular amongst SiteCatalyst end-users.

Analytics Strategy, Conferences/Community, General

Interview: John Lovett from Forrester Research

Posted on August 9, 2009April 12, 2017 by Eric Peterson

Following up my interview with Bill Gassman a few weeks ago I realized that I would be remiss if I didn’t build on Forrester’s recent Web Analytics Wave report with an interview with John Lovett. John, like Bill, totally, totally understands the web analytics industry, and in that understanding is able to clarify the marketplace in a way few others can. Don’t believe me? Check out his response to possibly the worst article about web analytics, ever. Measured, polite, even complimentary … that’s John.

I am personally honored that John accepted my invitation to return to the X Change this year and both lead the huddle on “Industry Standards (or a lack thereof)” and co-lead a huddle on technology with Bill Gassman. If you haven’t met John personally, and if you are able to join us at the X Change, I strongly recommend you make a point of introducing yourself to him.

Finally, before my questions and John’s answers, I wanted to point out how incredibly deft Mr. Lovett really is: in response to a high-and-hard fastball question about “which vendor is really the best,” John knocked the ball clear out of the park with his answer: none of them. I’ll let you read the rest for yourself …

Your recent Wave report really emphasized a lot of conventional wisdom about the web analytics vendors but had some surprises for folks. What surprised YOU the most about the Wave results?

Well Eric, I like to say that surprises are for birthdays and not for business. So in terms of actual surprises, there weren’t any big bombshells for me. I was however pleased that the vendors demonstrated innovation in a number of areas (like social media measurement) and that despite my attempts to develop extremely challenging criteria, the vendors continue to improve year over year.

One comment people have made to me is that they question the validity of comparing fee and free solutions in a single matrix due to the fundamental differences in their business model. How would (or do) you respond to that challenge?

That’s preposterous! I respond by saying that it’s negligent not to compare free vs. fee based solutions. In today’s economic environment if you’re not watching expenses by understanding the cost to benefit ratio of your Web analytics solution, you are acting irresponsibly. Free tools have merit for many organizations as both primary and secondary tools, while fee based solutions are more appropriate for others based on their capabilities. Organizations must do their diligence to understand what they need in a Web analytics solution to decide what’s right for them, which is really the insight the Wave attempts to provide.

I asked Bill Gassman from Gartner a variation on this question recently, but do you now or see in the near future a situation where you as a Forrester analyst are advising your clients to actively consider these free solutions in addition to “traditional” web analytics solutions from Omniture, Coremetrics, and Unica? As a follow-up, how do you see free tools impacting the market in the next 12 to 24 months?

I advocate that a single system for measurement is always the best way to go, yet recognize that this isn’t always feasible. Duality of Web analytics tools is a reality for myriad reasons. Thus, company’s need to manage their data dissemination practices to ensure comprehension and mitigate doubt. This is tricky, but certainly possible. I often help clients determine which solution is best suited to meet their needs and financial implications are always a part of that discussion.

With regard to how free tools will impact the market: we are just witnessing the beginning of the incoming tide on this one. By this I mean that “free” will continue to disrupt the market by placing pressure for improvement on all vendors. Just look at the recent Webtrends product upgrade announcement – the majority of press around it cited a “look out Google Analytics” slant. Why the comparison…they’re worried! Fee-based vendors have even more to fear now that Yahoo! Web Analytics opened up its partner program.

Another comment I hear about the Wave results, and forgive me this, is that they’re lame because they do nothing to differentiate the “market leaders” who appear as a tight cluster. The evidence cited is that all four vendors issued press releases declaring their “market leadership” which appears technically correct based on the Wave but as the Highlander said, “There can be only one.” First, how do you respond to this and second, who is the real market leader in web analytics?

Here’s the dirty little secret – the real market leader is the wildly talented Web analytics practitioner. It’s not the tools that differentiate it’s the craftsman. Any company that believes the Web analytic technology alone will make them incredibly successful is delusional or just plain out of touch. There is no get rich quick scheme here. Each of the leading vendors on the Wave offers a highly customized solution that can be tricked-out to meet nearly anyone’s individual needs. But this takes a great deal of work. For those organizations that are looking for the far-and-away winner in this technology category, guess what: the tools will only get you so far – you need talented people to really make it happen.

Rumors are that Omniture has a bunch of “800 lb gorillas” hanging in their offices right now. Clearly they’re proud of their position, but last quarters results highlighted that there are clear risks to their business that are beginning to manifest. What do you think are the greatest risks to Omniture’s business over the next 18 months?

Well, I don’t buy into rumors and sure don’t know where I left my crystal ball. But things are tough all over. As I stated earlier, free solutions are threatening all fee-based vendors and forcing them to work harder. I can tell you that measurement technologies are an imperative for executing on digital marketing endeavors. Solutions like Omniture’s, Webtrends’, Coremetrics’, Unica’s and everyone else’s will continue to play an important role in the evolution of organizations conducting business online. I believe that Web analytics is increasingly becoming an integrated service and expect to see things evolve to easier access to data through new and alternative means. The leading vendors, including Omniture, will play a role in this evolution.

What’s your taken on the current hype cycle around “open”? Omniture bangs the Genesis drum, Coremetrics connects, and now WebTrends appears to have decided that “open” will be the foundation of their future success (or lack thereof) … but some people think that “open” is a check-box requirement, not a competitive differentiator. What do you think?

Open is not a feature, it’s a philosophy. The ability to get data into and out of a Web analytics solution is the crux of the issue and leading vendors facilitate this through bi-directional API’s, other import and export functions and data dissemination capabilities. Webtrends is currently doing this as well as anyone, but “open” also means talking to your customers about development plans, listening to criticism and demonstrating a willingness to change. These qualities aren’t unique to Webtrends, they’re characteristics that all vendors should exhibit. Webtrends is just marketing around them and if that’s causing people to want open, then it appears to be working.

As a previous attendee to the X Change what do you like best about the conference and what would you like to see us change this year or next?

I appreciate the intimate conversational format of X Change. The huddles really facilitate deep thought, controversial leeway and provocative discussion. As someone who attends a number of conferences, it is refreshing to engage in dialogue with individuals who are passionate about what they do and to initiate a true collaborative thinking environment. As far as change goes, I really hope to be able to guide the huddles that I’m leading toward resolution. Within our industry, all too often we surface problems and issues without identifying solutions. I’ve taken your challenge to heart and hope to walk away with some tangible results from my huddles.

John will be joining Bill Gassman, Gary Angel, June Dershewitz, and over 100 expert users, consultants, and vendors at the 2009 X Change conference in San Francisco on September 9, 10, and 11. Registration is currently underway and we’d love to have you join us! For more information please visit:

http://www.xchangeconference.com

General, Reporting

My Favorite v14.6 New Features

Posted on August 6, 2009February 6, 2020 by Eric Peterson

A few weeks ago, with the release of SiteCatalyst v14.6, there were a few interface features added that people like me have been requesting for a long time. While there were many new items released, two of the more simple ones can go a long way to making the lives of power users easier. Below is a quick description of these two enhancements and why I like them.

Send Link
Have you ever worked hard to create a beautiful report in SiteCatalyst and wanted to share it with others at your company? To do so, you usually have to save it as a Bookmark or to a Dashboard and then share that Bookmark or Dashboard and then tell users how to find it and add it to their list of Bookmarks or Dashboards. Alternatively, you could send it to them in PDF/Excel/CSV format, but then they cannot manipulate it (change the dates, add different metrics, etc…). Well all of that is a thing of the past now since you can now easily send a link to the exact report you are looking at to one of your peers. The only prerequisite is that they have a log-in to SiteCatalyst and have security access to the report suite and variables used in the report. This is a real time-saver and I think will be useful in driving SiteCatalyst adoption by getting people into the tool to explore vs. always looking at reports sent via e-mail.

To send a link to a report, simply click the new icon found in the toolbar…

14_6_SendLink

…and you can copy this link and send it to people at your organization. I was told that these links would be good for a year which should be plenty of time. The way I am excited to use this feature is in PowerPoint presentations where you can put a screen shot of a report and then make the entire screen shot image a hyperlink to the real report so when you are presenting you can easily dive right into the report without having to fumble around to find different reports when you are short on time and/or in front of executives.

My only complaints/enhancement requests of this new feature are as follows:

I would like to be able to have this feature for Dashboards as well

It would be cool if you could e-mail the link to SiteCatalyst users be picking names from an address book since they all exist in the Admin Console anyway. Even better if you could set-up some groups for people who you commonly e-mail

In the future, it would be interesting if you could send the link to a Publishing List which would show the same report, but for a different report suite to different groups of people (however, this would mean you need to check a box to determine if the link is variable or not like Dashboard reportlets)

Update Dashboard Reportet
The second new feature I love is the ability to update Dashboard reportlets. Using this feature, you can now make changes to a Dashboard reportlet much more easily than in the past. Previously, to update a Dashboard reportlet, you would have to:

Open the Dashboard

Launch the reportlet into full view

Make your changes

Click to add the new version back to the Dashboard

Update the reportlet settings

Wait for the Dashboard to open

Delete the old version of the reportlet

Move the new version to the correct space (phew!)

Now you can accomplish the same thing by doing the following:

Open the Dashboard

Launch the reportlet into full view

Make your changes

Click the new link (shown below) to update the Dashboard reportlet

As you can see, this is much easier and much more intuitive for end-users. In addition, you can even change report suites and view the same reportlet for a different data set and update it and it will be saved back to the Dashboard tied to the new report suite! Very exciting for Omniture guys like me!

Well those are my two favorite enhancements, but I know there were many more made. Let me know if you agree/disagree that these two items are useful or if there are other feature updates that you have found useful or if you have additional suggestions on how these two can be improved (maybe Omniture Product Management will end up reading these!). Thanks!

Analytics Strategy

How much do you pay for web analytics?

Posted on August 3, 2009April 12, 2017 by Eric Peterson

I was just cruising through the just published WebTrends 9 update and thinking about how the web analytics vendor market is evolving. “9” looks neat and I’m sure glad to see some really important metrics like bounce rate appear in the UI. Still, I always scratch my head when I see vendors make statements like “[the] data visualization tool in Webtrends Analytics 9 lets anyone – even analytics novices – quickly and easily understand changes in key metrics” and then put up a feature list like this one.

Still, it’s nice to see WebTrends making some moves so congratulations to Jascha, Casey and the entire Portland crew for getting the update out the door!

Anyway …

I said I had been thinking about the evolution of the web analytics vendor market. A lot of my thinking this past week has been colored, well, purple, thanks to the announcement of Yahoo’s Web Analytics Consulting Network (the YWACN or, as I think about it “the Yack’n!”.) On July 30th Yahoo announced that they were making Yahoo! Web Analytics much easier to get through 48 partners around the globe.

Now, when you look at the partner list you might not recognize a lot of the names — I sure don’t — but a few should stand out. Specifically Stratigent, Semphonic, Sapient, and my own company Analytics Demystified. While I can’t speak directly for any of these companies, all are run by very smart people, and I have to wonder if they’re not thinking about YWA much the same way I have been.

I mean, if you think about it, Yahoo! has basically come to us and said “Go sell excellent implementations of YWA and provide awesome ongoing support” for an application that, according to Forrester Reseach, has 77% of the core functionality of Omniture SiteCatalyst. Or, put another way, “Find companies that are struggling to get value from their existing investment in {pick a vendor}, kick that vendor out, and then make money helping them be successful for less then they spend today.”

Sweet, thanks Yahoo!

Not to brag (since it was pretty obvious) but I did say this would happen back in April 2008 given the hard work Google Analytics (who is ironically NOT a YWA competitor) had done with their similarly badly acronym’d GAAC. Yahoo wisely avoids having to support customers directly, leverages some incredibly smart folks, and lets companies reduce their annual analytics spend without having to forgo core functionality like multiple custom variables and visitor-level segmentation.

Hell, we’re not even talking about real-time updates and demographic reporting and segmentation, which while the former often has more value ascribed than necessary the latter, if I can say so based on my own usage, is pretty fantastic and not available in any other web analytics application in the market today. I mean, who would have guessed that so many mature, responsible adults love to Twitalyze themselves!

Now I sincerely doubt that any of the YWACN members are going to suddenly stop supporting the big for-fee applications out there … I know I’m not! And I fully expect the adoption of YWA to be slow and methodical (mostly because of existing contracts, Yahoo’s terms of service, and the fact that Yahoo is somewhat limiting YWACN access to new accounts although I think their strategy is fair and makes perfect sense.) But at the end of the day Yahoo has made quite possible the single best move they could have if their goal was to provide an awesome service with excellent third-party support at the best possible price.

Now if you were paying attention you may have noticed I commented that Google Analytics and Yahoo Web Analytics are not competitive. Crazy, huh? But they’re not. Google Analytics (as it exists today) and Yahoo Web Analytics (as it exists today) serve two near completely distinct target markets.

Now I know I’ll get heat for saying this (again) but I just don’t think Google Analytics is appropriate for “free standing” use within the true Enterprise. I’ll point again to Bill Gassman’s recent note on the service (which I thought was excellent) and will obviously concede that it is well within Google’s power to make GA the “bestest, most Enterprisey” web analytics application the world has ever seen … but it isn’t today. More importantly when I go looking for companies mature in their use of web analytics who rely exclusively on Google Analytics and have chosen to do so explicitly, I simply don’t find them.

I could be wrong — if you’re an analytics samurai using nothing but GA please let me know — but what I see a lot of is mature businesses using Google Analytics to back-fill some limitation in their for-fee vendor’s service. For example, up until today it was amazingly difficult to get WebTrends to calculate a bounce rate and some people think setting up visitor segments in SiteCatalyst is a lot of work. More importantly, while lots and lots of people complain about how difficult their analytics application is to use, the team at Google has done a freaking brilliant job with the GA user interface and in my humble opinion it sets the bar for ease-of-use in web analytics.

Yahoo Web Analytics in an Enterprise context, and hopefully Dennis will forgive me this since he’s tanned and rested after a week or two in the islands, is not really that easy to use, not that easy to set up, and not that easy to configure — remember it’s 77% of Omniture SiteCatalyst which nobody ever describes as “easy to implement and easy to use” (except for Adam Greco, but he’s clearly an exception!)

But here’s the secret: Yahoo Web Analytics is not supposed to be easy to use, it’s supposed to be really, really powerful! Yahoo Web Analytics is an Enterprise-class web analytics application out of the box designed to support businesses with custom data collection needs, custom reporting needs, custom segmentation needs, and the challenges typically found within any company of size.

More importantly, because of this functionality I believe that Yahoo Web Analytics will be a gateway to a much deeper relationship between the YWACN and their customers than the GAAC have found for the most part.

Yes, Yahoo’s APIs are tightly held and thusly YWA is not as “open” as WebTrends or as “integrated” as Omniture. Yes, Yahoo is keeping Rubix under wraps so it is not as flexible as Affinium NetInsight or Coremetrics Explore. Yes the interface is kinda clunky and the terms of use were written by lawyers … I get the complaints and hear the FUD loud and clear. But given the massive adoption of Google Analytics I think that coupling exceptional services and support with a free Enterprise-class application has a lot of potential to be the permanent game changer that I first described last year.

What do you think? Is purple the new green? Is “9” too little, too late? Does Yahoo! have a chance to focus now that they have outsourced their search business? Or am I missing the point and despite two great free solutions will the world continue to pay for web analytics the same way we always have? I’m totally willing to be wrong about this one … but if you don’t believe me about how powerful Yahoo Web Analytics is either read this book or contact me directly and I’ll see about getting you your own YWA account.

As always your comments are welcome.

Adobe Analytics

Page Type Pathing

Posted on August 2, 2009February 6, 2020 by Eric Peterson

When using Omniture SiteCatalyst, Pathing analysis is one of the truly unique things that is not easily replicated by other analysis tools. While your company can find a way to track how often each of your key Success Events are taking place, you would be hard-pressed to get data warehouse tools to duplicate the Pathing analysis available in SiteCatalyst. However, too many Omniture clients are limited in their thinking when it comes to Pathing, relegating it to Pages or Site Sections. In past blog posts I have covered a few unconventional ways to use Pathing (i.e. Success Event Pathing), but there are many more ways to leverage Pathing. In this post, I will show you one of my favorite uses of Pathing – Page Type Pathing.

What is Page Type Pathing?
So what is Page Type Pathing? To fully understand it, I need to put it into context. Imagine you are a web analyst at a company and your boss comes to you and asks “What is the fallout of visitors starting from the Home Page and then navigating to Product Pages, then Product Sign-up Forms and finally the Product Form Thank You Page?” Well that sounds easy enough, but is it? You can create fallout reports from each product, but what if you have hundreds of products? You can look at Site Sections, but you may have many of those as well. After a while, you may resign yourself to creating a massive dashboard with fallout reports for each product. Just then, your boss reiterates that what she wanted was a fallout of all of the steps for all of the products in one overall fall-out report (and she wants it every week from now on!). Besides learning the valuable lesson that you should always ask more questions before doing analysis, you are bummed because you don’t know how to do this other than manually add together all of these individual fall-out reports.

What your boss is asking for is what I call Page Type Pathing. This is the ability to deconstruct your website so that you group all of your pages (or at least your key ones) into buckets that represent page types. I think of it in the same way that species are grouped into classes like mammals or amphibians. Many executives don’t have time or care about page or section-level Pathing since it contains too much “noise” (and they have limited attention spans!). By lumping pages into a small number of meaningful page types, you can take a step back and see a 50,000 foot view of where people are going on your website. Sometimes, page-level Pathing can make it hard to see that 30% of your visitors go from the Home Page to Product Pages since all you can see is individual page paths to product #1 or product #2. By implementing Page Type Pathing you can end up with a new pathing report that looks like this:

PageTypePathing_NextFlow

Plus, since having Pathing enabled allows you to see all Pathing reports, you can create high-level Fallout reports using the same Page Type Traffic Variable (shown here in Discover):

PageTypePathing_Fallout

Implementing Page Type Pathing
So how do you do this? There are actually a few different ways to do this so how you implement it will depend upon which Omniture products you have and your ability to get tagging done at your organization. I will outline the ways I recommend doing it, but there may be other ways.

The Old Fashioned Way
The most straightforward way to implement this is to create a new Traffic Variable (sProp) on every page and pass in the value that you have chosen as the Page Type for that page. Obviously, you need to identify what you think your Page Types are ahead of time. The values I recommend as Page Type Values are: Home Page, Product Page, Registration Page, Search Results Page, Checkout Page, Thank You Page, Content Page, etc… However, setting a new sProp on every page can be a tagging nightmare as many of you can attest to if you have worked with IT to clean up your Page Names. If you have a good Content Management System (CMS), you can add Page Type as a required meta-data field (be sure to make it a picklist!) for website pages and have your content owners enter it for each page. But if getting tagging done is too difficult or you have too many pages to make the CMS approach feasible, go on to the other options…

Discover
If your company is lucky enough to have Omniture Discover, it is your lucky day! Implementing Page Type Pathing in Discover can be done in less than 24 hours if you know what you are doing (or reading this blog!). One of the benefits of Omniture Discover is that you get Pathing on SAINT Classifications which is not possible in SiteCatalyst. Therefore, if you create a “Page Type” Classification of the Page Name sProp, you can simply use Microsoft Excel to fill in a Page Type value for each page on your site and upload it using SAINT. The next day, after Discover processes its data, it will pick up that new Classification and presto, you have Page Type Pathing in Discover! Just be sure to thank the person at your organization who got you Discover or maybe you can use this as a reason to get it!

DB Vista
If tagging or CMS aren’t options and you don’t have Discover, what then? Don’t despair, I won’t leave you in a lurch. You can use Omniture’s DB Vista tool to get Page Type Pathing working. Simply create a spreadsheet like described in the Discover approach above, but when you are done, tell your Omniture Account Manager that you would like to purchase a DB Vista Rule in which you upload a table of Page Type values for your Pages and have the DB Vista rule do a lookup on this table and pass the Page Type value to a Page Type sProp on every page. As you add new pages to your site, you simply upload new rows to the DB Vista table. There is a nominal charge for DB Vista rules, but it is worth it.

So there you have it, Page Type Pathing in a nutshell. Once you have this functionality working you will be amazed by what you can learn about how people are navigating your site and I have seen it used by companies to drastically simplify their website with great results. Finally, don’t forget to combine this functionality with segments using Advanced Segment Insight (ASI) or Discover so you can see the same cool Page Type Pathing reports for 1^st time visitors, people from Google, etc…

Reporting

Put-in-Play Percentage: A "Great Metric" for Youth Baseball?

Posted on July 30, 2009October 25, 2019 by Eric Peterson

BB Pitching My posts have gotten pretty sporadic (…again, sadly), and I’ll once again play the “lotta’ stuff goin’ on” card. Fortunately, it’s mostly fun stuff, but it does mean I’ve got a couple of posts written in my head that haven’t yet gotten digitized and up on the interweb. This post is one of them.

As I wrote about in my last post, I’ve recently rolled out the first version of a youth baseball scoring system that includes both a scoresheet for at-the-game scoring, as well as a companion spreadsheet that will automatically generate a number of individual and team statistics using the data from the scoresheets. The whole system came about because I’ve been scoring for my 10-year-old’s baseball team, and I was looking for a way to efficiently generate basic baseball statistics for the players and the team over the course of the season.

The Birth of a New Baseball Statistic

After sending the coach updated stats after a couple of games mid-season, he posed this question:

Do we have any offensive stats on putting the ball in play? I’m curious to know which, if any, of the kids are connecting with the ball better than their hit stats would suggest. That way I can work with them on power hitting.

How could I resist? I mulled the question over for a bit and then came up with a statistic I dubbed the “Put-in-Play Percentage,” or PIP. The formula is pretty simple:

Put-In-Play Percentage Formula

Now, of all the sports that track player stats, baseball is at the top of the list: sabermetrics is a term coined solely to describe the practice of studying baseball statistics, Moneyball was a best-selling book, and major league baseball itself is fundamentally evolving to increase teams’ focus on statistics (including some pretty crazy ones — I’ve written about that before). So, how on earth could I be coming up with a new metric (and a simple one at that) that could have any value?

The answer: because this metric is specifically geared towards youth baseball.

Blog Reader Timesaver Quiz

Question: In baseball, if a batter hits the ball, it gets fielded by the second baseman, and he throws the ball to first base and gets the batter out, did the batter get a hit?

If you answered, “Of course not!” then skip to the next section in this post. Otherwise, read on.

One of the quirks of baseball — and there are many adults as well as 10-year-olds on my son’s team who don’t understand this — is that a hit is only a hit if:

The player actually reaches first base safely, and
He doesn’t reach first base because a player on the other team screwed up (an error)

“Batting average” — one of the most sacred baseball statistics — is, basically, seeing what percentage of the time the player gets a hit (there’s more to it than that — if the player is walked, gets hit by a pitch, or sacrifices, the play doesn’t factor into the batting average equation…but this isn’t a post to define the ins and outs of batting average).

PIP vs. Batting Average

Batting average is a useful statistic, even with young players. But, as my son’s coach’s question alluded to, at this age, there are fundamentally two types of batters when it comes to a low batting average:

Players who struggle to make the split-second decision as to whether a ball is hittable or not — they strike out a lot because they pretty much just guess at when to swing
Players who pick good pitches to swing at…but who still lack some of the fundamental mechanics and timing of a good baseball swing — they’ll strike out some, but they’ll also hit a lot of soft grounders just because they don’t make good contact

(Side note: I’m actually one of the rare breed of people who fall into BOTH categories. That’s why I sit behind home plate and score the game…)

What the coach was looking for was some objective evidence to try to differentiate between these two types of players so that he could work with them differently. Just from observation, he knew a handful of players that fell heavily into one category or the other, but the question was whether I could provide quantitative evidence to confirm his observations and help him identify other players on the team who were more on the cusp.

And, that’s what the metric does. Excluding walks, hit by pitches, and sacrifices (just as a batting average calculation does), this statistic credits a player for, basically, not striking out.

But Is It a Great Metric?

Due to one of those “lotta’ things goin’ on” projects I referenced at the beginning of this post, I had an occasion to revisit one of my favorite Avinash Kaushik posts last week, in which he listed four attributes of a great metric. How does PIP stand up to them? Let’s see!

Attribute	Summary (Mine)	How Does PIP Do?
Uncomplex	The metric needs to be easily understandable — what it is and how it works	PIP works pretty well here. While it requires some basic understanding of baseball statistics — and that PIP is a derivation of batting average (as is on-base percentage, for that matter) — it is simply calculated and easy to explain
Relevant	The metric needs to be tailored to the specific strategy and objectives they are serving	This is actually why PIP isn’t a major league baseball stat — the coach’s primary objective in youth baseball is (or should be) to teach the players the fundamentals of the game (and to enjoy the game); at the professional level, the coach’s primary objective is to win as many games as possible. PIP is geared towards youth player skill development.
Timely	Metrics need to be provided in a timely fashion so decision-makers can make timely decisions	The metric is simple to calculate and can be updated immediately after a game. It takes me ~10 minutes to enter the data from my scorecard into my spreadsheet and generate updated statistics to send to the coach
“Instantly Useful”	The metric must be able to be quickly understood so that insights can be found as soon as it is looked at	PIP met this criteria — because it met the three criteria above, the coach was able to put the information to use at the very next practice.

I’d call it a good metric on that front!

But…Did It Really Work?

As it turned out, over the course of the next two games after I first provided the coach with PIP data, 9 of the 11 players improved their season batting average. Clearly, PIP can’t entirely claim credit for that. The two teams we played were on the weaker end of the spectrum, and balls just seemed to drop a little better for us. But, I like to think it helped!

Analytics Strategy, Conferences/Community, General

Interview: Bill Gassman (Gartner) on Google Analytics

Posted on July 26, 2009April 12, 2017 by Eric Peterson

Bill Gassman from Gartner is one of those guys that just “gets” what we’re trying to do in the web and digital analytics industry. Perhaps because he’s been covering this space for nearly as long as I’ve been around, or perhaps because he has a deep business intelligence background and sees where all this is going. I dunno, but Bill gets it.

Recently Bill, who is incidentally coming to the 2009 X Change and leading a huddle on organizational issues and co-leading a huddle on technology with John Lovett from Forrester, published a short brief on Google Analytics that I thought really hit the nail on the head. Clear, honest, and fully taking the Enterprise into account, Bill’s report clarified a lot about how companies should be thinking about Google’s analytics solution.

Since I could not get permission to republish Bill’s report I did the next best thing — I came up with some questions and put them to the man himself. The following are my questions and Bill’s responses. Incidentally, if you want to follow-up on this interview Bill graciously said he would monitor the comments and respond there (so comment away!)

Or, you could just come to San Francisco on September 9, 10, and 11 and debate the goodness of Google Analytics with Bill in person.

Regarding your recent note on Google Analytics, can you characterize how the companies who are asking you about “free” analytics have changed in the last 12 months? Is any one thing driving that change, do you think?

Since Google Analytics improved last October, most client inquires about Web analytics touch on Google Analytics. That is why I published the note “Is Google Analytics Right for You?”. (Gartner account required to access) Marketing departments ask if it is all they need, purchasing agents wonder why they should spend money on commercial tools and corporate lawyers wonder about Google’s terms and conditions. The economy and budget constraints trigger the questions, but the major driver to Google is its simplicity. Many organizations do not have the processes in place to make use of the high-end products or have Web sites that do not need the sophistication they offer. They perceive Google Analytics as good enough and “free” is a tempting offer.

If Google asked you which three things were most important to add to their functional set to be considered “Enterprise” what would those three things be?

Getting to functional parity with the commercial tools is not enough for Google Analytics to be considered enterprise class. Google should charge for an enterprise class offering, because a lot is required as is the accountability that goes with the exchange of money. They must also provide enterprise class support and address issues with the terms of service policy. The key function missing is a visitor centric repository, so users can define complex multi-session segments and export visitor information to campaign and content management tools. Google would also have to extend personalized service from customers with significant AdWords accounts to those willing to pay for it. Finally, the terms of use policy has no service level guarantee and users must look for a FAQ to find assurances of privacy.

On the subject of “Enterprise-class” analytics … Google appears obsessed with this designation, regardless of their clear dominance from a deployment standpoint and the gains they’ve made within larger companies. What do you think is behind their obsession?

Google appears to be fighting an asymmetric war with IBM, Microsoft and others, investing relatively little yet forcing competitors to take notice. We see this especially for office applications and cloud computing. Google is looking for products that give them credibility at the enterprise level, and Google Analytics is part of that story. Other parts of the story include the recent news about the Chrome OS, Google Search Appliance, Google apps (premier edition), Geospatial Solutions, and Google App Engine for cloud computing. While many large organizations are using some or all of these offerings, with few exceptions, only the Search Appliance has gained strategic status. Google still brings in 97% of its revenue through advertising. They might be showing obsession because of where they want to be, but then again, they could be throwing up a smoke screen to keep the competition too busy to attack Google on advertising.

What do you consider the single greatest risk to Google’s analytics business in the next 24 months?

There is no threat to Google’s analytics business, because Web analytics is not their business, yet. Out of 72 million active Web servers (as reported by Netcraft), about 20,000 organizations pay for Web analytics. Google gives away Google Analytics so that millions of Web site owners can see the impact of AdWords and buy more Google ads. If there is a threat, it is Yahoo Web Analytics, who is using a similar tactic to go after Google’s advertising revenue.

Are you now, or do you see in the near future, a situation where as a Gartner analyst you are advising your clients to actively consider free solutions from Google and Yahoo alongside “traditional” web analytics solutions like Omniture, WebTrends, and Coremetrics?

Running two sets of tools on the same Web pages can be a recipe for trouble, because reported numbers will not match, reducing respect and therefore value for both tools. There are situations however where two tools make sense. It would be great if all organizations had the leadership, investment, skills and processes to use commercial tools to meet everyone’s needs, but for too many, it has not worked out that way. When analytic resources are limited, it is pragmatic to focus commercial tools on the high-value parts of the site and let other site stakeholders use free tools. Analysis is a critical part of a customer centric Web strategy. If some departments are happy with the free tools and a central group cannot support them, it is OK to let chaos reign until the business justification, investment and leadership are available to do things right.

Reporting

Perfect Game / Pretty Good Youth Baseball Scoring System

Posted on July 24, 2009October 25, 2019 by Eric Peterson

I’ve had this post half-written for a few days, but it became more timely last night when Mark Buehrle pitched a perfect game for the Chicago White Sox, so it just became a “finish it up over lunch” priority. I’ve got my own little baseball-related accomplishment that I added to my site earlier this week — it seemed worth a blog post to formally announce it.

I discovered baseball relatively late in life (as in — not as a kid), and there’s been something of a perfect storm that’s made that happen:

I had several friends who were interested in Texas Longhorn baseball, and I got hooked on going to their games when I lived in Austin
My career evolved towards business data, and there are a lot of parallels between business metrics and baseball statistics — I’ve written on that in the past (and have a new post soon to come on the subject)
My oldest son really, really enjoys baseball, and I’m worthless as an assistant coach due to my lack of eye-hand coordination and my lack of “coaching kids” skill; so, the best way I have to be a parent contributor is to be the team scorer…which is really what led to this post

My son’s coach for the past two seasons ia an ex-college baseball player and current IT executive, so he has a deep understanding of the game and how/why stats can help identify players’ strengths and weaknesses. In other words…he’s become something of an enabler of my enthusiasm. 🙂

Partly for fun (okay…mostly for fun), I developed a spreadsheet last season that could take my box scores for each game and generate individual and team stats. In between last season and this season, I developed a scoresheet that would integrate well with that spreadsheet. One key aspect is that the scoresheet was designed specifically for youth baseball — typically, there are more frequent defensive position changes and, up through a certain age, all players on the team bat, rather than just the nine players who are playing in the field. This is the case for both Little League and Pony baseball.

I’ve now added a permanent page on this site with the whole scoring system. It works great for me and for the age group/league in which my son plays. I’m hoping to get some other users of the system who can provide feedback to make it more robust. Check it out!

Analytics Strategy

Columbus WAW July 2009 Recap — Bizwatch and More!

Posted on July 19, 2009October 25, 2019 by Eric Peterson

We had another great Columbus Web Analytics Wednesday last week at Barley’s Smokehouse and Brewpub. This month’s sponsors were Bizresearch and the Web Analytics Wednesdays Global Sponsors. We had right at 30 people attending:

Laura Thieme of Bizresearch presented on search marketing and the challenges of trend analysis therein. She walked through one in-depth case study and sprinkled examples from other clients into the discussion as well.

Bizresearch has a product called Bizwatch that, when combined with some fundamental best practices of SEO and SEM, looks like it can yield some handy insights in a hurry! Laura is a self-professed constant tinkerer with her presentations, but I think the one below is pretty close to what she walked us through:

Bizwatch Search Analytics Solving PPC & SEO Problemswith Bizwatch http://static.slidesharecdn.com/swf/ssplayer2.swf?doc=bizwatchsearchanalytics-solvingppcseoproblemswithbizwatch-090706000851-phpapp02&stripped_title=bizwatch-search-analytics-solving-ppc-seo-problemswith-bizwatch

View more presentations from Laura Thieme.

As the group grows, I’m finding that the evenings wind down and there are people I didn’t even get to say “Hi” to — both new attendees and long-timers. But, some of the discussions I had included:

Chatting more in-depth with Chris Dooley of Foresee Results. I’m looking forward to a future WAW when Chris will be challenging the group to think about the offline and post-visit behavior of site visitors and how often that doesn’t get considered by internet marketers. I also picked up a new blog to follow, as Chris mentioned that Kevin Ertell had joined Foresee Results and recommended his www.retailshakennotstirred.com blog. Ertell only started the blog last month, so it remains to be seen if it has legs. So far, his posts look to be pretty in-depth and grounded in real experience. Chris also mentioned that Eric Peterson wrote a white paper for Foresee Results, which had me poking around on the White Papers area of their site — it looks like there are some really good reads there! I’m not sure which one Eric wrote…so I may just have to download several of them!
Chris also mentioned a new “session replay” tool that Foresee Results just introduced called CS Session Replay. It sounds like a direct threat to Tealeaf, but it is apparently wayyyyy slicker. I don’t know if Chris (right below) was extolling the virtues of the product to Scott Zakrajsek of Victoria’s Secret or not. They might have just been discussing who put those half-drunk beers down next to the glasses of water they were drinking…

I chatted with Paul Hall of the Mastery Marketing Group about the work they’re doing to drive a “360 degree view of the customer” using data from multiple systems (CRM and other). That discussion led to me bringing up Webtrends, Omniture, and Eloqua all as tools that I know of that have very real capability to do user-level tracking and analysis of web activity.
Our “farthest travelled to attend” award (not really an award…just me pondering after the fact) went to Kim Merritt-Butler of TheURLdr.com. Kim was in town from Cumberland, Maryland, and is very interested in getting a similar group started up in the Washington, D.C. area. So, if you know of anyone in that area whom Kim should get in touch with, please let me know and I’ll pass the information along! We were both surprised that there is not already a WAW — even an older/dormant one — in that area already.
I didn’t get to chat with him much, but Gareth Dismore could’ve made a case that he’d actually travelled the farthest, as he’s now based in Colorado Springs with SearchSpring. He was back in town for a couple of weeks, and he didn’t move away long enough ago to count. Or so I decided. After the fact. For an award that exists merely as a construct within this blog post.
I talked to several people who were new to web analytics — were starting to see it crop up and are attending WAWs as a way to dip their toes in the water (which is a great place to start!). I found myself giving my standard recommendations on that front: Occam’s Razor blog, Eric Peterson’s Analytics Demystified blog, and Web Analytics: An Hour a Day. I know scads of blogs and books have cropped up over the past few years…but these still nail the basics, IMHO.

I wrapped up the evening with a lengthy discussion in the parking lot with Bryan Cristina and headed home thinking Thursday was going to come awfully quick. Then I spent an hour getting my neighbor’s garage door unmangled, as she’d backed into it with her minivan before it was fully open and was leaving on vacation the next morning. I am clearly not as young as I used to be, as I didn’t fully recover until I cratered at 9:45 PM on Thursday night and got in a solid eight hours!

Next month’s Columbus WAW is already scheduled. It will be on August 12th at 6:30 PM, again at Barley’s Smokehouse and Brewpub. The event is being sponsored by IQ Workforce, and Corry Prohens will be presenting on job hunting and career management in web analytics and search marketing. I hope to see you there!

General

SPSS Expertise? Job Opportunity — Full-time/Contract/Flexible

Posted on July 14, 2009October 25, 2019 by Eric Peterson

The Austin-based division of a qualitative and quantitative research company is looking for someone with SPSS expertise — they’re pretty flexible as to how the work gets set up, and there is not a requirement that the role be based in Austin. Below are the requirements:

Minimum require (per project): 15-20 hours a week

Software Skills (in order of importance):

SPSS (v15 to V17) – Intermediate to Expert (4+ years)
Ability to:
- Navigate using command code (little reliance on GUI)
- Develop concise syntax
- Clearly document processes
- Working knowledge of data import and export procedures
- Techniques to manipulate string and numeric data
- Logic construction and deconstruction
- Loop and do repeat functions
- Descriptive statistics
- Custom tables
- Macros
Text Editor – Intermediate (3+ years)
Excel – Intermediate (3+ years)

General Experience (in order of importance):

Data management:
- Processing
- Troubleshooting: data / logic issues
- Basic analysis
- Basic to complex reporting
Work environment: small programming teams
Online surveys
Research / market research (consumer and/or B2B)
Technology sector

General Skills

Technical orientation
Strong attention to detail
Strong problem solver

Nice to Have Skills:

Python
Visual Basic

Ping me for details (the details being “here’s the company and a contact person”) at: tim at gilliganondata dot com.

Adobe Analytics, Analytics Strategy, Conferences/Community, General

Want to Debate Standards?

Posted on July 12, 2009April 12, 2017 by Eric Peterson

One of the biggest problems we face in web analytics today is our industry’s lack of standards and common definitions. And while a great number of incredibly bright folks have put a ton of energy into solving these problems, in my humble opinion we are more or less where we started years ago — agreeing politely to disagree. Those of you who have been reading my blog for awhile know that I’m not shy about disagreement — perhaps more than anything my analyst’s mind loves a spirited debate — but I also am somewhat anxious about creating tangible outcomes.

To this end I am incredibly excited about two huddles at X Change 2009, one that was just added! The first is Forrester’s John Lovett’s “Web Analytics Standards (or a Lack Thereof)” in which John will be leading us through the current state of industry standards, proposed definitions and our collective understanding of analytics terminology. The second, and one just added to the X Change, is Jim Hassert’s “When is a Visitor Not a Real Person?” huddle in which Jim will take John’s huddle one step further and drill-down into the often irreconcilable differences found in the seemingly harmless “visitor” metric and dimension.

Last year I was forced to miss a lot of good huddles. This year a team of wild horses couldn’t keep me from missing these two.

While I have little doubt that both of these huddles will live up to the spirit of the X Change my hope is that they will go one step further. I would love to see both produce some kind of actionable outcome, something that we can carry forth into our careers and the wider conversation about our industry. Given that some serious talent is already signed up for the X Change — including some of the brightest minds in the practitioner and vendor community — I have little doubt that we have the brain power … now all we need is the resolve to do something and not just push words around on paper.

If you’re a reader of this blog and want to join us at the X Change I’m happy to help you out. If you act before July 31st I am offering a 15% discount on the registration (a $300 savings!)

Come to the X Change. Agree to do more than “politely disagree” — take a stand, defend your ideas, and help shape tangible and positive outcomes.

Presentation

Dashboard Development and Unleashing Creative Juices

Posted on July 9, 2009October 25, 2019 by Eric Peterson

Ryan Goodman of Centigon Solutions wrote up his take on a recent discussion on LinkedIn that centered on the tension between data visualization that is “flashy” versus data visualization that rigorously adheres to the teachings of Tufte and Few.

The third point in Goodman’s take is worth quoting almost in its entirety, as it is both spot-on and eloquent:

Everyone has a creative side, but someone who has never picked up a design book with an emphasis on data visualization should not implement dashboards for their own company and certainly not as a consultant. Dashboard development is not the forum to unleash creative juices when the intent is to monitor business performance. Working with clients who have educated themselves have[sic] definitely facilitated more productive engagements. Reading a book does not make you an expert, but it does allow for more constructive discussions and a smoother delivery of a dashboard.

“The book” of choice (in my mind, and, I suspect, in Goodman’s) is Few’s Information Dashboard Design: The Effective Visual Communication of Data (which I’ve written about before). Data visualization is one of those areas where spending just an hour or two understanding some best practices, and, more importantly, why those are best practices, can drive a permanent and positive change in behavior, both for analytical-types with little visual design aptitude and for visual design-types with little analytical background.

Goodman goes on in his post to be somewhat ambivalent about tool vendors’ responsibility and culpability when it comes to data visualization misfires. On the one hand, he feels like Few is overly harsh when it comes to criticizing vendors whose demos illustrate worst practice visualizations (I agree with Few on this one). But, he also acknowledges that vendors need to “put their best foot forward to prove that their technology can deliver adequate dashboard execution as well as marketing sizzle.” I agree there, too.

Analysis, General, Reporting

Where BI Is Heading (Must Head) to Stay Relevant

Posted on July 7, 2009October 25, 2019 by Eric Peterson

I stumbled across a post by Don Campbell (CTO of BI and Performance Management at IBM — he was at Cognos when they got acquired) today that really got my gears turning. His 10 Red Hot BI Trends provide a lot of food for thought for a single post (for one thing, the post only lists eight trends…huh?). It’s worth clicking over to the post for a read, as I’m not going to repeat the content here.

BUT…I can’t help but add in my own ~~drool~~ thoughts on some of his ideas:

Green Computing — not much to add here; this is more about next generation mainframes that run on a less power than the processors of yesteryear
Social Networking — it stands to reason that Web 2.0 has a place in BI, and Campbell starts to explain the wherefore and the why. One gap I’ve never seen a BI tool fill effectively is the ability to embed ad hoc comments and explanations within a report. That’s one of the reasons that Excel sticks around — because an Excel based report has to be “produced” in some fashion, there is an opportunity to review, analyze, and provide an assessment within the report. Enterprise BI tools have a much harder time enabling this — when it’s come up with BI tool vendors, it tends to get treated more as a data problem than a tool problem. In other words, “Sure, if you’ve got data about the reports stored somewhere, you can use our tool to display it.” What Campbell starts to touch on in his post is the potential for incorporating social bookmarking (“this view of this data is interesting and here is why”) and commenting/collaboration to truly start blending BI with knowledge management. The challenge is going to be that reports are becoming increasingly dynamic, and users are getting greater control over what they see and how. With roles-based data access, the data that users see on the same report varies from user to user. That’s going to make it challenging to manage “social” collaboration. Challenging…but something that I hope the enterprise BI vendors are trying to overcome.
Data Visualization — I wouldn’t have a category on this blog dedicated to data visualization if I didn’t think this was important. I can’t help but wonder if Campbell is realizing that Cognos was as guilty as the other major BI players of confusing “demo-y neat” with “effective” when it comes to past BI tool feature development. From his post: “The best visualizations do not necessarily involve the most complex graphics or charts, but rather the best representation of the data.” Amen, brother!!! Effective data visualization is finally starting to get some traction — or, at least, a growing list of vocal advocates (side note: Jon Peltier has started up a Chart Busters category on his blog — worth checking out). What I would like to see: BI vendors taking more responsibility for helping their users present data effectively. Maybe a wizard in report builders that ask questions about the type of data being presented? Maybe a blinking red popup warning (preferably with loud sirens) whenever someone selects the 3D effect for a chart? The challenge with data visualization is that soooooo many analysts: 1) are not inherently wired for effective visualization, and 2) wildly underestimate how important it is.
Mobile — I attended a session on mobile BI almost five years ago at a TDWI conference…and I still don’t see this as being a particularly hot topic. Even Campbell, with his mention of RFIDs, seems to think this is as much about new data sources as it is about reporting and analysis in a handheld environment.
Predictive Analytics — this has been the Holy Grail of BI for years. I don’t have enough exposure to enough companies who have successfully operationalized predictive analytics to speak with too much authority here. But, I’d bet good money that every company that is successful in this area has long since mastered the fundamentals of performance measurement. In other words, predictive analytics is the future, but too many businesses are thinking they can run (predictive analytics) before they crawl (performance measurement / KPIs / effective scorecards).
Composite Applications — this seems like a fancy way to say “user-controlled portals.” This really ties into the social networking (or at least Web 2.0), I think, in that a user’s ability to build a custom home page with “widgets” from different data sources that focus on what he/she truly views as important. Taking this a step farther — measuring the usage of those widgets — which ones are turned on, as well as which ones are drilled into — seems like a good way to assess whether what the corporate party line says is important is what line management is really using. There are some intriguing possibilities there as an extension of the “reports on the usage of reports” that gets bandied about any time a company starts coming to terms with report explosion in their BI (or web analytics) environment.
Cloud Computing — I actually had to go and look up the definition of cloud computing a couple of weeks ago after asking a co-worker who used the term if cloud computing and SaaS were the same thing (answer: SaaS is a subset of cloud computing…but probably the most dominant form). This is a must-have for the future of BI — as our lives become increasingly computerized, the days of a locally installed BI client are numbered. I regularly float between three different computers and two Blackberries…and lose patience when what I need to do is tied to only one machine.
Multitouch — think of the zoom in / zoom out capabilities of an iPhone. This, like mobile computing, doesn’t seem so much “hot” to me as somewhat futuristic. The best example of multitouch data exploration that I can think of is John King’s widely-mocked electoral maps on CNN (never did I miss Tim Russert and his handheld whiteboard more than when watching King on election night!). I get the theoretical possibilities…but we’ve got a long ways to go before there is truly a practical application of multitouch.

As I started with, there are a lot of exciting possibilities to consider here. I hope all of these topics are considered “hot” by BI vendors and BI practicitioners — making headway on just a few of them would get us off the plateau we’ve been on for the past few years.

Analytics Strategy

Data Management — As Sexy As a High Quality Mattress

Posted on July 1, 2009October 25, 2019 by Eric Peterson

Steve Woods of Eloqua invited me to write a guest post on his Digital Body Language blog after we’d gone back and forth a bit about contact data management and marketing automation. Over the past six or seven years, I’ve been thumped on the back of the ear with data management issues again and again. It always hurts, and, by the time I’ve realize I’ve got a mess…it’s a heckuva challenge to recover.

In my current job, I’m a full-time customer data management guy. It is not sexy. Like many large companies, we’ve got customer data that is created and managed in a wide range of disparate systems on diverse platforms, each with multiple decades of system evolution. It’s important. It’s painful.

There are some great opportunities in our increasingly electronic and e-based world to make some real headway with data management. In the case of the guest blog post, I focussed on opportunities to use marketing automation tools and your web site to drive improvements in the quality of your customer data. As for how exactly I made the “high quality mattress” analogy? Click on over and check out the post!

Analytics Strategy

Columbus Web Analytics Wednesday — July 2009 with Bizresearch

Posted on June 30, 2009October 25, 2019 by Eric Peterson

Web Analytics Wednesdays are an opportunity for full-time web analysts, part-time web analysts, and anyone who is interested in learning more about web analytics to get together and share their experiences! We will informally network for a bit before sitting down and ordering food, at which point we will have a brief presentation/discussion about Bizwatch led by Laura Thieme.

Details:

When: Wednesday, July 15th at 6:30 PM

Where: Barley’s Smokehouse and Brewpub, 1130 Dublin Road, Columbus, OH 43215

Registration: the Web Analytics Wednesday site

How to find us: We have a room reserved — just go to the back of Barley’s and hang a right

We are excited to welcome a new sponsor this month! Bizresearch will be co-sponsoring the event with the Web Analytics Wednesdays Global Sponsors. The sponsors will be covering food and nonalcoholic beverages only, although you are welcome (and encouraged) to sample Barley’s fine offering of frothy beverages on your own tab.

Laura Thieme, a 12-year search marketing and analytics veteran, has developed a new search analytics application: Bizwatch. Observing the challenges of monthly trend search marketing reporting and analysis, she developed a new application that combines SEO, competitors, keyword research, paid search and web analytics. It focuses on data integration amongst the three areas of search marketing. It focuses on trend analysis and keywords that convert.

Thieme is looking for feedback from industry colleagues on the search analytics application. She is also hoping to hear from search marketers regarding monthly reporting, applications they are using, and other search analytics data integration challenges they are experiencing.

It should be an engaging discussion!

Excel Tips, Presentation

Data Visualization that Is Colorblind-Friendly — Excel 2007?

Posted on June 18, 2009October 25, 2019 by Eric Peterson

Wow. This post started out not as a post, but as what I thought was going to be a 5-minute exercise with Google to download a colorblind-friendly palette for Excel charts. That was two weeks ago, and this post is just scratching the surface.

Several weeks ago, one of the presenters in a meeting showed some data as a map overlay. As soon as she projected the first map, someone in the meeting quipped, “Good luck understanding this one, Jim!” Jim, you see, is colorblind. And, apparently, most of the people in the meeting knew it. Approximately 8% of men have some form of color blindness (it’s much more rare in women — only 1 in 200). And the overlays on the map were color-coded very subtly. Jim commented that it was hopeless!

As it happened, I was exploring a fresh set of data that same week, as we’d recently rolled out some new customer data capture capabilities. As I worked through how best to present the results, I decided to grab a colorblind-friendly palette from the web and use it in the visualization of the information. I’d hoped to find a site with one or more Excel files that I could download with such a palette, but, worst case, I was prepared to snag a palette and manually update my Excel file (for future sharing on this blog, of course!).

No. Such. Luck!

What I did find was a slew of information on the different types of color blindness (which I’ll touch on briefly in a bit), as well as a bevy of almost-useful tools and palettes:

How to make figures and presentations that are friendly to Colorblind people — ultimately, I used the palette that is ~2/3 of the way down this page for my spreadsheet (the figure labeled “Set of colors that is unambiguous both to colorblinds and non-colorblinds”). Mr. Excel actually references this palette and provides a macro that will update a workbook’s palette with this palette. The downside of this palette is that, while it may be plenty functional, I can’t say I’m wild about it from an aesthetic viewpoint. But, I’d spent the 30 minutes I’d given myself to dig, so I ran with it.
Colorjack — a nifty tool for finding a color palette. Unfortunately…there’s no way to test how colorblind-friendly any of the palettes are
Colorblind Web Page Filter — there were a number of tools for sale that would simulate how content would appear to people with different forms of colorblindness, but this is the (free) online tool I wound up using for the exercise below. It couldn’t be easier to use — you just provide a URL and what form of color blindness you’re interested in, and it renders it

So, aside from the one palette that was solely focussed on functionality and not at all on aesthetics, I struck out. As I pondered this over the next few days, it occurred to me that, perhaps Excel’s default colors always seemed so gosh-awful because they were actually developed explicitly with colorblindness in mind. I could not find any documentation to support the theory…so I turned left and headed down that rathole to see if I could figure it out myself.

The exercise was pretty simple. I created a 10-color bar chart using the Excel 2007 default palette. Note: This was created purely for palette-testing — this actual chart is a great example of needlessly using more color than is needed! Here’s the chart:

Like the one colorblind-friendly palette I found online, I really don’t like the aesthetics of this palette. It’s been toned down a bit from the Excel 2003 (and earlier) versions, which is good, but it still seems rather harsh. Could that be for colorblind compatibility? I think so! I took the chart above and ran it through the Colorblind Web Page Filter mentioned above for the four most common types of color blindness (as described in a Pearson report by Betsy J. Case):

Excel 2007 Default Chart Colors -- Deuteranomaly (Affects 4.9% of Males) — Deuteranomaly (Affects 4.9% of Men)

Excel 2007 Default Chart Colors -- Deuteranopia (Affects 1.1% of Men) — Deuteranopia (Affects 1.1% of Men)

Excel 2007 Default Chart Colors -- Protanopia (Affects 1% of Men) — Protanopia (Affects 1% of Men)

Excel 2007 Default Chart Colors -- Protanomaly (Affects 1% of Men) — Protanomaly (Affects 1% of Men)

Overall, the palette seems workable in all four situations. The first three colors absolutely work. Color 4, as well as color 5, start to lose a little contrast from color 1, but they still seem manageable. Color 5 and color 7, as well as color 10, start to get a little problematic in some cases, but, if you’re going beyond four colors in a single chart, you might need to reconsider your chart type anyway. Right?

Now, one final test: for achromatopsia. On the one hand, this is extremely rare. On the other hand…it’s common when your office has a lot of black-and-white printers:

Excel 2007 Default Chart Colors -- Achromatopsia — Achromatopsia (Extremely Rare)

Apparently, any palette that works in grayscale is a quick way to check for compatibility with all forms of colorblindness. It’s also…a best practice. Interestingly, the Excel 2007 palette really lays an egg here, in that colors 1, 2, and 4 are all barely distinguishable!

Clearly, there is an opportunity here to test a variety of functional, attractive palettes for grayscale printability and the top four forms of colorblindness and develop something better than the Excel defaults. But, that’s an exercise for another time. I think I’ll aim for the first four colors of the palette being “highly distinguishable” in all scenarios and the next four being “functionally distinguishable.” What do you think? Would this be useful? What else should I take into consideration?

Analytics Strategy, General

The Truth About Mobile Analytics

Posted on June 11, 2009April 12, 2017 by Eric Peterson

Perhaps the only thing hotter than social media right now is mobile. And with good reason — smartphones like the iPhone and Palm Pre are taking our ability to get information to entirely new levels and ushering in an era of “digital ubiquity” that is clearly without precedent. Unsurprisingly business is responding by actively exploring how they can participate in the mobile opportunity, either by optimizing their site for small screens or going so far as to build cool, new iPhone applications to support long-standing offline initiatives.

Fortunately most business owners have learned from past mistakes and are showing interest in measuring the effect of their investment into mobile. But measuring mobile isn’t easy — the sheer diversity of technologies involved and the rapid evolution of the industry has created a monsterous landscape of devices, communication protocols, and requirements.

As a result dozens of companies have sprung up, all making claim to a unique ability to measure the mobile opportunity. Unfortunately some of these companies have decided that relying on hype, hyperbole, and sometimes outright lies are a better sales strategy than building a great product with a unique value proposition. We have seen CEOs bash other CEOs, sales people obfuscate their identity and try and provide “objective” answers, and antics that can only be described as “juvenile.”

Because the mobile opportunity is so great Analytics Demystified started taking a closer look at measurement earlier this year. I was fortunate enough to be able to rely on the expertise of folks like Michiel Berger and Thomas Pottjegort at Nedstat, the mobile team at NBC, dozens of analytics end-users, and some of the brightest product managers in the analytics sector tasked with integrating mobile into existing digital measurement offerings.

What I found was a series of surprising truths about how mobile analytics is evolving. Nedstat was kind enough to sponsor this research — and clear disclosure: Nedstat has been measuring and integrating mobile data into their web analytics offerings for years — and I am happy to announce the availablity of this research in a new white paper titled “The Truth about Mobile Analytics.”

You can download this paper from the Nedstat web site for free (but they do ask your name, email, and company name):

DOWNLOAD THE TRUTH ABOUT MOBILE ANALYTICS

We are also holding a special webcast on the subject on June 23rd at 10 AM Central European Time (CET) which is unfortunately quite late in the evening for those of us in the U.S. but quite well timed for Nedstat’s customers. I suspect the webcast will either be repeated or rebroadcast at a later date and time.

SIGN UP TO JOIN THE MOBILE ANALYTICS WEBCAST ON JUNE 23

Also, if you’re really into mobile and mobile analytics please consider joining us at the X Change Conference September 9, 10, and 11 in San Francisco. More details will be out next week but our mobile sessions will be led by Greg Dowling from Nokia (a company with some knowledge of mobile I am told.)

I encourage everyone to download the paper and give it a read, regardless of your position on mobile and mobile analytics today. As always I welcome your feedback and commentary.

Conferences/Community

X Change Keynote Announced

Posted on June 10, 2009April 11, 2017 by Eric Peterson

I am incredibly excited to announce the keynote presentation for X Change 2009 to be held September 9, 10, and 11 in San Francisco at the St. Regis hotel. This year to kick things off we have arranged to have four guys that have done more than anyone to define the web analytics industry join us for a special “Four Founder’s Perspective” session, moderated by yours truly.

Brett Crosby, Matt Cutler, John Pestana, and Bob Page are four names that every web analytics insider knows. Co-founders of Urchin, NetGenesis, Omniture, and Accrue respectively and now senior managers at Google Analytics, Visible Measures, ObservePoint, and Yahoo! Web Analytics, each of these gentlemen continue to shape digital measurement to this day.

In the keynote session we’ll be focusing on the past, present, and future of digital measurement. These guys were active participants in the early foundations of the industry — hell, Matt Cutler co-authored with Jim Sterne the seminal work Emetrics: Business Metrics for the New Economy back in 2000 which more or less kicked off the whole ball of wax — and all four have a history of participating in the early days of Emetrics in Santa Barbara (which is the model for the X Change, an intimate gathering of peers and friends.)

The audience will have a chance to ask questions.

Registration for X Change 2009 is now open and you will save 10% off the cost of registration if you sign up to join us before July 31st! I have more information about the X Change here in the “Community” section of the site and will be adding more content very soon!

I look forward to meeting many of you at the Founding Father’s keynote at X Change 2009!

Analytics Strategy, Social Media

Columbus Web Analytics Wednesday Meets #fiestamovement

Posted on June 4, 2009October 25, 2019 by Eric Peterson

Last night was the monthly Columbus Web Analytics Wednesday at Barley’s Smokehouse and Brewpub, and we were fortunate to have Webtrends sponsor for the second time this year! This time, we managed to get it scheduled in a way that lined up with Noé Garcia‘s travel plans, so he wore the dual crown of “Traveled Farthest to the Event” (from Portland, OR) and “Sponsor Representative.” The dual crown looked surprisingly like an empty beer glass:

Noe and Bryan Cristina of Nationwide co-facilitated a discussion about going beyond the application of web analytics tools within the confines of the tool itself. The most active discussion on that front was spawned by one of the regular participants in the group who works at a major, Columbus-based online retailer. Not necessarily this guy, but maybe it was him. My lips are sealed.

We talked about how web analytics data, tied to order information, and then matched back to offline marketing channels such as printed catalogs, can be very effective at driving marketing efficiency. In the examples that triggered the discussion, as well as from the other participants’ experiences, the consensus was that, while the ideal world would have all of this data hooked together automatically…rolling up your sleeves and tying the data together manually can still yield a substantial payback. Part of the discussion got into volume — for companies that do a lot of direct mail-oriented promotion, using web analytics data to cut the mail volume by even a fraction of a percent (by using that data to better target who does/does not respond to printed mail) can provide significant and quantifiable savings for a company.

I didn’t think I’d ever hear anyone at a WAW say “Zip+4” (that’s shorthand for the 5-digit zip code plus the four additional digits that you see on a lot of your mail)…other than me! But I did! The person who said that may or may not be a different person pictured in the photo above. Again…my lips are sealed!

And…Ford’s Fiesta Movement

Dave Culbertson, a WAW promotional channel unto himself, kicked off an entirely different, but equally intriguing discussion:

It all started as Dave was driving his Mazda in Grandview a couple of weeks ago. He got quasi-cut off by a 2011 Ford Fiesta two cars ahead of him. That prompted this tweet:

Now, Dave regularly mocks people who promote themselves as being social media gurus/experts/mavens…but he’s one of the most social media savvy marketers I know. He also knows his cars. For one of those reasons (or maybe both) he immediately recognized that the car in front of him was part of Ford’s Fiesta Movement so he nailed a very relevant hashtag with his tweet. As it happened, someone else on Twitter saw the tweet, quickly realized who the likely culprit was, tweeted to her, and she wound up apologizing via Twitter less than an hour after the incident!

Ms. Single Mama is a popular blogger, and this was the first time that she and Dave met in person. Everyone was curious about her Ford Fiesta agent experience. She obliged us by explaining, and, later, a good chunk of us headed out to the parking lot to see the 2011 Ford Fiesta she is driving for six months:

Yes, we had name tags. Yes, the intial group that followed Alaina out to look at her car was entirely male. Yes, all told, about twice as many people as this wound up checking out the car. And, finally, yes, Alaina made a call in the midst of this picture! Andrew (far left) commented that the dashboard looked like the head of a Transformer. He…was right!

Dave even demonstrated his social media hipness by snapping a picture of the vehicle with his iPhone and then tweeting it:

All in all, it was an engaging, informative evening. I’m sure I’ll miss some of the companies that were represented, but they included JPMorgan Chase, Nationwide, Victoria’s Secret Online, Webtrends, Clearsaleing, Bath&Body Works, Cardinal Solutions, Highlights for Children, Rosetta, Foresee Results, Acappella Limited, DK Business Consulting, Lightbulb Interactive…and others! Not. A. Bad. Crowd!

The next WAW will be July 15th. We’re working hard to get our calendar for the rest of the year nailed down, which means we are looking for sponsors and presenters. Please contact me at tim at <this domain> if you are interested on either front.

Conferences/Community

Davos, TED, X Change, …

Posted on May 25, 2009April 12, 2017 by Eric Peterson

Okay, so maybe the headline for this post is a wee hyperbolic, but if you’ve been to the X Change in the past I know you’ll forgive me my excitement. Yes, it’s that time of the year again, time to get ramped up for the X Change!

This year’s conference is being held at the extra fancy-schmancy St. Regis hotel in San Francisco, immediately adjacent to San Francisco MoMa and as central as you can possibly get while still suffering Starwood 5-star luxury accommodations. In a word, the venue is SWEET!

But, as with past X Change events, the venue will immediately become secondary to the excellent conversation, excellent company, and excellent insights being shared. As with the 2007 and 2008 events we plan to have the brightest practitioners from the best companies leading the conversation. Confirmed participants already include Best Buy, Intuit, Nokia, AOL, Forrester Research, Charles Schwab, Turner Broadcasting Systems, and more!

Also, as Gary alludes to in his post about the conference, I had a pretty good idea for this year’s conference keynote … we’re still pinning down details but I can honestly say the keynote this year is something that none of us have seen before at a web analytics conference or event.

We’re also excited to announce that on September 9th we will be holding the first-ever X Change Think Tank training day! Credit Gary this one, and it makes perfect sense to me given the strength of the Semphonic crew, but we will be taking the ideals of the X Change and extending them to an extremely intimate learning environment. I will be leading two classes and I hope to get my new business partner Aurélie Pols to lead one or two as well!

If you have budget for training in 2009 I definitely encourage you to have a look at the Think Tank and feel free to ping me directly for more details.

One of the things I love the most about the X Change is the transparency we have and that we learn from our participants. Every attendee helps us make the X Change a better conference, every year! To this end I am actively seeking input about the conference via this site, Twitter, email, … heck, you can call me directly if you have a good idea!

You can register now for the 2009 X Change and will save 10% if you do so before July 31st! Head on over to the Semphonic web site and start the registration process — and don’t forget this is an event that has sold out every year it has been offered! Because we limit the conference to 100 participants we fully expect to sell out in advance again … don’t get caught waiting!

I hope to see you at the X Change!

Analytics Strategy

The Teeter-Totter of Customer Data Management

Posted on May 18, 2009October 25, 2019 by Eric Peterson

I had a professor in business school who used to explain the relationship between the stock market and the bond market as a teeter-totter (in rural southeast Texas, I grew up knowing this as a see-saw): as the yields on one went up, the yields on the other went down and vice versa.

Managing your customer data can be like that, too — the more of a burden you put on your customers and prospects to keep your data about them clean, the less of a burden you put on yourself. And, likewise, the more of a burden you take on yourself, the less of a burden you’re putting on your customer.

While bouncing through links from a tweet, I stumbled across Steve Woods’s original Contact Washing Machine post, and it set some alarm bells off. Steve’s a damn sharp guy — he was a co-founder and remains the CTO of Eloqua, and he is pretty much an undisputed visionary when it comes to marketing automation technology. Yet, this post sparked an immediate reaction, as well as teeter-totter imagery. Since then, Steve has clarified…and I think I misread his initial premise. His point is that data cleansing should happen as early in the data acquisition process as possible — cleanse the data as it comes in, rather than crossing your fingers and waiting to run batch processes after the fact in the hopes that the data will get cleaned up.

That’s a valid point, but, after digging deeper into the cross-links in the post, I still think there’s some under-estimating of what it takes to “fix” dirty data as it comes in. For starters, when it comes to customer/prospect data, there are typically a range of incoming data entry points:

Web Data Entry

In the world o’ the web, data can come into your systems directly as typed by a visitor to your site — when a user is filling out a web form, for instance. On the surface, that’s a great place to do data validation, because you’ve got the actual user right there to clarify anything that has gone amiss. If he’s fat-fingered his phone number or put in an e-mail address that is clearly not valid, it’s best to prompt him right then and there to correct the mistake. But, the teeter-totter comes into play: if that piece of data is really not germaine (as perceived by the user), it doesn’t take long for your cleansing to lead to a frustrated visitor to your. Worse, if you don’t allow the user to bypass the validation step (with a “I don’t care what you think, I’ve entered the information correctly, so just keep it that way and let me move on” option), there is a very good chance that you will keep some visitors from ever getting to where they and you want them to!

If you include field validation on your web forms, and if you don’t allow the user to override that validation, it behooves you to include detailed form abandonment tracking in your web analytics to make sure you haven’t set up an insurmountable barrier for some of your customers.

Human Data Entry

Call centers almost always serve a data entry function as part of the customer service process. In addition, many companies have dedicated data entry staff to translate mail, fax, tradeshow-collected leads, or other transactions. This can be a great opportunity to clean your data up front, as you can certainly place a higher burden of getting the data right and enforced data validation on employees of your own company than you can on your customers and prospects.

BUT, this turns out to be a stickier wicket than it seems at first blush. If I had a nickel for every time I heard someone living in world of backend data propose data augmentation or enhancement by updating the human data entry processes to “just add one more quick step,” I’d be able to buy a Starbucks Venti Caramel Frapuccino^® blended coffee (which is a lot of nickels, if you think about it). Two reasons that there should be a proceed-with-extreme-caution label placed prominently on any solution that heads down this path:

Call centers typically live and die by the average handle time (AHT) for their calls; yes, they want to meet the customer’s needs, but they also, out of necessity, can save big dollars by cutting the AHT by a few seconds on average. Adding 5 or 10 seconds to every call can have a very real impact (and can make you some quick enemies with call center managers)
It’s easy to identify the benefits of more, more complete, or cleaner data…when it comes to backend processes and data analysis. But, is that benefit readily evident to the people whom you’re relying on to capture it? Does it benefit them directly, either through smoothing the immediate next steps in their process or by impacting their compensation? Due to the high-volume nature of call center and data entry work, data that is “just another field you need to fill out” is data that is at risk of falling prey to shortcuts (the first value in the dropdown, “aaa” in a text field, etc.). The most successful introductions of process changes have a net-no-change or net decrease in the number of steps/time/complexity of the process into which it is being introduced.

Human data entry offers opportunities to get data that is more complete and cleaner…but those opportunities don’t come automatically.

There are many other ways that data can enter your systems: provided by an intermediary (often semi-independent sales channels: distributors, resellers, etc.), sourced from a third-party lead sourcing company, passed in from another system within your company (often a system that doesn’t store the data in the same format or even have the same definitions for what specific fields mean and are used for), etc. There’s value in inspecting the sources of your customer data, assessing how clean the data is that comes from those different sources, and then, with the teeter-totter firmly in mind, investigating where and how to get that data coming in cleaner!

Photo courtesy of jhirtz.

Analytics Strategy, General

Demystifying Europe …

Posted on May 13, 2009April 8, 2017 by Eric Peterson

When I quit my job at Visual Sciences back in May 2007 to form Analytics Demystified I did so because I had a vision of a new type of web analytics consulting group. I very much wanted to build a small practice made up of very senior people capable of solving the really hard problems most companies have after they’ve made the investment in web analytic technology. I wanted to establish a firm that would compliment the highly tactical firms that I respected so much — companies like Semphonic, Stratigent, and Europe’s OX2.

After two years I am very proud of the work I’ve done and the clients I’ve worked with. I have had the opportunity to work with some of the best brands, the best companies, and the most visionary management teams who are actively wokring to do more than simply “run reports” and instead want to actively compete on web analytics. That said, I have come to the realization that there is no way I could satisfy the global need on my own … so I did what every good business owner should do: I went out and got someone smarter, more eloquent, and better looking to be my business partner!

At Emetrics last week in San Jose I was incredibly excited to announce that Aurélie Pols, Europe’s most widely known and well respected web analytics consultant, has joined Analytics Demystified as a Principal Consultant. Aurélie brings depth and experience in web analytics that is rare anywhere in the world and exceedingly rare in Europe, she was the first consultant to break the “one vendor” stranglehold in Europe that forced firms to work exclusively with a single technology, and she brings a brilliance to the explanation and use of these tools that amazes even me.

Now Aurelie and I will be working together in Europe to “demystify web analytics” and help companies make significantly better use of their technology investment. Between the two of us and our contacts across Europe Analytics Demystified will now be providing a far greater level of service than was previously possible.

I highly recommend that you read Aurélie‘s “Hello, World” blog post and start following her at aurelie.analyticsdemystified.com. If you have any questions about Aurélie’s practice or how Analytics Demystified can help you regardless of where you’re located, please don’t hesitate to contact us directly.

I hope you will welcome me in welcoming Aurélie to the Analytics Demystified team.

Analysis, Reporting

What is "Analysis?"

Posted on May 5, 2009October 25, 2019 by Eric Peterson

Stephen Few had a recent post, Can Computers Analyze Data?, that started: “Since ‘business analytics’ has come into vogue, like all newly popular technologies, everyone is talking about it but few are defining what it is.” Few’s post was largely a riff off of an article by Merv Adrian on the BeyeNETWORK: Today’s ‘Analytic Applications’ — Misnamed and Mistargeted. Few takes issue (rightly so), with Adrian’s implied definition of the terms “analysis” and “analytics.” Adrian outlines some fair criticisms of BI tool vendors, but Few’s beef regarding his definitions are justified.

Few defines data analysis as “what we do to make sense of data.” I actually think that is a bit too broad, but I agree with him that analysis, by definition, requires human beings.

With data “coming into vogue,” it’s hard to walk through a Marketing department without hearing references to “data mining” and “analytics.” Given the marketing departments I tend to walk through, and given what I know of their overall data maturity, this is often analogous to someone filling the ice cube trays in their freezer with water and speaking about it in terms of the third law of thermodynamics.

I’ve got a 3-year-old daughter, and it’s through her that I’ve discovered the Fancy Nancy series of books, in which the main character likes to be elegant and sophisticated well beyond her single-digit age. She regularly uses a word and then qualifies it as “that’s a fancy way to say…” a simpler word. For instance, she notes that “perplexed” is a fancy word for “mixed up.”

“Analytics” is a Fancy Nancy word. “Web analytics” is a wild misnomer. Most web analysts will tell you there’s a lot of work to do with just basic web site measurement. And, that work is seldom what I would consider “analytics.” As cliché as it is, you can think about data usage as a pyramid, with metrics forming the foundation and analysis (and analytics) being built on top of them.

There are two main types of data usage:

Metrics / Reporting — this is the foundation of using data effectively; it’s the way you assess whether you are meeting your objectives and achieving meaningful outcomes. Key Performance Indicators (KPIs) live squarely in the world of metrics (KPIs are a fancy way to say “meaningful metrics”). Avinash Kaushik defines KPIs brilliantly: “Measures that help you understand how you are doing against your objectives.” Metrics are backward-looking. They answer the question: “Did I achieve what I set out to do?” They are assessed against targets that were set long before the latest report was pulled. Without metrics, analysis is meaningless.
Analysis — analysis is all about hypothesis testing. The key with analysis is that you must have a clear objective, you must have clearly articulated hypotheses, and, unless you are simply looking to throw time and money away, you must validate that the analysis will lead to different future actions based on different possible outcomes. Analysis tends to be backward looking as well — asking questions, “Why did that happen?”…but with the expectation that, once you understand why something happened, you will take different future actions using the knowledge.

So, what about “analytics?” I asked that question of the manager of a very successful business intelligence department some years back. Her take has always resonated with me: “analytics” are forward-looking and are explicitly intended to be predictive. So, in my pyramid view, analytics is at the top of the structure — it’s “advanced analysis,” in many ways. While analysis may be performed by anyone with a spreadsheet, and hypotheses can be tested using basic charts and graphs, analytics gets into a more rigorous statistical world: more complex analysis that requires more sophisticated techniques, often using larger data sets and looking for results that are much more subtle. AND, using those results, in many cases, to build a predictive model that is truly forward-looking.

The key is that the foundation of your business (whether it’s the entire company, or just your department, or even just your own individual role) is your vision. From your vision comes your strategy. From your strategy come your objectives and your tactics. If you’re looking to use data, the best place to start is with those objectives — how can you measure whether you are meeting them, and, with the measures you settle on, what is the threshold whereby you would consider that you achieved your objective? Attempting to do any analysis (much less analytics!) before really nailing down a solid foundation of objectives-oriented metrics is like trying to build a pyramid from the top down. It won’t work.

Adobe Analytics, Conferences/Community, General

Are You Coming to Emetrics?

Posted on April 29, 2009April 12, 2017 by Eric Peterson

It’s almost amazing to consider that it has been a full year since the last Emetrics “West” event in California — what with so many changes and little Luca Dechamps Otamendi turning one — but it is again time to gather together and bask in the glory of Mr. Sterne’s excellent event. I am again honored to be presenting to a combined track, this time on Wednesday, May 6th at 11:00 AM, and will be giving an update of my “Competing on Web Analytics” presentation that resonates so well with, well, pretty much everyone who has seen it.

The update is important and stems from a bunch of research I have been doing for the past six months. Given the launch of Yahoo Web Analytics 9.5 today and the recent opening up of the Google Analytics APIs I am busier than ever talking with companies who are trying to find the “right” balance of technology, people, and process.

Also, as I do from time to time I have a really big announcement that I will be making at the beginning of my talk. Last time I quit my job at Visual Sciences to start Analytics Demystified … this time? Come to the talk and be the first to find out!

I hope you’ll drop by and see my talk, again: Wednesday, May 6th at 11:00 AM.

I am also speaking briefly in the “Softer Side of Metrics” panel with Mr. Stephen “Recently Elected to the WAA Board” Hamel and folks from BT Buckets and Firefox on Thursday, May 7th at 11:00 AM. This should be fun since I’ll get to introduce the larger web analytics community to the work I have been doing with Twitalyzer.

Also, don’t forget about the Emetrics edition of Web Analytics Wednesday which is, as always, open to conference attendees and the local community alike. We have something special planned to honor our recently deceased colleague Hosam Elkhodary so I hope you’ll sign up (so we can get a good count) and join us at the Fairmont Hotel in San Jose.

Finally, as always I go to Emetrics to meet with as many people as I possibly can and operate under the “I can sleep when I get home” mentality. If you’ve read my books, read my blog, enjoy Twitalyzer, or just have always wanted to ask me something please feel free to reach out … literally if you see me passing by or by Twittering me at @erictpeterson and setting up a time to meet.

(If you can’t make it to San Jose the next big analytics event in the U.S. is the X Change Conference September 9, 10, and 11 in San Francisco. I’m a huge fan (and partner) in the X Change so I’d love to tell you more about it if you’re interested!)

I hope to see you in San Jose!

General

Recovery.gov Needs Some Few and Some Tufte

Posted on April 27, 2009October 25, 2019 by Eric Peterson

I caught an NPR story about recovery.gov last week, and it sounded really promising. Depending on where you fall on the political spectrum, the various rounds of stimulus and bailout funding that have come through over the past six months fall somewhere between “throwing money away,” “ready, fire, aim,” and “point in what seems what might be a good direction, pull the finger, and shoot.” No one can stand up and say, with 100% certainty, that we’re not going to look back on this approach in a decade or two and say, “Um…oops?”

It’s hard to imagine anyone taking issue with the proclaimed intent of recovery.gov, though — make the process as transparent as possible, including how much money is going where, when it’s going, and what ultimately comes of it. It was a day or two before I found myself at a computer with time to check out the site…and I was disappointed. In the NPR interview, the interviewer commented how the site was slick and clean. Reality is “not so much.”

Now, I did once take a run at downloading the federal budget to try to scratch a curiousity itch regarding, at a macro level, where the federal government allocates its funds. On the one hand, I was pleased that I was able to find a .csv file with a sea of data that I could easily download and open with Excel. On the other hand, the budget is incredibly complex, and it takes someone with a deeper understanding of our government to really translate that sea of data into the answers I was looking for. Really, though, that wasn’t a surprise:

The data is ALWAYS more complex than you would like…when you’re trying to answer a specific question.

To the credit of recovery.gov, they clearly intended to show some high-level charts that would answer some of the more common questions citizens are asking. Unfortunately, it looks like they turned over the exercise to a web designer who had no experience in data visualization.

Examples from the featured area on the home page:

recovery.gov Funds Distribution Reported by Week

The overall dark/inverse style itself I won’t knock too much (althought it bothers me). And, the fact that the gridlines are kept to a minimum is definitely a good thing. My main beef is admittedly a bit ticky-tack. There was an earlier version where there was a $30 B gridline, and that has since been removed — that gridline clearly showed the “30.5 B point” being below the midway point between 20 B and 40 B. Clearly, someone would have to really be scrutinizing the graph to identify this hiccup, but someone will.

When presenting data to an audience, the data as it stands alone needs to be rock solid. If it contradicts itself, even in a minor way, it risks having its overall credibility questioned.

So, moving on to some more egregious examples:

recover.gov Relief for America's Working Families

We get a triple-whammy with this one:

Pie charts are inherently difficult for the human brain to interpret accurately
Pie charts are even worse when they are “tilted” to give a 3D effect — the wedges on the right and left get “shrunk” while wedges on the top or bottom get “stretched”
Exploding a pie chart and then providing a pie chart of just the wedge…just ain’t good

Two questions this visualization might have been trying to answer:

How much of the stimulus plan is devoted to tax benefits?
How much of the stimulus plan is going to the “Making Work Pay” tax credit?

Without doing any math, can you estimate either one of these? For the first question, you’re estimating the size of the small wedge on the left pie chart. It looks like it’s ~ 1/4 of the pie, doesn’t it? In reality, it’s 37%! For the second question, you have to combine your first estimate with an estimate of the lavender wedge in the right pie chart…and that’s way more work than it’s worth. If you do the math, you’ll get that the lavender wedge works out to ~7% of the entire left pie. A simple table or a bar graph would be more effective.

And, finally, the estimated distribution of Highway Infrastructure Funds:

recovery.gov Distribution of Highway Infrastructure Funding

Well, that’s just silly. There is NO value of making these bars come flying out of the graph. Really.

Now, to the site’s credit, it takes all of 3 clicks to get from the home page to downloading .csv files with department-specific data and weekly updates (which includes human-entered context as to major activities during the prior week). That’s good (assuming it’s not unduly cumbersome to maintain)! And, I’m sure the site will continue to evolve. But, I’d love to see them bring in some data visualization expertise. The model for the visualization should be pretty simple:

Identify the questions that citizens are asking about the stimulus money
Present the data in the way that answers those questions most effectively
Link to the underlying data — the aggregate and the detail — directly from each visualization

As it turns out, Edward Tufte has already been engaged (thanks to Peter Couvares for that tip via Twitter), and is doing some pro bono work. But, it’s not clear that he’s focussing on the high-level stuff. I would love to see Stephen Few get involved as well — pro bono or not! Or, hell, I’d offer my services…but might as well get the Top Dog for something like this.

Starting today, the site is hosting a weeklong online dialogue to engage the public, potential recipients, solution providers, and state, local and tribal partners about how to make Recovery.gov better. I’ve submitted a couple of ideas already!

General

Interview on Social Media and Analytics

Posted on April 26, 2009April 12, 2017 by Eric Peterson

I have done hundreds of interviews with all kinds of media in my years in web analytics. Some of these interviews have turned out well, some less well, but rarely do I get to participate in a conversation about analytics that afterwards I think “Phew, that was cool.”

A few weeks ago I got to do exactly that thanks to Brent Leary at CRM Essentials.

If you have a few minutes and want to hear my recent thoughts on a variety of subjects including getting started in analytics, the impact of analytics on social media, and the work I’ve done recently on Twitalyzer, please take the time to listen to this interview.

Brent is a totally engaging interviewer and he pushed the conversation along in unexpected ways. I have been getting tons of good feedback already but, as always, I welcome your thoughts and comments.

Conferences/Community

Columbus Web Analytics Wednesday: A Speedy April

Posted on April 25, 2009October 25, 2019 by Eric Peterson

We had our monthly Web Analytics Wednesday meetup at Barley’s Smokehouse and Brewpub last week. Once again, the Web Analytics Wednesday Global Sponsors (Coremetrics, Analytics Demystified, and SiteSpect) sponsored the event, which is always appreciated!

This month, in lieu of a formal topic, Dave Culbertson facilitated a round of speed networking — like speed dating, but with the purpose of driving interaction beyond everyone’s immediate tablemates. Each round lasted for 1 minute, and the main challenge was getting people to stop talking and shift on to the next person! It was a little intense, but Dave cut it off after 15-20 minutes, and the overwhelming consensus was that it was fun and useful!

At the end of the exercise, Dave commented that he really hoped we could start extending these 1:1 connections and interactions through social media. As it is, Dave (@daveculbertson) is one of the most interesting people I follow on Twitter, especially when it comes to finding and tweeting links to content that I find interesting and informative. We’d actually thought ahead (if “six hours before the event” counts as “ahead”) and made a sign-up sheet that included a space for the attendees to write their Twitter usernames and indicate if it would be okay to post them. I then proceeded to leave the sign-in sheet behind when I left! Something about Barley’s — last month, I left my notebook behind and had to go and retrieve it the next day (2 beers over 2.5 hours plus a full meal…in case you’re wondering — it’s just something in the air there!).

So, instead, we’re broadening our social media presence. Consider joining one or all of the following, depending on where/how you hang out on the ‘net:

Facebook — we’ve had a WAW Columbus group there for some time
Twitter Group — this was Dave’s suggestion, and I haven’t used twittgroup.com before, but we’ve now got a cbuswaw group there as well
LinkedIn — might as well kick it old school, too, so we’ve now got a Columbus WAW LinkedIn group

Pick your poison, one or all!

Overall, the event had a great mix of both practicing web analysts (from companies like Resource Interactive, Highlights for Children, Victoria’s Secret, Lightbulb Interactive, Coldwell Banker, …and I’m just rattling off the companies I can remember, so this is an incomplete list) as well as some web analytics-centric companies: BizResearch, ClearSaleing, SearchSpring, and WebTech Analytics (all the way up from Cincinnati!). And, with a handful of sharp people in the crowd who are currently looking for full-time work, it was great that TeamBuilder Search came out as well! From a quick count of faces in my brain, the attendance broke down to be ~25% first-timers, ~25% loooonnnngg-time attendees, and 50% who have attended 1-5 times before. All in all, a great mix!

The most-interesting-but-random site/tool that I learned about this month was City-Data.com — think The World Factbook, but for U.S. cities rather than for countries! And, with a slew of charts that are pretty clean and provide a pretty good way to get the flavor of a town — weather, jobs, houses, and so on.

Analytics Strategy, General

Is Your Attribution Model Appropriate?

Posted on April 20, 2009 by Eric Peterson

Recently I have spent an awful lot of time thinking about and talking about data accuracy issues in the field of web analytics. The widespread use of cookies as a tracking mechanism and the underlying assumption that “one cookie = one visitor” is a big part of the problem, but cookies are not the only problem. Another problem, one that I actually believe to be more substantial than cookies and visitors, is the challenge of campaign attribution.

Challenge? What’s hard about campaign attribution? You tag campaigns and web analytics tells you what works, right? You get pretty ROI graphs and click-reports and all that fun stuff? Campaign analytics is easy!

Wrong.

One of the best-kept secrets in online marketing is that most campaign attribution data is completely wrong and the models used to evaluate campaign performance are wholly inappropriate. The relative nascence of digital marketing practices, combined with conflicting measurement systems and poorly understood interaction between online marketing channels, likely means that hundreds of millions of dollars are wasted annually on marketing efforts that don’t produce their intended results.

Companies are increasingly responding to this observation by re-examining their marketing measurement systems. Even the most cursory analysis yields a great deal of information about the “campaign attribution problem.” Popularized recently by Microsoft with their “Engagement Mapping” efforts as well as analysis published by Forrester Research and others, it is clear that the most widely used online campaign attribution model is inherently flawed.

To correct these flaws and begin to improve both the accuracy of measurement and the general understanding of how marketing really works online, Analytics Demystified recommends a new approach to campaign analysis. Dubbed “Appropriate Attribution”, the approach leverages widely available but infrequently used data to triangulate towards the true value of online marketing efforts.

Given that the majority of online advertisers have direct response goals, and that most marketers are still generally unsatisfied with the campaign measurement tools at their disposal, Analytics Demystified believes that Appropriate Attribution is the first step towards improving companies’ collective understanding of their digital marketing efforts.

Eventually marketers will have access to robust warehouses of data detailing consumer interaction with online media and advertising, but the adage “you must walk before you can run” is as true in digital marketing as it is in life. Before business owners and marketers become fully equipped to benefit from complex marketing mix analysis of online and offline channels, they are well advised to address the campaign attribution problem to increase the return on their valuable dollars spent for online marketing efforts.

Thanks to the fine folks at Coremetrics you can read all about Appropriate Attribution and learn how you can start to get a better understanding of your online marketing efforts today.

Download your copy of the Appropriate Attribution paper from Coremetrics today.

General, Presentation

PowerPoint / Presentations / Data Visualization

Posted on April 16, 2009October 25, 2019 by Eric Peterson

I wrote a post last week about PowerPoint and how easy it is to use it carelessly — to just open it up and start dumping in a bunch of thoughts and then rearranging the slides. That post wound up being, largely, a big, fat nod to Garr Reynolds / Presentation Zen. Since then, I’ve been getting hit right and left with schtuff that’s had me thinking more broadly about effective communication of information in a business environment:

The Intermediate Stats class I’ve been taking (taught by an Ohio State professor on-site at my company, Nationwide…as in Nationwide: Car Insurance) is wrapping up, and the last day of the class includes a group presentation
Jon Peltier really knocked one out of the park on the PTS Blog with his re-working of a weird, radial chart showing oil prices over time
Jim Knight wrote Will Death by PowerPoint Soon be a Thing of the Past, where he summarized the highlights of a one day Presentation // Reboot workshop put on by Garr Reynolds and Nancy Duarte
Knight’s post introduced me to Duarte (and her blog), who, according to Knight, is “most famous for developing the design of Al Gore’s Academy Award-winning An Inconvenient Truth presentation“
In Duarte’s latest post — a video entry while she is on vacation in Hawaii — she reviews Advanced Presentations by Design: Creating Communication that Drives Action by Andrew Abela, which sounds like it is a bit dry, but makes a strong case for putting a lot more rigor behind developing a presentation

Put all of those together, and I’ve got a mental convergence of PowerPoint usage, presenting effectively (which goes well beyond “the deck”), and data visualization. These are all components of “effective communication” — the story, the content, how the content is displayed, how the content is talked to. In one of Reynolds’s sets of sample slides, you can clearly see the convergence of data visualization and PowerPoint. And, even he admits that this is a tricky thing to post…because it removes overall context for the content and it removes the presenter. Clearly, there are lots of resources out there that lay out fundamental best practices for effectively communicating in a presentation-style format. Three interrelated challenges, though:

The importance of learning these fundamentals is wildly undervalued — it sounds like Abela’s book tries to quantify this value through tangible examples…but it’s a niche book that, I suspect, will not get widely read by the people who would most benefit from reading it
“I need to put together a presentation for <tomorrow>/<Friday>/<next week>” — we’re living under enormous time pressure, and it’s incredibly easy to get caught up in “delivering a substantive deliverable” rather than “effectively communicating the information.” When I think about the number of presentations that I’ve developed and delivered over the past 15 years, the percentage that were truly effective, compelling, and engaging is abysmally small. And that’s a waste.
Culture/expectations — every company has its own culture and norms. For many companies, the norms regarding presentations are that they are linear, slide-heavy, logically compiled, and mechanically delivered affairs. For recurring meetings, there is often the “template we use every month” whereby the structure is pre-defined, and each subsequent presentation is an update to the skeleton from the prior meeting. Walk into one of those meetings and deliver a truly rich, meaningful, presentation…and your liable to be shuttled off for a mandatory drug test, followed by a dressing down about “lack of proper preparation” because the slides were not sufficiently text/fact/content-heavy. <sigh>

What’s interesting to me is that I have spent a lot of time and energy boning up on my data visualization skills over the past few years. And, even if it takes me an extra 5-10 minutes in Excel, I never send out something that doesn’t have data viz best practices applied to some extent. As you would expect, applying those best practices is getting easier and faster with repetition and practice. So, can I do the same for presentations? And, again, that’s presentations-the-whole-enchilada, rather than presentations-the-PowerPoint-deck. Can I balance that with cultural norms — gently pushing the envelope rather than making a radical break? Can you? Should you?

Analytics Strategy

Columbus Web Analytics Wednesday: April 22, 2009

Posted on April 15, 2009October 25, 2019 by Eric Peterson

In the interest of not messing with a good thing, we’re returning to Barley’s Smokehouse and Brewpub this month for our regular gathering of full-time, part-time, and just-generally-interested web analyst types.

We had a great turnout last month, and we’re on pace to match that this month, which means we’re needing to go easy on the Web Analytics Wednesday Global Sponsors. Rather than asking them to cover the full bill, we’re just having them cover the food and having everyone be on their own for beverages, which is still a wickedly good deal!

We’ve gotten feedback in the past that it’s good to have every second or third meetup be presentation-free, and this month will be one of those. However, we are looking into doing a little post-dinner speed networking — it’ll be quick, and we’ll find out whether it works or not. The inimitable Dave Culbertson of Lightbulb Interactive is running point on that and has been noodling around as to the best approach. It should be fun!

The details:

When: Wednesday, April 22nd at 6:30 PM

Where: Barley’s Smokehouse and Brewpub (1130 Dublin Road, Columbus, OH)

RSVP: http://tinyurl.com/AprilWAW

I hope to see you there!

And, if you, or anyone you know, would be interested in sponsoring a future Columbus Web Analytics Wednesday, please drop me a line at tim at gilliganondata.com. Our sponsorship flexibility is unparalleled in the industry — think rhythmic gymnastics meets Reed Richards. It’s a great way to get high visibility in a group of elite local marketing professionals. It’s a great way to support Columbus as a hotbed of web analytics thought leadership. It’s good karma.

Analytics Strategy

Hosam Elkhodary

Posted on April 10, 2009April 12, 2017 by Eric Peterson

UPDATE: There is information at the bottom of this post about how to make a donation in Hosam’s memory from June Li in the Web Analytics Forum.

On Tuesday of this week the web analytics community lost a passionate advocate with the passing of Hosam Elkhodary. I had the pleasure of working with Hosam just after founding Analytics Demystified as well as spending time with him at many an Emetrics. There are many blog posts about Hosam out there but the most touching is Mike Sukmanowsky’s — Hosam clearly had the same impact on Mike as he did on many of us. I encourage you to read Mike’s post and comment there if you knew Hosam.

Hosam will be missed.

FROM THE WEB ANALYTICS FORUM:

Some additional information for those who are interested in contributing to the Heart and Stroke Foundation fund for Hosam. You can donate online here:

http://www.heartandstroke.on.ca/site/c.pvI3IeNWJwE/b.3581623/k.C08D/Donate.htm

In addition designating that the contribution is in memoriam of Hosam, please make sure you address the card as follows, which will doubly ensure the donation is directed properly:

Abdalla Elkhodary
7 Delaney Drive
Ajax, Ontario
L1T 4B2

General

PowerPoint the Application vs. the Application of PowerPoint

Posted on April 8, 2009October 25, 2019 by Eric Peterson

Slightly off-topic for this blog, and a little dated, but worth sharing nonetheless.

During a discussion with a couple of my co-workers today, I made an observation about how my current company, as well as one of the major consulting firms we use, seem to really be in love with PowerPoint as the documentation/presentation/communication/general-purpose tool of choice. This prompted an immediate and emphatic response from one of those co-workers, who insisted that he “loved PowerPoint.”

The exchange reminded me of the news last year that Katsuaki Watanabe, the President and CEO of Toyota, had decreed that employees stop using PowerPoint for the creation of documents. Garr Reynolds (aka, Presentation Zen…master), had a great take on the news. A couple of the highlights:

To be clear, Watanabe did not “ban PowerPoint use,” as was mis-circulated at the time
Watanabe did severely discourage the use of PowerPoint as a documentation tool — Reynolds calls these “slideuments” (slides + documents), which is a wickedly apt designation (and the core of this post)
“…visuals projected on a screen in support of a live talk are very different from material that is to be printed and read and/or analyzed.”

And, a longer excerpt that is also key:

…there is often no distinction made between documents (slideuments made in PowerPoint) and presentation slides prepared for projection. They are often interchangeable. Sounds efficient, right? And it would be funny if it was not so inefficient, wasteful, and unproductive. The slideuments produced in Japan make understanding and precision harder when printed, and when used for projected slides in a darkened conference room, they are the country’s number one cure for insomnia.

This was fundamentally the distinction that I was trying to get my co-worker to understand…without much luck. He’s clearly got some PowerPoint chops — he kept pulling up different slides he had done that had intricate builds and snazzy palettes and templates. But, the slides he was most proud of were heavily laden with annotations and text — they were standalone, comprehensive pictorial representations of complex concepts or systems.

Once he let loose with, “The point of PowerPoint is not the retention of the information — it’s the ‘wow’ factor,” I admitted defeat.

The title of this post is really the gist of my thesis here: Powerpoint the application is not the same thing as the application of PowerPoint. All too often, we don’t make that distinction. As Reynolds puts it, “Slideware is not a method, it’s simply a kind of tool.”

Think of a sledgehammer. It’s a tool — an application, if you will. But, it can applied for vastly different purposes:

Used with a wedge to split firewood
Used to drive a metal fencepost into the ground
Used to prop open a door that keeps blowing shut

These are very different applications of the tool, and you would be clear as to what it was you were trying to accomplish when you hiked it over your shoulder and headed off to the task at hand.

It's not what the software does...
[Cartoon by Hugh MacLeod — see oodles more at gapingvoid.com]

The same holds true for PowerPoint. It has several different distinct possible uses…and it’s worth being clear as to which one you are tackling:

A live, in-person demonstration — think simple, minimalist visual backup that supports an engaging presenter without distracting from what he/she is saying; think Steve Jobs (and, if you’re not familiar, check out one of the Presentation Zen posts on that subject)
A live, online presentation via a webinar or web conferencing solution — this is a stickier wicket, in a lot of ways; it’s tempting to hedge against technology quirks by distributing the .ppt/.pptx file to all of the attendees via e-mail so they can simply pull up the deck and follow along, but this can be problematic, as the audience can then jump ahead and jump back. Generally, this sort of presentation is “the best alternative we have” when, ideally, you’d be doing a live, in-person demonstration. I would think this means the same minimalist approach described in the prior bullet would apply.
Documentation never intended to be presented — slideuments — these really are problematic and should be avoided.

All too often, there is a blurring of all three of these: a live presentation for some people, while other people are participating remotely, and the “presenter” has distributed the presentation as a handout that has all of the detail that he/she is going to present. That leaves the participants cognitively vascillating between listening to the presenter’s words and reading through the detail in the presentation that is either being projected or is printed in front of them. It’s just not effective. Make the presentation a presentation. If there is supplemental detail or review material, put that in a document and distribute it separately — before, during, or after the presentation. Let the presentation be truly visual and let it support the concepts and information being presented, with an emphasis on the concepts.

Aside: To bridge back to the topic of this site, I’ve even seen PowerPoint used as a poor man’s BI presentation tool: PowerPoint 2007 linked to Excel 2007, which was in turn linked to Access 2007, which was in turn hooked into a SQL Server database. On the one hand…<shudder>. On the other hand, when it came to a portable (once the link to Excel was removed), shareable report, it wasn’t half bad! (Our intent was for it to also be a prototype that we could iterate on quickly as we developed requirements for a true BI tool…but that didn’t pan out for other reasons.)

So, that’s my mini-rant. It’s a problem. A clear problem. But, not one that I intend to solve. If you’re interested in thinking more about the topic check out:

Presentation Zen (obviously)
Laura Fitton / Pistachio Consulting — you can just look at her posts that have the presentation tag
For the militant/extreme death-to-PowerPoint take, there’s the inimitable Edward Tufte…but he really does go a bit overboard

Analytics Strategy

40 Million Reasons Your Customer Data Isn't As Current as You Think (or Hope)

Posted on April 3, 2009October 25, 2019 by Eric Peterson

While not getting as much buzz as social media when it comes to hot topics for in 2009, “customer data management” is something that marketers are starting to take seriously. It’s easy to start envisioning fancy pictures of capturing and using customer data:

Using behavioral data to drive timely and relevant emails
Integrating information across different customer touchpoints/channels to deduce customers’ and prospects’ preferred communications medium
Building analytic models to predict which customers are most likely to churn and making special offers to retain them

Those are all admirable goals. And, they’re all attainable. AND, they’re all going to be expected baseline capabilities within five years.

Before you tackle these higher order applications, it’s worth grounding yourself in an understanding of how rapidly customer data decays. Here are a couple of fun facts to wrap your head around on that front:

The U.S. Postal Service processes over 40 million address changes annually [source]
The population of the United States is estimated as being just north of 300 million [source]

Clearly, this isn’t an apples-to-apples comparison. But, we tend to imagine that our customers and prospects are more static than, in reality, they are — who they work for, what their job title is, and, yes, even where they live.

Conferences/Community

WAA Board Election: Don't Forget to Vote!

Posted on April 1, 2009April 12, 2017 by Eric Peterson

I finally had a chance to look at the fine group of folks running for Web Analytics Association Board of Directors in this go-around and I have to say I am mighty impressed! Not that the WAA doesn’t already have an amazing group of Directors, but wow, some serious contenders each with a ton of experience in the sector running this time around. If you haven’t already voted, have a look at the consultants, practitioners, and vendors running in this year’s election.

This is actually a unique election given that none of the existing Directors who have the ability to run again have chosen to do so. The few I have talked to about this have cited “time” as the major factor but no doubt the WAA will miss April Wilson’s passion, Neil Mason and Laura Paxia’s experience and European perspective, and Seth Romanow’s historical knowledge of the Association. Thankfully Jim Sterne will still be serving as Chairman of the Board (right Jim?)

I really like the approach the WAA took this time interviewing each candidate and putting the text and recording on the Association web site. It certainly helps us learn a little more about each candidate which will make a difficult vote a little easier. I also like that some candidates are actively campaigning in the Web Analytics Forum, in Twitter, at Emetrics, etc. This more than anything emphasizes the importance of these Board positions — and of the Association in general.

If there is any one question I would have posed to the candidates that did not get asked it would be this:

“If there was one thing you could go back in time and change about the web analytics industry, what would it be?”

Who knows, maybe some of the folks running for WAA Directorships will take the time to answer the question. I know what my answer would be 😉

Best of luck to EVERYONE running in this election!

Excel Tips, Presentation

Data Visualization — March Madness Style

Posted on March 28, 2009October 25, 2019 by Eric Peterson

I got an e-mail last week just a few hours into Round 1 of this year’s NCAA men’s basketball tournament. The subject of the email was simply “dumb graph,” and the key line in the note was:

The “game flow” graph…how in the WORLD is that telling me anything? That the score goes up as the game goes on? Really? Ya think?

My friend was referring to the diagrams that ESPN.com is providing for every game in the tournament. The concept of these graphs is pretty simple: plot the score for each team over the course of the game. For instance, the “Game Flow” graph for the Oklahoma vs. Morgan State game looks like this (you can see the actual graph on the game recap page — just scroll down a bit and it’s on the right):

Oklahoma vs. Morgan State

This isn’t an exact replication, but it’s pretty close — best I could manage in Excel 2007 (the raw data is courtesy of the ESPN.com play-by-play page for the game). ESPN’s graph is a Flash-based chart, so it’s got some interactivity that the image above does not (we’ll get to that in a bit).

The graph shows that the game was tight for the first 4-5 minutes, then Oklahoma pulled away, Morgan State made it really close mid-way through the first half, and then Oklahoma pulled away and never looked back. My friend had a point, though — the dominant feature of the graph is that both lines trend up and to the right…and any chart of a basketball game is going to exhibit that pattern (actually, the play-by-play for that game has a couple of hiccups such that, when I originally pulled the data, I had a couple places where the score went down due to out-of-sequence free throw placement…but I noticed the issue and fixed it). In business, we’re pretty well conditioned to see “up and to the right” as a good thing…but it’s meaningless in the case of a basketball game.

Compare that graph to a game that was much closer — the Clemson vs. Michigan game (the graph on ESPN’s site is on the recap page, and the raw data is on the play-by-play page):

Clemson vs. Michigan

This was a tighter game all through the first half. Clemson led for the first 7-8 minutes, Michigan pulled substantially ahead early in the second half, and then things got tight in the last few minutes of the game. But, again, both lines moved up and to the right.

These charts are not difficult to interpret:

The line on top is the team that is leading
The distance between the lines is the size of the lead
The lines crossing signifies a lead change

But, could we do better? Well, my wife and kids are out-of-town for the week (spring break), I have the social life you’d expect from someone who blogs about data and data visualization, and the fridge is well-stocked with beer. Party. ON!

At best, my level of basketball fan-ness hovers right around “casual.” Still, I follow it enough to know the key factors of a game update or game upset (Think: “Hey, Joe. What’s the score?”). Basically:

Who’s winning?
By how much?

(If there’s time for a third data point, the actual score is an indication of whether it’s a high scoring shootout or a low scoring defense-oriented game.)

Given these two factors as the key measures of a game, take another look at the graphs above. When the game is tight, you have to look closely to assess who is winning. And, determining how much they’re winning by requires some mental exertion (try it yourself: look back at the last graph and ask yourself how much Michigan was winning by halfway through the second half).

This is just begging for a Stephen Few-style exercise to see if I can do better.

First, the Oklahoma/Morgan State game:

Oklahoma vs. Morgan State

Rather than plotting both team’s scores, with the total score on the Y-axis, this chart plots a single line with the size of the lead — whichever side of the “0” line the plot is on is the team that is winning. The team on the top is the higher seed, and the team on the bottom is the lower seed. I added the actual score at halftime and the end of the game, as well as each team’s seed. Compare that chart to the much closer Clemson/Michigan game:

Clemson vs. Michigan

The chart looks very different — focussing on what information fans really want and presenting it directly, rather than presenting the data in a way that requires mental exertion to derive what the fan is really interested in: who’s winning and by how much? While the graphs on ESPN’s site allow you to mouse over any point in the game and see the exact score and the exact amount of time remaining, it’s hard to imagine who would actually care to do that — better to come up with an information-rich and easy-to-interpret static chart than to get fancy with unnecessary interactivity.

A few other subtle changes to the alternative representation:

I tried to dramatically increase the “data-pixel ratio” (Few’s principle that the ratio of actual data to decoration should be maximized) — this is a little unfair to ESPN, as their site is working with an overall style and palette for the site, but it’s still worth keeping in mind
I used color on the Y-axis to show which team’s lead is above/below the mid-line. The numbers below the middle horizontal line are actually negative numbers, but with a little Excel trickery, I was able to remove the “-” and change the color of the labels (all done through Custom number formatting)
By putting the top seed on the top, looking at a full page of these charts would quickly highlight the games that were upsets

I’m my own worst critic, so here are two things I don’t like about the alternate charts above:

The overall palette still feels a little clunky — the main data plot doesn’t seem to “pop” as much as it should, even though it’s black, and the shaded heading doesn’t feel right
While the interpretation of the data requires less mental effort once you understand what the chart is showing, it does seem like this approach requires another half-second of interpretation upr front that the original charts don’t require

What do you think? What else could I try to improve the representation?

General

Wanted: Senior Analyst with Marketing Chops for a 4-6 Month Contract

Posted on March 27, 2009October 25, 2019 by Eric Peterson

I got pinged this morning by a colleague who is looking for someone with marketing chops (preferably B2B) and fairly advanced analytic skills for a 4-6 month engagement. The engagement would require heavy travel on the front end, and the main deliverable is a model that the client can use to assess the effectiveness of various marketing programs when it comes to driving service renewals.

I don’t have much more detail than that, but if this piques your interest, drop me a line at tim at gilliganondata dot com and I’ll put you in touch with someone who can provide more information.

Analytics Strategy, Conferences/Community, General

Unique Visitors ONLY Come in One Size

Posted on March 22, 2009April 12, 2017 by Eric Peterson

Back in January I published a note about the proposed IAB Audience Reach Measurement Guidelines that generated a fair amount of interest. At the time I applauded the IAB for providing guidance regarding the definition of a “unique user” or “unique visitor” while noting some concerns about how the proposed definition would actually manifest. In summary, the new IAB definition of “unique visitor” needed to have some basis in underlying data that is based on secondary research that can be directly tied to “a person.” Now that the IAB Audience Reach Measurement Guidelines have been officially published we can use the IAB’s own words:

“… in order to report a Unique User, the measurement organization must utilitze in its identification and attribution processes underlying data that is, at least in reasonable proportion, attributed directly to a person” and “In no instance may a census measurement organization report Unique Users purely through algorithms or modeling that is not at least partially traceable to information obtained directly from people, as opposed to browsers, computers, or any other non-human element.” (Section 1.2.4)

The last little bit references, I believe, the IAB’s distinction of four types of unique “countables” — Unique Cookies (Section 1.2.1), Unique Browsers (1.2.2), Unique Devices (1.2.3) and Unique Users or Unique Visitors (1.2.4). The term “measurement organization” was a little, well, mystifying as was evidenced in my January post, and sadly the final document does little to clarify this term other than to say the “document is principally applicable to Internet Publishers, Ad-serving organizations, Syndicated Measurement Organizations and auditors” on the IAB web site.

This definition is important since in my last post the real conundrum appeared to be that if “measurement organization” included Omniture, WebTrends, Google, Coremetrics, etc. then the IAB was essentially saying that the vendors needed to change the way they reported Unique Visitors, at least for their clients who would be subject to the perview of the IAB and MRC. What’s more, George Ivey from MRC never got back to my repeated requests for information, despite two members of the IAB working group (Josh Chasin from comScore and Pete Black from BPA Worldwide) openly disagreeing in their interpretation of the definition …

Well, a few weeks back I got a call from Joe Laszlo, an old co-worker of mine at JupiterResearch who is now the IAB’s Director for Analytics, the guy basically responsible for the document. I always liked Joe and it was nice to hear from him again. And Joe did clarify for me what a “measurement organization” is … he just didn’t directly clarify the impact on web analytics vendors.

According to Joe (and he will surely correct me publicly if I am misinterpreting our conversation) the “measurement organizations” that should be guided by this new definition of “Unique Users” are publishing organizations who are outwardly reporting their metrics for consideration by advertisers in the open market. Companies like AOL, Weather.com, ESPN, etc. This is, I think, much more clear than the sentence a few paragraphs up that includes “Syndicated Measurement Organizations and auditors” and puts at least this part of the document in context: Essentially when using numbers coming from census-based systems, the IAB and MRC want publishers to start reporting Unique Visitor counts that have some basis in reality.

Pretty hard to disagree with Joe and the IAB on that point. We all pretty much agree that cookie-based visitor counting is messed up, and I think we can even agree that the degree to which these counts are “messed up” is a function of the target audience, the duration under examination, and the type of site. For example, we expect cookie-based counts on sites that attract highly technical users on a daily basis to be much more impacted over a 90-day measurement period than, say, sites that attract largely non-technical users on a monthly basis over the same 90-day period.

So I’ll make one really bold statement right now, the kind that I have a tendency to regret but hey, it’s Monday and I’m feeling pretty good about the coming week:

The IAB are to be applauded for taking such a bold stand on the subject of counting and reporting unique visitors based on what we traditionally consider “web analytic” data.

I said as much in my last post … right after I said that the likelihood of the web analytics vendors following these recommendations was about the same as everyone waking up tomorrow to realize that the financial meltdown was a bad dream and the Dow is still over 14,000 (zero). The team of folks that the IAB brought together, which I understand included both Omniture and WebTrends, should be congratulated for taking a firm stand on one of the most dogged issues plaguing our collective industries (web analytics, online advertising, online publishing, syndicated research, etc.) for at least the past five years.

It is about time that we all agreed that “Unique Visitor” reports coming from census-based technologies frequently have no basis in reality. Further, we should all admit that cookie deletion, cookie blocking, multiple computers, multiple devices, etc. have enough potential to distort the numbers as to render the resulting numbers useless when used to quantify the number of human beings visiting a site or property.

Yes, before you grieve on me with your “but they are probably directionally correct” response I agree with you, they probably are, but fundamentally I believe that advertising buyers are at least as interested in the raw numbers as they are the direction they are moving. I say “probably are” because if you’re not taking the IAB’s advice and reconciling census-based data with data derived directly from people, well, you’re never sure if that change in direction is because your audience is changing, technology is changing, or there is a real and substantial increase or decline.

I mentioned above that my conversation with Joe didn’t really clarify the impact on web analytics vendors under the IAB’s new definition. Since I spent a fair amount of time thinking about the IAB guideline’s impact in this regard, I will make another bigger and bolder statement:

Starting immediately, I think the web analytics vendors and any company reporting a cookie-based count that is not in compliance with the IAB’s definition of “Unique Visitor” should stop calling said metric “Unique Visitors (or Users)” and correctly rename the metric “Unique Cookies”.

Yep, I am 100% in favor of using the IAB’s new terminology and being semantically precise whenever possible. The “Unique Visitor” counts in the popular web analytics applications are always actually counting cookies and so we should just go ahead and say that explicitly by calling them “Unique Cookies”. This change would actually give the web analytics vendors a neat opportunity … to battle to be the first to have a real “Unique Visitor” count that is based, as the IAB has suggested, on underlying data that is, at least in reasonable proportion, attributed directly to a person.

How could they do this? Let me count the ways:

Develop a standard practice around the use of log-in and registered user data
Work with third-party partners who are focused on gathering more qualitative data (for example, Voice of Customer vendors like ForeSee Results)
Work with third-party partners who are estimating cookie-deletion rates, or at least have the potential to (for example, Quantcast)
Work with third-party partners who can actually calculate cookie-deletion and multiple-machine use rates with some accuracy (for example, comScore, Google, Yahoo!)

I’m sure there are a few ways I am not thinking of, but these are the big four that have been talked about since 2005. While I expect to get some grief from paying clients about this statement, and I fully expect my suggestion to be widely ignored by the vendor community (no offense taken), I think this change would be a big step towards the recognition that there is only ONE DEFINITION of a “Unique Visitor” and this definition is only tangentially related to the number of cookies being passed around.

Like Soylent Green(TM), “Unique Visitors” are PEOPLE and our industry will go a long way towards maturation when we collectively agree on this fundamental truth. It is not to say that Unique Cookies is not a valuable count — hell, in the absence of a strategy for reconciling cookies against people-based data unique cookies are all we have. But I do not believe that after nearly 15 years we are doing the online measurement community any justice by plugging our ears and signing “LA LA LA LA I CANNOT HEAR YOU GO AWAY!!!!!”

Which brings me to my last point …

I was really, really bummed out to read Jodi McDermott’s MediaPost article titled “Unique Visitors Come in Two Shapes and Sizes.” I was bummed because I have always liked Jodi since we worked together at Visual Sciences, because I think she is a brilliant member of our community, and because I knew I was going to end up writing these words … Jodi’s thesis is wrong and does the web analytics community a dis-service in attempting to defend a mistake by asking to water down a good definition just because it isn’t “hers” (in quotes since Jodi is a member of a larger committee charged with defining standards within the WAA.)

From Jodi’s article (which I recommend you read, especially the comments, and the emphasis is mine):

“Bravo to the IAB for forcing the issue with audience measurement companies to standardize the way that they report uniques, but from a Web analyst’s perspective — and as a member of the WAA Standards committee — I wish they would have not allowed the term “unique visitors” to be redefined in such a way as to allow for multiple definitions in the space. Web analysts and media planners today have a hard enough time trying to figure out which data source to use and which standard to apply when performing their job — but that issue is now compounded even more by multiple definitions of unique visitors. In defense of the IAB, its membership is comprised of some heavy-hitter companies who are not about to change that “tab” in their reporting UI that says “Unique Visitors” on it. But in defense of WAA individual and company members, which include vendors such as Omniture and WebTrends (who were both listed as “Project Participants” on the IAB document, interestingly enough), neither are we. The term will live on in both places.”

I think what Jodi has missed here is that the IAB has actually given the world a useful and more accurate definition of “Unique Visitors” than any used in the web analytics industry today. More importantly, given the relative weight, clout, and respect enjoyed by the IAB in the wider world, I don’t think their definition allows for “multiple definitions” … I rather think that over time the IAB expects their member companies, especially those who want to have their numbers audited and publicly used, will consider the IAB definition the definition of “Unique Visitors” and properly consider the term we web analysts widely use today to be “Unique Cookies.”

I’m not sure what Jodi means by “heavy-hitter companies who are not about to change their “tab”” since I’m aware of very few companies today that have implemented the IAB recommendation for practical and ongoing use. But I was incredulous when I read the statement regarding using the IAB’s new definition, “in defense of the WAA individual and company members, which include vendors such as Omniture and WebTrends, neither are we. The term will live on in both places.”

Seriously? Rather than start calling our cookie counts “Unique Cookies” and having a rational conversation with our bosses to explain that the technology we use is limited in its ability to discern real people, you prefer to throw down the gauntlet with the IAB and say “screw your definition?” Despite the criticism that has been both wrongly and rightly heaped on the WAA’s “standard” definitions, despite the considerable group that crafted the IAB’s definitions, and considering the fact that the WAA’s definition is wrong, you want to pick a fight?

Two wrongs never make a right, and you’re wrong twice here. Sorry.

I am not on the WAA Standards Committee, I am not on the WAA Board of Directors, and my dues with the WAA are about to lapse so I have no basis for representing the organization. Perhaps reading more into Jodi’s post given my knowledge of her passionate work in the WAA, but I would strongly encourage the current Board of Directors to examine Jodi’s statements in the context of the IAB relationship and the “bigger picture” at play. Because while Jodi may speak for the WAA Standards Committee and by extension the entire WAA, she certainly does not speak for me.

I will gladly use the term “Unique Cookies” when I am talking about a cookie-based count and reserve the term “Unique Visitors” for those situations where I have some basis for doing so. More importantly I will encourge my clients and vendor friends to consider doing same. The IAB has given the entire measurement community a reason to take a huge leap forward and gain clarity around one of our most important metrics. To turn our back on this opportuntity because it will necessitate change, require additional explanation, or because “we like our definition better” is wrong, wrong, wrong.

Harrrrumph.

I suspect like previous posts on the subject this will generate some conversation. As usual I do not pretend to have all the answers and I welcome your feedback. I am, unfortunately, traveling all day Monday and will have limited ability to approve and respond to comments but I promise to do so as quickly as possible.

Analytics Strategy

A Record-Setting Web Analytics Wednesday in Columbus

Posted on March 19, 2009October 25, 2019 by Eric Peterson

It turns out, as I’ve been looking at my records (read: old blog posts), that this month’s Columbus Web Analytics Wednesday marked our one year anniversary, which means it was our 13th WAW. As we were discussing it last night, we thought next month was the one-year mark, but, in a post from last May, I referenced that gathering as our third, and the Internet just doesn’t lie!

It’s only taken a year, but we just might have hit upon the perfect spot: Barley’s Smokehouse and Brewpub. For a nominal fee, we were able to reserve a room, which gave us some real volume control when we got into the group discussion portion of the evening, and there’s plenty of room for future expansion. As it was, the final headcount was 28 people, which was right around 50% higher than we’ve ever had before. Credit for that goes to:

WebTrends, our sponsor for the evening — we finally got some folk to come out who have been meaning to in the past, but haven’t quite made it; some of those people are WebTrends user; unfortunately, due to a scheduling conflict, Noe Garcia, the WebTrends account executive who supported our sponsorship request, wasn’t able to make the trip from Portland for the event.
Webtrends, who provided the evening’s speaker/topic — John Defoe, VP of Solution Services and A Bunch of Other Stuff, kicked off a discussion about using web analytics data outside of the web analytics environment; more on that in a bit.
The Amazing Blanqueras — we had some repeats who discovered us through past WAW promotions on Columbus Tech Life Meetup site, and we again brought in some fresh faces from the site; as Columbus Tech Life grows, so will WAW!
Dave Culbertson — ‘nary a WAW goes by that someone isn’t there because Dave ran into them and encouraged them to attend
Twitter — a number of people tweeted about the event, but my unofficial observations put Jenny Wells of TeamBuilder Search as the lead tweeter on that front

Other than that, we ran our usual gamut of promotions, and,presumably,picked up people through those channels as well. I’m sure we picked up a person or two who was Googling Monish Datta and wound up on this site (I’m up in the top 5 results for a Google search for Monish — I don’t think I’m ever going to overtake his LinkedIn profile). Feel free to take a crack about a WAW blog post not having definitive data on traffic sources…

As to the topic, John kicked off the discussion by sharing some examples of WebTrends customers who are using web analytics data beyond the web analytics environment:

A motorcycle manufacturer who uses web analytics data to score leads (site visitors) before passing them on to their dealers for follow-up — giving the dealers a prioritized list of who is more likely to be ready to buy
A media site that uses web analytics data to do an hourly refresh of the “most popular articles” on its home page (which led to a $2 million uplift in ad revenue, if I heard correctly) — I’ve always wondered how much that sort of functionality gets hit by a feedback loop (an article just barely cracks the “most popular” list, but, then, by being on the list, it gets more clicks and remains there), but I didn’t get a chance to ask
A company that uses web analytics data for targeted re-marketing via e-mail — identifying what content a person has viewed and using that to tailor e-mails promoting the same or similar products

�

John used those examples as a way to launch discussions of where others are using web analytics data outside of the web analytics environment:

I chimed in with my experiences with using web analytics data for lead scoring that combines web activity data with CRM information and then pushes the lead score into the CRM system
Scott Zakrajsek briefly explained out Victoria’s Secret uses web analytics data for targeted e-mail re-marketing
Bryan Cristina shared a Nationwide: Car Insurance example…that I totally missed (um…they’re my employer; you’d think I would’ve paid more attention there!)

The wrap-up thoughts, I think, could be summarized as follows:

Soooooo many companies aren’t even trying to do any of these sorts of things today
It won’t be long before these sorts of uses of web analytics data will be a must-have rather than a cutting-edge differentiation opportunity
It sounds easy enough, but, when you get down to it, getting different systems to really talk to each other (or to build a layer to pass information back and forth between them in a meaningful way) takes some roll-up-your-sleeves hard work and the tenacity to stick with it until it works
Having an engaged executive sponsor is darn near a must to pull these off
Having someone driving the project who really, really “gets it” makes things go a lot smoother…but outsourcing is a viable option

If you’re interested in learning more, it’s not too late to book a trip to Vegas for WebTrends Engage ’09! John’s got a whole session on this basic subject on Wednesday, April 8th.

If you’re interested in sponsoring a future Web Analytics Wednesday, drop me a line at tim at gilliganondata.com!

Presentation

Data Visualization — Few's Examples

Posted on March 14, 2009October 25, 2019 by Eric Peterson

I attended a United Way meeting last week that was hosted at an overburdened county government agency site in south Columbus. The gist of the meeting was discussing the bleakness of the economy and what that could or should mean to the work of the committee. The head of the government agency did a brief presentation on what the agency does and what they are seeing, and the presentation included the distribution of a packet of charts with data the agency tracks.

I was struck by how absolutely horridly the information was presented. A note at the bottom of each chart indicated that the same staff member had compiled each chart. Yet, there was absolutely no consistency from one chart to the next: the color palette changed from chart to chart (and none of the palettes were particularly good), a 3-D effect was used on some charts and not others (3-D effects are always bad, so I suppose I’d rather inconsistency than having 3-D effects on every chart), and totally different chart types were used to present similar information. On several of the bar charts, each bar was a different color, which made for an extremely distracting visualization of the information.

I glanced around the room and saw that most of the other committee members had furrowed brows as they studied the information. It occurred to me that there was an undue amount of mental exertion going on to understand what was being presented that would have been better spent thinking through the implications of the information.

Ineffective presentation of data can significantly mute the value of a fundamentally useful report or analysis.

Later that evening, I found myself popping around the web — ordering my own copy of Stephen Few’s Show Me the Numbers, and, later, poking around on Few’s site. Specifically, I spent some time on his Examples page, browsing through the myriad before/after examples that clearly illustrate how the same information, presented with the same amount of effort, but using some basic common principles, dramatically reduce the mental effort required to understand what is going on.

It’s a fascinating collection of examples. And Show Me the Numbers is a seminal book on the topic.

Analytics Strategy

Columbus Web Analytics Wednesday: March 18, 2009

Posted on March 8, 2009October 25, 2019 by Eric Peterson

We’re a week later than maybe would’ve been ideal, but our sponsors are coming all the way from Portland, Oregon, and I expect it to be worth the wait! WebTrends will be sponsoring the event, and Noé Garcia (Strategic Account Executive) and John Defoe (VP of Solution Services) will be attending in person. John will be presenting and facilitating the topic for the evening: “Web analytics data beyond the web analytics platform.” This is a hot topic in a lot circles and includes, in my mind:

Integrating user-level web behavior into a CRM system to provide Sales with greater insight into the interests of specific prospects and customers (this is standard Eloqua functionality…but don’t read that as me thinking Eloqua offers a remotely robust web analytics solution)
Using user-level web behavior to score leads (in conjunction with non-web analytics data) to improve the lead qualification and lead nurturing process
Closing the loop and using web analytics insights to dynamically drive relevant web content — something that lots of people talk and wave their hands about…but that is extremely hard to actually build in a way that really works

I’m sure there are other aspects of this topic. I, for one, am looking forward to hearing John’s thoughts on the subject.

As to the details:

When: Wednesday, March 18th at 6:30 PM

Where: Barley’s Smokehouse and Brewpub (1130 Dublin Road, Columbus, OH)

RSVP: http://tinyurl.com/Mar09WAW

I’m looking forward to the new venue. We continue to struggle to find a place that has a suitably decent food and drink menu, is suitably centrally located, and is suitably non-noisy for us to be able to handle a presentation and discussion. Dave Culbertson, one of the co-organizers of the event, suggested Barley’s, and two other organizers responded enthusiastically (with the e-mail equivalent of a V8-style self-applied palm to the forehead), so I’m optimistic.

I’m hoping we have a good turnout!

Analysis, Analytics Strategy, Excel Tips, General, Presentation, Reporting

The Best Little Book on Data

Posted on March 5, 2009October 25, 2019 by Eric Peterson

How’s that for a book title? Would it pique your interest? Would you download it and read it? Do you have friends or co-workers who would be interested in it?

Why am I asking?

Because it doesn’t exist. Yet. Call it a working title for a project I’ve been kicking around in my head for a couple of years. In a lot of ways, this blog has been and continues to be a way for me to jot down and try out ideas to include in the book. This is my first stab at trying to capture a real structure, though.

The Best Little Book on Data

In my mind, the book will be a quick, easy read — as entertaining as a greased pig loose at a black-tie political fundraiser — but will really hammer home some key concepts around how to use data effectively. If I’m lucky, I’ll talk a cartoonist into some pen-and-ink, one-panel chucklers to sprinkle throughout it. I’ll come up with some sort of theme that will tie the chapter titles together — “myths” would be good…except that means every title is basically a negative of the subject; “Commandments” could work…but I’m too inherently politically correct to really be comfortable with biblical overtones; an “…In which our hero…” style (the “hero” being the reader, I guess?). Obviously, I need to work that out.

First cut at the structure:

Introduction — who this book is for; in a nutshell, it’s targeted at anyone in business who knows they have a lot of data, who knows they need to be using that data…but who wants some practical tips and concepts as to how to actually go about doing just that.
Chapter 1: Start with the Data…If You Want to Guarantee Failure — it’s tempting to think that, to use data effectively, the first thing you should do is go out and query/pull the data that you’re interested in. That’s a great way to get lost in spreadsheets and emerge hours (or days!) later with some charts that are, at best, interesting but not actionable, and, at worst, not even interesting.
Chapter 2: Metrics vs. Analysis — providing some real clarity regarding the fundamentally different ways to “use data.” Metrics are for performance measurement and monitoring — they are all about the “what” and are tied to objectives and targets. Analysis is all about the “why” — it’s exploratory and needs to be hypothesis driven. Operational data is a third way, but not really covered in the book, so probably described here just to complete the framework.
Chapter 3: Objective Clarity — a deeper dive into setting up metrics/performance measurement, and how to start with being clear as to the objectives for what’s being measured, going from there to identifying metrics (direct measures combined with proxy measures), establishing targets for the metrics (and why, “I can’t set one until I’ve tracked it for a while” is a total copout), and validating the framework
Chapter 4: When “The Metric Went Up” Doesn’t Mean a Gosh Darn Thing — another chapter on metrics/performance measuremen. A discussion of the temptation to over-interpret time-based performance metrics. If a key metric is higher this month than last month…it doesn’t necessarily mean things are improving. This includes a high-level discussion of “signal vs. noise,” an illustration of how easy it is to get lulled into believing something is “good” or “bad” when it’s really “inconclusive,” and some techniques for avoiding this pitfall (such as using simple, rudimentary control limits to frame trend data).
Chapter 5: Remember the Scientific Method? — a deeper dive on analysis and how it needs to be hypothesis-driven…but with the twist that you should validate that the results will be actionable just by assessing the hypothesis before actually pulling data and conducting the analysis
Chapter 6: Data Visualization Matters — largely, a summary/highlights of the stellar work that Stephen Few has done (and, since he built on Tufte’s work, I’m sure there would be some level of homage to him as well). This will include a discussion of how graphic designers tend to not be wired to think about data and analysis, while highly data-oriented people tend to fall short when it comes to visual talent. Yet…to really deliver useful information, these have to come together. And, of course, illustrative before/after examples.
Chapter 7: Microsoft Excel…and Why BI Vendors Hate It — the BI industry has tried to equate MS Excel with “spreadmarts” and, by extension, deride any company that is relying heavily on Excel for reporting and/or analysis as being wildly early on the maturity curve when it comes to using data. This chapter will blow some holes in that…while also providing guidance on when/where/how BI tools are needed (I don’t know where data warehousing will fit in — this chapter, a new chapter, or not at all). This chapter would also reference some freely downloadable spreadsheets with examples, macros, and instructions for customizing an Excel implementation to do some of the data visualization work that Excel can do…but doesn’t default to. Hmmm… JT? Miriam? I’m seeing myself snooping for some help from the experts on these!
Chapter 8: Your Data is Dirty. Get Over It. — CRM data, ERP data, web analytics data, it doesn’t matter what kind of data. It’s always dirtier than the people who haven’t really drilled down into it assume. It’s really easy to get hung up on this when you start digging into it…and that’s a good way to waste a lot of effort. Which isn’t to say that some understanding of data gaps and shortcomings isn’t important.
Chapter 9: Web Analytics — I’m not sure exactly where this fits, but it feels like it would be a mistake to not provide at least a basic overview of web analytics, pitfalls (which really go to not applying the core concepts already covered, but web analytics tools make it easy to forget them), and maybe even providing some thoughts on social media measurement.
Chapter 10: A Collection of Data Cliches and Myths — This may actually be more of an appendix, but it’s worth sharing the cliches that are wrong and myths that are worth filing away, I think: “the myth of the step function” (unrealistic expectations), “the myth that people are cows” (might put this in the web analytics section), “if you can’t measure it, don’t do it” (and why that’s just plain silliness)
Chapter 11: Bringing It All Together — I assume there will be such a chapter, but I’m going to have to rely on nailing the theme and the overall structure before I know how it will shake out.

What do you think? What’s missing? Which of these remind you of anecdotes in your own experience (haven’t you always dreamed of being included in the Acknowledgments section of a book? Even if it’s a free eBook?)? What topic(s) are you most interested in? Back to the questions I opened this post with — would you be interested in reading this book, and do you have friends or co-workers who would be interested? Or, am I just imagining that this would fill a gap that many businesses are struggling with?

Adobe Analytics, Analytics Strategy, General, Social Media

Omniture, Europe, SAS, WebTrends, and Twitter!

Posted on March 4, 2009April 12, 2017 by Eric Peterson

You may be wondering “What do those things have in common?” You may also be wondering “Did Eric drop off the face of the Earth?” The answer to the first question is the explanation to the second …

Despite changes in Analytics Demystified’s client portfolio–changes that I believe accurately reflect the current economic climate–we are busier than ever here in Portland, Oregon. Or rather not in Portland, Oregon as Q1 2009 has me bouncing around the globe to talk about web analytics, something I enjoy tremendously.

World Tour 2009 (Part I) got started a few weeks back at the Omniture Summit in Salt Lake City, Utah. If you haven’t been to an Omniture Summit, assuming you are an Omniture, WebSideStory, Visual Sciences, Instadia, Mercado, Offermatica … am I forgetting anyone?! … I definitely recommend attending if you have the chance. Aside from excellent production and plenty of attention to detail I felt like Omniture did a great job on the content, something they took some criticism for in years past. The break-out sessions I saw paired an Omniture employee with a customer, analyst, or industry leader and in general the result was informative without being overly sales-y.

Perhaps the thing I enjoyed the most was that, despite my occasional open criticism of Omniture and some of their practices, senior management seemed (or at least pretended) to be happy enough to see me. I had a wonderful conversation with President of Sales Chris Harrington, spent some time with Gail Ennis and John Mellor, and even got to share Swedish Fish with Brett Error (who is now in Twitter @bretterror) Even Josh James and I had a chance to catch up … but no, I didn’t hug it out with Matt Belkin 😉

The World Tour continues here in Portland, then off to Milan, Madrid, and Washington, D.C. Locally I am excited to get to present at SearchFest 2009, but I have to admit I’m somewhat more excited about my first trip to Milan, Italy for Web Analytics Strategies 2009 and my first return to Madrid in several years. Perhaps most excitedly, following a special presentation with MV Consultoria, I will get to meet Rene and Aurelie’s new baby Lucca! After a brief return home (to spend time reading with my five year old daughter who has recently adopted her dad’s great love for reading) I fly to D.C. to deliver a keynote presentation at the SAS Global Forum.

And that is just the beginning. You can see the complete schedule under “Consulting” at Analytics Demystified, and I am actively booking conferences and presentations in June and July.

Which brings me to Twitter …

I wouldn’t say I was an early adopter of Twitter, not by a long shot. I actually met co-founder Biz Stone in Rotterdam and admitted “No, I don’t really understand the service …” I was eventually goaded into trying Twitter by Aaron Gray of WebTrends and started seeing the inherent value after getting people to use the #wa hashtag to identify web analytics (and Washington State) related content.

Of course, if you know me, you know I was unlikely to stop there …

After a short beta test with something I called the “Twitter Influence Calculator”, last week I rolled out The Twitalyzer. With tongue-in-cheek I have described the service as “Google Analytics for Twitter” and by all measures the service has taken off. To date nearly 20,000 unique Twitter users have tried the service which summarizes your use of Twitter and provides a handful of interesting measures of success (influence, generosity, velocity, clout, and the signal-to-noise ratio.)

Rather than spend a bunch of time telling you about it I encourage you to check it out at http://twitalyzer.com

While I have been incredibly busy between these travels, client work, writing proposals, and messing with Twitter I am of course always happy to hear from readers. Send email, Twitter me (@erictpeterson), or look for me at one of the conferences above!

Analysis, Reporting

Performance Measurement vs. Analysis

Posted on February 15, 2009October 25, 2019 by Eric Peterson

I’ve picked up some new terminology over the course of the past few weeks thanks to an intermediate statistics class I’m taking. Specifically — what inspired this post — is the distinction between two types of statistical studies, as defined by one of the fathers of statisical process control, W. Edwards Deming. There’s a Wikipedia entry that actually defines them and the point of making the distinction quite well:

Enumerative study: A statistical study in which action will be taken on the material in the frame being studied.

Analytic study: A statistical study in which action will be taken on the process or cause-system that produced the frame being studied. The aim being to improve practice in the future.

…In other words, an enumerative study is a statistical study in which the focus is on judgment of results, and an analytic study is one in which the focus is on improvement of the process or system which created the results being evaluated and which will continue creating results in the future. A statistical study can be enumerative or analytic, but it cannot be both.

I’ve now been at three different schools in three different states where one of the favorite examples used for processes and process control is a process for producing plastic yogurt cups. I don’t know if Yoplait just pumps an insane amount of funding into academia-based research, or if there is some other reason, but I’ll go ahead and perpetuate it by using the same as an example here:

Enumerative study — imagine that the yogurt cup manufacturer is contractually bound to provide shipments where less than 0.1% of the cups are defective. Imagine, also, that to fully test a cup requires destroying it in the process of the test. Using statistics, the manufacturer can pull a sample from each shipment, test those cups, and, if the sampling is set up properly, be able to predict with reasonable confidence the proportion of defective cups in the entire shipment. If the prediction exceeds 0.1%, then the entire shipment can be scrapped rather than risking a contract breach. The same test would be conducted on each shipment.
Analytic study — now, suppose the yogurt cup manufacturer finds that he is scrapping one shipment in five based on the process described in the enumerative study. This isn’t a financially viable way to continue. So, he decides to conduct a study to try to determine what factors in his process are causing cups to come out defective. In this case, he may set up a very different study — isolating as many factors in the process as he can to see if can identify where the trouble spots in the process itself are and fix them.

It’s not an either/or scenario. Even if an analytics study (or series of studies) enables him to improve the process, he will likely still need to continue the enumerative studies to identify bad batches when they do occur.

In the class, we have talked about how, in marketing, we are much more often faced with analytic situations rather than enumerative ones. I don’t think this is the case. As I’ve mulled it over, it seems like enumerative studies are typically about performance measurement, while analytic studies are about diagnostics and continuous improvement. See if the following table makes sense:

Enumerative	Analytic
Performance management	Analysis for continuous improvement
How did we do in the past?	How can we do better in the future?
Report	Analysis

Achievement tests administered to schoolchildren are more enumerative than analytic — they are not geared towards determining which teaching techniques work better or worse, or even to provide the student with information about what to focus on and how going forward. They are merely an assessment of the student’s knowledge. In aggregate, they can be used as an assessment of a teacher’s effectiveness, or a school’s, or a school district’s, or even a state’s.

“But…wait!” you cry! “If an achievement test can be used to identify which teachers are performing better than others, then your so-called ‘process’ can be improved by simply getting rid of the lowest performing teachers, and that’s inherently an analytic outcome!” Maybe so…but I don’t think so. It simply assumes that each teacher is either good, bad, or somewhere in between. Achievement tests do nothing to indicate why a bad teacher is a bad teacher and a good teacher is a good teacher. Now, if the results of the achievement tests are used to identify a sample of good and bad teachers, and then they are observed and studied, then we’re back to an analytic scenario.

Let’s look at a marketing campaign. All too often, we throw out that we want to “measure the results of the campaign.” My claim is that there are two very distinct purposes for doing so…and both the measurement methods and the type of action to be taken are very different:

Enumerative/performance measurement — Did the campaign perform as it was planned? Did we achieve the results we expected? Did the people who planned and executed the campaign deliver on what was expected of them?
Analytic/analysis — What aspects of the campaign were the most/least effective? What learnings can we take forward to the next campaign so that we will achieve better results the next time?

In practice, you will want to do both. And, you will have to do both at the same time. I would argue that you need to think about the two different types and purposes as separate animals, though, rather than expecting to “measure the results” and muddle them together.

Analytics Strategy, Social Media

Customer service done right in Twitter, #wa style

Posted on February 12, 2009April 12, 2017 by Eric Peterson

Like many people, over the past few months I have become quite the Twitter-wonk. I find myself spending an increasing amount of time monitoring the #wa channel in Twitter, even if my individual contribution has a tendency to ebb and flow. And while I watch the Twits ramble on, one thing I have developed is an appreciation for the work that Ben Gaines is doing on behalf of Omniture.

Who is Ben Gaines? Ben is the guy who monitors all of Twitter for things like “reported 25 hour latency in omniture conversion reporting. good thing we’re not ecommerce” and “really productive omniture call – happiness is helpful reporting tools!!” More importantly, Ben is the guy who is paid by Omniture to take the time to reach out to anyone and everyone who has a problem in an attempt to engage them in a positive conversation.

Yep, Ben Gaines is @OmnitureCare.

Given the challenges that every web analytics vendor faces, combined with the naked conversations happening in Twitter, the fact that the management team at Omniture has dedicated an even-keel like Ben it is a testament to the company’s awareness of the marketplace around them. And while other vendors have slowly started to dedicate similar resources, Ben has established himself (at least in my mind) as the standard against which all other analytics vendor’s representatives in Twitter will be judged.

Even though I’m heading to Salt Lake City in a few days and will have the opportunity to meet Ben face-to-face, I reached out to the team at Omniture and asked to interview him for my blog. My questions and Ben’s responses follow.

Q: Tell me a little about yourself … who is “Ben Gaines” and how did you get into web analytics?

A: I never quite know what to say in introducing myself, so I’m going to give you 10 words/phrases to describe me: Husband. Father. Boston expatriate (and, yes, Red Sox fan). Computer geek. Wannabe athlete. Omniture-ite. Web analytics student. MBA candidate. Writer. That’s me in a nutshell, I suppose. And it’s slightly embarrassing how hard it was for me to come up with that list.

Would it be cliché for me to say that I first got into web analytics in seventh grade when I put a hit counter on my first web site? My first serious foray into web analytics was at my last company, where I helped to run what was then Utah’s official travel web site. Analytics wasn’t part of my primary responsibilities, but I remember being fascinated by the technology involved and the business logic that defined how we used the data. When the opportunity to move to Omniture came along, I jumped at the chance.

Q: When did you start at Omniture and how did you get appointed to the role of “Twitter Support Rep?”

A: I started here in April 2006 in our ClientCare support group (then called “Live Support”), and moved into a role as a support engineer, with more of a programming emphasis, about a year later. Both of these positions helped me to become personally invested in our clients’ success, and I have tried applied that sense of responsibility to everything I’ve done at Omniture.

I don’t believe that I have been given the opportunity to represent ClientCare on Twitter because I am singularly capable of doing so; my colleagues are similarly accomplished and insightful. What I believe I do offer is a strong understanding of the “under the hood” aspects of Omniture tools and implementation, a decent amount of experience working with these products as well as with our clients, and a strong desire to be out there helping people get the best value out of their Omniture experience.

Q: Do you do something else at Omniture other than monitor Twitter?

A: I currently help to manage our online documentation efforts (with particular emphasis on our Knowledge Base), and am involved with support issues in certain cases. I also dabble in building internal tools and scripts to help us serve our clients better and/or faster. While I do monitor Twitter very closely, I’ve always got something else going on my other monitor. There is more than enough to keep me busy.

Q: Describe the tools you use to monitor Twitter for Omniture?

A: I’ve tried probably a dozen Twitter apps. My favorite is currently TweetDeck, primarily because it allows me to monitor mentions of Omniture, SiteCatalyst, etc. perpetually in a separate column. That is really the most critical feature of any tool I’d consider using to interact with Twitter for customer service purposes. Most support requests via Twitter aren’t in replies to me directly; they’re found because someone—often someone not even following me—mentioned Omniture in their tweet. That’s when I step in, if I believe I can help in any way.

Q: Tell us a little about how you help customers using Twitter?

A: There are a few ways that I try to help customers using Twitter. One is to disseminate information quickly to a large group of people. During my time at Omniture, I’ve really tried to learn the “ins and outs” of SiteCatalyst and our other products, and I love sharing those hidden gems whenever possible. When there is an issue that everyone needs to know about, or a tip that I learned in a conversation with a colleague that I believe would benefit our users generally, I’ll throw it out there. I’ve gotten really good feedback on that practice.

Another way is as a resource for quick questions—things that may not warrant calling in to our ClientCare team and that I can handle on the spot or with just a minute or two of research—which clients are welcome to throw at me. These are actually my favorite in the context of Twitter because they often allow others to learn and contribute along with whoever is asking the question. What’s really cool about this is seeing other clients jump in and nail the answers to these questions before I do.

We’ve seen that our efforts on Twitter can sometimes even reduce the amount of support calls. Many of these questions/issues are actually fairly straightforward, and can be resolved in one or two tweets.
Finally, of course, I watch for mentions of Omniture or our products that may be support or feature requests and do what I can with them. We’ve gotten some really excellent feature requests via Twitter, and our Product Management team very much appreciates it.

Q: Tell us a little about how you deal with non-customers / complaints about Omniture?

A: I suppose this depends on the nature of the tweet. There are certain complaints (as well as non-customer questions) which are completely legitimate, and I do my best either to address them or to point the individual in the direction of someone who can. We’ve seen that our efforts on Twitter can sometimes even reduce the amount of support calls. I am not sure I can help people who are negative for the sake of negativity in 140 characters.

Q: What is the funniest Tweet you’ve seen/received about the company?

A: The funniest tweet about the company was one that said, “wondering when omniture will be able to provide users with a brain plug-in as part of the suite.” We’re working on it. I think it’s in beta.

Q: Who do you follow in Twitter?

A: The people I follow typically fall into two categories. Of course, I follow our customers. Finding our customers on Twitter can be tricky, so I often have to wait until one of them tweets about Omniture before I can follow them. Then I also follow industry thought leaders—yourself, Avinash, and others—from whom I am learning a ton about web analytics in general.

When someone begins to follow me without having tweeted about Omniture, I usually check his or her profile to see whether or not the person is likely to be a customer or to tweet about web analytics or Internet marketing (SEO, SEM, etc.). If so, I’ll follow. If not, I won’t.

The thing about using Twitter (or other social media) for customer support is that by following dozens or hundreds of people, I end up with a lot of updates regarding what so-and-so is eating for lunch, while I’m there mostly for professional, rather than personal, purposes. Maybe I’m a good candidate to represent ClientCare on Twitter because I don’t mind the personal updates at all. Frequently I find myself getting jealous of what our clients are eating for lunch, though.

Q: How important do you think Twitter is to customer relationship management?

First of all, I think it’s important to note that Twitter is only a part of our overall social media efforts. I will be starting to post on blogs.omniture.com shortly, and we’ve already got a ton of great content out there from 15 different experts. We want to hear from our customers about the issues they are facing and share information that will help them do their jobs better. The most important thing is staying on top of the latest trends in this area; today, a lot of our customers are on Twitter, but in six months it might be some other tool. Whatever it turns out to be, we’ll be there.

Regarding Twitter and customer relationship management, I know it’s been hugely important for us—ClientCare, and really for Omniture as a whole. I love the idea that we can listen to our customers so easily. When there are support issues, we can deal with them quicker than ever before. When there are feature requests, it’s easy to gauge whether there is a groundswell of support for the idea. When there are complaints, we can deal with them immediately and, in many cases, put customers’ minds at ease.

We’ve received a lot of very positive feedback regarding our efforts on Twitter. I think it’s important for customers to know that we are listening. It empowers them to interact with us in a new and powerful way. And that’s not just rhetoric—we really are listening.

The other way that Twitter is important is that it feeds into the two other main thrusts of ClientCare’s efforts—support and documentation—while those elements also feed into Twitter, allowing us to solve issues and answer questions more completely than ever before. When someone asks a question via Twitter, it often feeds into the Knowledge Base. Conversely, as I am working on our documentation I frequently find information that I believe would be useful to many of our clients, and will post it on Twitter. Support issues feed into the Knowledge Base and Twitter as well; when there are general questions asked of our ClientCare team, those will often find their way into both our documentation and onto Twitter. And tweets often result in support tickets being opened, and subsequently in additions to our documentation, when questions and issues go beyond what I can handle in 140 characters.

Q: What are your measures of success as a Twitter Support Rep?

A: I think I’m still trying to feel out what the correct metrics are. Certainly time to response and time to resolution are KPIs, but that goes without saying in customer support and relationship management. At this point, I suppose my goal is to leave 100% of clients who interact with me feeling more confident in their Omniture abilities. It’s always a success when I’m able to disseminate knowledge and help our customers get better value out of our tools.

Thanks to Ben and his managers for allowing me to conduct this interview. If you know of someone else in the web analytics arena doing excellent work in Twitter I’d love to hear about it.

Analytics Strategy

Monish Datta: "I can’t believe Sasha skipped WAW for the US-Mexico World Cup Qualifier!"

Posted on February 12, 2009October 25, 2019 by Eric Peterson

Actually, THAT’s almost a direct quote. Sarcastic as it may be. We were actually competing with a US-Mexico World Cup qualifying match that was being played in Columbus (in some crazy weather…but I’m getting ahead of myself). The US won 2-0, for what it’s worth, and I’m sure Sasha enjoyed the game. I’ll get an update next month!

This month’s WAW was something of a last-minute adventure. We once again had the event sponsored by the Web Analytics Wednesday Global Sponsors, and we had a drawing for a WASP for Analyst license, which David Ruen won. I’ve got to give a tip of the hat to Sandy and Ben Blanquera for the amazing work they’ve done getting Columbus Tech Life up and running, as we continue to bring in a few fresh faces from the Columbus Tech Life Meetup postings for WAW.

The real adventure this month was that Columbus had projected high winds for Wednesday evening. And, as the day progressed, there were rumors floating around about a Level 2 Storm Alert and 60 mph winds. After a brief flurry of e-mails, we decided that we would go ahead and have WAW, and I sent out a quick note to that effect, but let people know not to worry if they’d registered but then weren’t going to make it due to the weather. Power is still out in some neighborhoods in the area a full day later — the wind lived up to the hype.

I was heading to the event in between one wet-windy-heavy storm and what later turned out to be mostly just high winds storm and caught a pretty spectactular double rainbow out my window. Lucky for me, I was heading to WAW, so had more than just my Blackberry available to snag a picture!

We tried another new venue this month — Bar Louie in the Arena District. And, overall…too loud (a common theme). But, good food and good drink, topped off with a good crowd (we rattled off seven people who intended to come and then didn’t either because of a last-minute conflict or the weather):

Ironically, Monish Datta — the target of my running gag to make this site dominate organic searches for his name — is almost entirely obscured behind Brian in this picture.

The new faces this month included:

A co-founder of SearchSpring, which is an ASP site search tool geared towards small- to medium-sized e-commerce sites
The founder/owner of Jones Insight, a customer and marketing analytics consulting firm
A couple of folk from Bizresearch, which has developed a service for providing easy-to-understand SEO/SEM reporting
A jack-of-all-things-web-marketing marketer from Scotts

And, oh dear, I’m just not going to get into listing where everyone was from. As always, it was interesting to watch the interactions — the people who realized they actually had worked with each other, but only over the phone, the people who had 2 degrees of separation from each other, and, of course, the web analytics chatter.

Due to the noise level, we only did a half-hearted attempt to run our planned round table question of, “What is the most interesting (or entertaining…or terrifying) example of MISinterpretation of web analytics data you have seen?” We got a few chucklers:

The company that had spent a lot of development time and money to roll out a new feature on their home page. The analytics showed that 0.03% of the visitors to the home page were using the feature. The analyst who provided that insight got a call from the person who had championed the development. She told him, “Thanks so much for that data. It helped me justify keeping that feature on the home page!” The analyst wondered…how?!
A related example from a different participant. He had a client who had a “My” feature on their home page — a “My Favorites”-type of link-saving feature on the site. They were just about to spend $15,000 (and it was a fairly small company) to have someone update the feature. The analyst spent 5 minutes demonstrating that there was virtually no actual use of the feature, and the updates they were planning weren’t really geared towards that, anyway. The project got canned. Hmmm… turns out that wasn’t a misuse of web analytics at all, was it? Well, we are a wild and crazy bunch, so we let the rebels say their piece.
The time that a product manager who did a lot of self-service on the web analytics front saw a sudden 10X increase in visits to one of his product pages several weeks after he made some minor content updates for SEO purposes. He showed the results to his manager, then he shared them with the VP of Marketing, then he shared them in a large staff meeting. He developed quite a spiel about his SEO results. Then he shared the data with the web analyst, who immediately applied one of his favorite filters: the “common sense” filter. It took some digging to find out that the web infrastructure team was testing a new web site monitoring service…and that page was one of the pages they used for the test. And the company was using a log-based analytics package. And the user agent for the monitoring service wasn’t being filtered. The step function was entirely bogus.

The event started to wind down earlier than normal. I was drifting out myself after 2.5 hours. Dave and Andrew had started to head out earlier, but had gotten engrossed in a conversation and wound up sitting down to finish it. As I walked out, I got engrossed in their conversation. A half-hour later, as I started to leave (again), I realized that a number of Deloitte consultants who I work with had drifted in to watch the UNC-Duke basketball game. I wandered over for a quick, “Hey”…and didn’t leave until 11:15.

Which is why I’m going to end this post here and go to bed!

Analysis

The "Right" Talent: an MVT-Meets-Fractional-Factorial-Design Anecdote

Posted on February 8, 2009October 25, 2019 by Eric Peterson

When it comes to business/management books, one of my favorites is First, Break All the Rules. When I first read it, it didn’t strike me as particularly profound. I was relatively new to managing people, and I was being “forced” to read it for an internal class, so my natural reaction was to view it cynically. I’m not proud of it, but that’s how I roll.

Over time, I’ve found myself quoting and recommending the book again and again (I can’t say the same for the follow-up book — Now, Discover Your Strengths — but that’s a topic for another post). The fundamental premise of First, Break All the Rules, goes something like:

There is a difference between skills and talents — the former can be taught, whereas the latter are more innate characteristics, the combination of which make each person unique
Conventional wisdom has managers focussing on hiring for skills and then focussing on employees’ weaknesses and trying to “fix” them
The “weaknesses” are, all too often, talents the employee simply does not have
It’s much better to identify each employee’s talents/strengths and then help them figure out how to capitalize on those strengths rather than simply managing to their weaknesses

The book also explains the fallacy of spending a disproportionate amount of time with your weakest employees, which is where day-to-day management tends to pull you.

And, I’m possibly butchering the book in my summary — it’s been a few years since I reread it!

When it comes to data-driven job roles — think most roles where the word “analyst” appears in the job title — the real challenge is finding people who have the right mix of talents. A big part of what we worked on in the Business Intelligence department at National Instruments was building a capability to operate effectively in two different dimensions:

As business experts — we recognized that we needed to know our business and the business of sales and marketing as well or better than our internal customers. We had to genuinely want to understand the business problems they were facing — know them well enough that we could articulate them effectively on our own.
As data geeks — at the end of the day, we were expected to be able to pull data, analyze it, explain the results, and present them effectively.

What made our team effective, in my view, is that everyone in the department was very strong in one of these areas, and at least competent in the other. And, we paired up people with complementary skills when it came to tackling any project. On the one hand, this sounds like it was inefficient, but it really wasn’t — it didn’t mean that these teams were joined at the hip and never operated alone. Rather, it meant that they collaborated — both directly with our internal customers as well as offline with each other — to come at each project from multiple angles.

Now, here I am several years later, taking an Intermediate Statistics class through The Ohio State University. The class is taught on-site at my company, and it’s taught by a professor who spent a big chunk of his career in applied statistics working for Battelle. He’s a good professor, and I particularly like that he beats a pretty hard drum when it comes to the parallel talents needed to effectively use data in a business setting: subject matter expertise, effective problem formulation, and statistical/analytical knowledge. He rails against trained statisticians and even “applied mathematicians” who don’t want to really address the first two requirements head-on — those who jump to crunching the data prematurely, relying on their technical tools and skills to route them to “the answer.”

Bravo, I say!

At the same time, even as the professor is eloquently speaking to this issue (and politely patting himself on the back for not falling into the trap), it’s readily apparent that he’s coming at the analysis of data from a heavy background of hardcore statistics. And, while he spent much of his career working on industrial (and defense) processes and problems, he is now mired full-time in the world of marketing and consumer behavior. He is not the first data guru to cross over, by any means, but he is clearly new to the space, and, in many ways, seems hell-bent on retreading ground that has been covered already.

As one example, there is the case of MVT, or multivariate testing. MVT has been getting a lot of buzz in marketing over the past five years or so. It’s been touted as a way to accurately test many different variables without having to run a gazillion experiments to test every combination of them. One place that MVT gets used these days is with web landing page design — enabling a marketer to test various color schemes, banner ad taglines, and headline placements to derive the optimal combination without breaking the bank with the number of tests that have to be run to make a valid, data-driven decision. That’s all well and good, and it’s clearly gotten enough traction that it works.

An old-school process improvement expert I worked with back when I was first starting to hear about MVT pulled me aside one day and said, “You know, Tim, none of this stuff is really knew — MVT’s been around since World War II. It just wasn’t applied to Marketing until recently, so there are a lot of people capitalizing on it.” And…he was right!

So, back to the present day. In a recent class, this OSU professor was walking through various ways to design experiments and how they could be analyzed, and he kept referencing “fractional factorial design.” We’re only going to touch on the technique in the class, he assured us, but he explained how there were trade-offs you have to make when using that approach. From his explanation, it sounded like fractional factorial design was a lot like MVT, and I asked him about it after class. He had never heard of the “MVT” acronym, but said it sure sounded a lot like fractional factorial design (which was a term that was entirely knew to me). It only took a few seconds on Google to find out that our suspicions were correct.

I was surprised that MVT was a new term for this professor, as it’s hard to do much of any poking around in marketing circles these days without stumbling into it.

At the same time, “fractional factorial design” was a new term for me. But, then again, I’m coming at things from a background much more grounded in marketing and business, rather than deep mathematics. I understand the point of MVT, but I’m still wildly fuzzy on the actual mechanics of it.

And…that’s the short point of this lengthy post: it seems like, in order to truly use data effectively, requires a mix of talents that rarely occur naturally in a single person. This professor has the deep statistical chops and an awareness that he is not a subject matter expert when it comes to marketing and consumer behavior, so he needs to increase that knowledge. Partner him with someone with a deep marketing background (who is almost certainly not also a statistician), and, as long as that person has an awareness of analytics and an interest in applying analysis effectively, you’ve got a winning formula.

It’s a Venn Diagram, really. If there is no intersection of the talents, then chances are it’s a combination of talents doomed to fail.

UPDATE: Check the first comment below for a clarification — MVT is not the same thing as fractional factorial design. Rather, fractional factorial is one approach for conducting MVT experiment analysis. The link on the subject earlier in this post makes this same point. I was unclear/ambiguous. I still think that, in Marketing circles, when MVT has gotten recent play, that it is fractional factorial design and analysis that gets the buzz. But, I don’t know for sure.

Adobe Analytics, Analytics Strategy, General

Free webinars on February 11th and 12th

Posted on February 4, 2009April 12, 2017 by Eric Peterson

If you are one of the many, many people out there who have been told in no uncertain terms that there is “no travel budget” to attend conferences in the near future and you’re bummed out about missing out on some great learning opportunities Analytics Demystified has a great solution! Rather than mope around the office, complaining about missing Ian Ayres at WebTrends Engage or Maroon 5 at the Omniture Summit, why not join Analytics Demystified, Forrester Research, Coremetrics, and Tealeaf for two free webinars next week!

And who doesn’t love “free?”

The first webinar will be held at 10 AM Pacific next Wednesday, February 11th and is sponsored by the nice folks at Coremetrics. The topic is campaign attribution, and while the “official title” of the presentation is “Effectively Managing Your Online Marketing Mix with Advanced Attribution” my personal subtitle for the event is “How LAST-Based Attribution is Wrecking Your Marketing Budget (and What To Do About It!)”

While I love the topic, I’m doubly excited about this webinar since I get to co-present with John Lovett from Forrester Research. The best thing about presenting with John is that he is never shy about his opinion and we frequently get into “heated” debates, even in front of a live audience. We will also be joined by Coremetrics own John Squire, a gentleman also known for his willingness to have his opinion heard.

If you’d like to participate in this webcast with Analytics Demystified, Forrester Research, and Coremetrics please register for this totally free event through Coremetrics.

The second webinar will be held at 9 AM Pacific next Thursday, February 12th and is sponsored by the nice folks at Tealeaf. The topic here is the Web Site Optimization Ecosystem that I first described with Tealeaf and Foresee Results back in 2007. The ecosystem is a great topic when times are tough since most companies have at least two of the technologies we’ll discuss (web analytics, voice of customer, customer experience management, testing, personalization) but have done very little to actually integrate the systems.

I will be joined on the call by Geoff Galat, Tealeaf’s VP of Marketing and Product Strategy. If you’d like to register and listen to Geoff Galat from Tealeaf and I, please register for this totally free webcast through Tealeaf.

So there you have it. Two great topics, five smart presenters, one best-of-all-prices … FREE.

I hope you’ll be able to join us, and as always, please keep up with Analytics Demystified via our web analytics events page at http://www.analyticsdemystified.com.

Adobe Analytics, Conferences/Community

Web Analytics Wednesday partners with the WAA and eMetrics!

Posted on February 3, 2009April 11, 2017 by Eric Peterson

I am incredibly pleased to be able to announce that thanks in part to the hard work and determination of my good friend (and soon-to-be-Mommy!) June Dershewitz the Web Analytics Association and eMetrics Marketing Optimization Summit have both signed on to become “Community Sponsors” of Web Analytics Wednesday! You can read the entire press release if you’re so inclined, or have a look at the updated Web Analytics Wednesday Sponsors page.

These two great organizations will join Analytics Demystified, SiteSpect, and Coremetrics as official sponsors of Web Analytics Wednesday in 2009. The WAA will be helping “spread the word” about Web Analytics Wednesday among the Association’s growing membership, and eMetrics will help make sure that there is a WAW event at every Marketing Optimization Summit conference around the world — starting in Toronto on March 29th!

Thanks to the financial and community support Web Analytics Wednesday recieves, our goal for this coming year is to help at least two dozen new cities start to to have regular WAW events! If you’re interested in starting a WAW group in your community or have any questions about the Community Sponsors please don’t hesitate to contact us directly.

On behalf of Analytics Demystified, the WAA, Jim and Matthew, SiteSpect, Coremetrics, and all the great local sponsors of WAW events around the globe, thank you to everyone who hosts and participates!

Analytics Strategy

Win a WASP v1.09 for Analyst license at Web Analytics Wednesday

Posted on February 2, 2009October 25, 2019 by Eric Peterson

It hardly seems like it’s been a month since the last Web Analytics Wednesday in Columbus! Maybe that’s because it hasn’t been, but it’s that time again anyway!

Come share food and drink with a group of above-average people who are interested in and working with web analytics. A good time will be had by all…and this will be the first Columbus WAW with a door prize!

Jumping right to the details:

When: Wednesday, February 11th at 6:30 PM

Where: Bar Louie in the Arena District (504 N Park Street)

RSVP: http://tinyurl.com/WAWFeb09

Format

We got rave reviews about last month’s format where we went around the table and asked a question about web analytics (read a summary at http://tinyurl.com/WAWJan09), so we’re going to give that approach another shot this month. This month’s question:

“What is the most interesting (or entertaining…or terrifying) example of MISinterpretation of web analytics data you have seen?”

Answering the question is entirely optional — if you are new to web analytics or simply have had nothing but the most sophisticated of business partners, then a “Pass” is an entirely acceptable answer.

What’s this WASP v1.09 door prize?

WASP is the Web Analytics Solutions Profiler — a Firefox extension that comes in handy in oodles of ways when it comes to sniffing out issues, debugging deployment problems, and generally just snooping around web pages to see what’s what on the tagging front. Read more at: http://webanalyticssolutionprofiler.com/. We’ll be conducting a drawing for an Analyst license (a $49 value) at the event.

As always, please forward this post along to your friends and colleagues who are interested in web analytics. The more the merrier!

Adobe Analytics, Analytics Strategy, Conferences/Community, General

Analytics Demystified speaking engagements

Posted on February 1, 2009April 12, 2017 by Eric Peterson

One of the things I love the most about my job is the work I do as a professional speaker and industry evangelist. While the economy is not showing any great signs of improving, the speaking circuit shows no signs of slowing down. Here is a summary of some of the web analytics events Analytics Demystified will be at between now and the end of the summer, keeping in mind that more events will undoubtedly be added. If you’d like to have Eric T. Peterson speak at your event please contact us directly.

Online Marketing Summit, San Diego, February 5th

At the Online Marketing Summit I will be giving a presentation titled “Attribution, Influence, and Engagement: The Digital Marketer’s New Nightmare” on Thursday, February 5th. We will also be having a special Web Analytics Wednesday on Thursday event at the Westin Hotel downtown. Learn more about OMS and sign-up to join us at Web Analytics Wednesday.

The MindMeld at the Omniture Summit, Salt Lake City, February 17th

As part of the Omniture Summit, Matt Langie has organized an “X Change” like, invitation-only event called the “MindMeld(TM)”. Co-hosted by both Jim Sterne (WAA, eMetrics) and John Lovett (Forrester Research) the afternoon event will attempt to debate and develop solutions for Social Media measurement, Mobile and Video, and discuss how we can collectively elevate the role of analytics in the organization.

Given the somewhat tumultuous history I have with this organization’s management team I am honored that Matt invited me and I am looking forward to seeing the pagentry of the Omniture Summit first-hand. If you’d like an invitation to the MindMeld please let me know.

SearchFEST, Portland, March 10th

SearchFEST is a full-day search marketing conference hosted by SEMpdx and the fine folks at Anvil Media. This year’s keynote speaker is the great Danny Sullivan and I am honored to be on an analytics panel with Portent Interactive’s Ian Lurie and Widemile’s Bob Garcia. Hallie Jansen at Anvil was kind enough to give me a discount code so if you’d like to join us at SearchFEST and save a little money, reach out directly and I will hook you up!

Web Analytics Strategies, Milan Italy, March 17th

At the Web Analytics Strategies Conference and Expo I am a keynote speaker and listed as a “special GURU” which is nice (although it makes me feel kind of old.) I believe this is the first Web Analytics Expo in Milan and so I’m doubly excited about giving my presentation on “Competing on Web Analytics” If you’re a reader of my blog in Italy and can make this event please contact me so that we might meet for coffee! I believe we will also be having a special Web Analytics Wednesday event the evening of the 17th in Milan so watch the Web Analytics Wednesday calendar.

Los Desayunos MVC con Analitica Web, Madrid Spain, March 18th

At this special web analytics event in Madrid, Spain I will be speaking on the “New Measures for Online Marketing” with the delightful Sergio Maldonado. I’m especially excited to be returning to Madrid since my very good friends Rene and Aurelie will be coming down from Brussels with their young baby Lucca who I have not had the pleasure of meeting yet. If you’re in Madrid, let’s meet for sangria, shall we?

SAS Global Forum, Washington, March 23rd

At the SAS Global Forum I am honored to be giving a keynote presentation on “Competing on Web Analytics.” I’m very excited about presenting at the SAS conference and hope to connect at the conference with web analytics professionals who are also using SAS. If you’ll be at the Global Forum and would like to meet, please let me know.

WebTrends Engage, Las Vegas, April 7th

Being a Portlander and having gotten my start at WebTrends back in the late 90’s I was already excited to be going to the Engage conference. Then the company announced that Ian Ayres, author of Super Crunchers, will be giving a keynote presentation. I love what the company is doing with their conference web site and it seems like I’m constantly seeing news on Twitter about the event. If you’re heading to Vegas for Engage, make sure to look for me by the blackjack table.

DMA ACCM 2009, New Orleans, May 4th

I’ve never had the pleasure of presenting at the Annual Conference for Catalog and Multichannel Merchants but am very excited to be giving two presentations at ACCM 2009. The first presentation will be more an intermediate class in web analytics and the second advanced. Both will be highly interactive and so I’m looking forward to meeting the DMA and ACCM audience.

eMetrics Marketing Optimization Summit, San Jose, May 6th

The big event, the really big show, the grand-daddy of them all … eMetrics. I am still proud to say I have never missed an eMetrics here in the USA, in part because I love the event and in part because of the profound respect I have for the conference’s organizer, Mr. Jim Sterne. I am more or less racing from New Orleans to San Jose but please look for me at eMetrics if you’d like to catch up! Plus, we will definitely be having a blow-out Web Analytics Wednesday at eMetrics this year …

Internet Retailer 2009 Conference and Exhibition, Boston, June 16th

I have always loved the Internet Retailer conference but have rarely been able to attend due to schedule conflict. Fortunately Kurt Peters got me this year before things started picking up and so I’m happy to be presenting with Mike Fried from Backcountry.com and getting deep into the details that online retailers need in their web analytics efforts.

The X Change Conference, San Francisco, September 9, 10, and 11

I could not be more excited about this year’s X Change conference for a variety of reasons. We have great buzz after last year’s event, I have talked to dozens of past participants who have told me they’re saving their limited conference dollars for the X Change, and I love that Gary and Joel from Semphonic are always looking for ways to make a great event even better. If you’d like to join us at the web analytics industry’s premier event, or if you’d like to talk about possibly leading a huddle, please don’t hesitate to email me directly.

Well, that’s most of it, at least for now. There are a handful of webcasts I’m doing on behalf of companies like Coremetrics (with John Lovett from Forrester) and Tealeaf (on the Web Site Optimization Ecosystem) but I’ll try and cover those in another, shorter post.