Year: 2009

Analytics Strategy

Fear vs. Convenience — The Customer Data Conundrum

Posted on by Eric Peterson

I was in a presentation today where the presenter spoke enthusiastically and at length about how customer data is king — how companies that gather customer data (both explicit data provided by the customer, behavioral data collected about the customer, and external data acquired from third party sources) and put it together effectively can serve the customer (and prospective customers) in a more personalized fashion. Putting that data to use benefits the customer, which, in turn, benefits the company.

That’s entirely true. And, it’s a common theme in marketing these days: “one-to-one marketing” is the catchphrase that marketers use. It’s what being “customer centric” is all about. Waiting for Your Cat to Bark is based on this concept, as are countless other books, blog posts, conference sessions, classes, podcasts, and so on. My local paper ran a piece last weekend about some of the ins and outs of how Kroger uses data from its loyalty card program to provide customers with coupons that are most likely to be of interest to them (the article is actually about the company that Kroger contracts this work out to — dunnhumbyUSA). I’ve long suspected, and the article confirmed, that many grocery chains put these loyalty card programs into place and then find that it’s a lot harder to actually use the data they collect in a meaningful way than to set up the processes to collect the data…but that’s another post entirely.

What all of these specific examples miss — or at least drastically underestimate — is the fear component. And that was touched on briefly in the January 8, 2009 Slate Political Gabfest podcast. John Dickerson, the host of the gabfest, launched into a riff about electronic medical records. The initial point of the discussion was how the Obama administration is expected to be much more tech-savvy and tech-interested than the Bush administration, and how modernizing the medical record system was one possible initiative that might get funded by the economic stimulus package. This led to a discussion among all three of the gabfesters about how this seemed like a no-brainer and would lead to all sorts of improvements. But, Dickerson then explained how there is a lot of fear associated with the initiative — how members of Congress who are proponents of this sort of legislation are coached to talk about “medical modernization” rather than “electronic medical records.” We’ve all seen articles about the privacy concerns around electronic records of this sort — the prospective employer who finds out you have been treated by a psychiatrist, the health insurance company that finds out you have a genetic marker that increases your risk of contracting cancer (and thus increases your premium), etc.

And THAT is the crux of the biscuit.

For every benefit of having a centralized, comprehensive record of who you are and what your likes and dislikes are, there is a risk that that data will fall into the wrong hands and be put to nefarious uses:

  • By storing your credit card information on an online retail site, you’re risking that someone could hack into their system and steal it
  • By giving your social security number when registering for college, you’re risking that a hacker could get in and steal your identity
  • By letting a web site store your login information in a cookie, you’re risking that you lose your laptop and someone goes back to the site and completes transactions as you

The more comprehensive the information that gets stored about you in a single location, the greater the potential for convenience and benefits to you…but the greater the damage if that information gets misused (think Orwell’s 1984 for the bleakest, most extreme example).

Which means it all comes down to the balance of fear versus trust, and how long it will be before the societal balance tips from the former to the latter.

Think about the airplane. In the earliest days of manned flight, anyone who flew had to either overcome a high level of fear or be possessed with a high degree of recklessness. Flying was risky and dangerous. The technology improved rapidly and dramatically, and air travel became increasingly safer. “Fear of flying” has gone from being commonplace in my grandparents’ generation to being rare in mine.

Comprehensive, centralized (or at least cross-referenceable), electronic data about you and me can be viewed in a similar light. Which leads me to a plea for corporate responsibility and stewardship of the customer’s trust on two fronts:

  1. Protecting the privacy of your customer data is paramount. Every “lost data” story that hits the mainstream media is a backwards step towards what really is the common good
  2. When using the customer data you do have, be triply sure to provide twice as much benefit to your customer as you garner for your company

It’s not going to be an immediate shift. It will take years and will rely on the overwhelming majority of companies and the government to be hyper-vigilant on both of these fronts before trust and value trump fear.

0
General

Barack Obama should not fear cookies!

Posted on by Eric Peterson

Just after President Obama was elected back in November I wrote a blog post that had been kicking around in my head for a long time calling for the “legalization” of browser cookies by Federal Government run web sites. The response to the post was great, but now it appears that the first comment from Brent Hieggelke (who was head of marketing at WebTrends for several years) was destined to become ironic.  Brent (who is my neighbor in Portland) waxed philosophical about government and cookies with this comment:

“As someone who 4 years ago spent ALL of New Years Day on the phone with the White House Communications Team because their site was “outed” by CNN and other media as <> using cookies in a completely innocent manner, I couldn’t agree more.”

Turns out that Jascha Kaykas-Wolff, the new head of marketing at WebTrends, is probably having the exact same conversation thanks to so called “privacy advocates” according to this article in InformationWeek. What’s more, the privacy advocates, rather than educating themselves about the real risks associated with the use of browser cookies are apparently patting themselves on the back for getting the Obama administration to make a simple, cosmetic change at WhiteHouse.gov regarding the use of YouTube video.

Giving himself full credit for the change, Chris Soghoian from CNET’s “surveli@nce st@te” blog says:

“It seems that someone in the White House read my blog post yesterday–as within 12 hours of the story going live, Obama’s Web team rolled out a technical fix that severely limits YouTube’s ability to track most visitors to the White House Web site.”

Congratulations Chris. Instead of giving the President’s team the lattitude to focus on, oh, THE ECONOMY, THE THREAT OF TERRORISM, THE HOUSING CRISIS, UNEMPLOYMENT, and HEALTH CARE you single-handedly managed to force the Administration to waste their time worrying about whether or not Google was getting just a little more of the world’s data.  President Obama, in the midst of rolling out a truly revolutionary use of technology in government in an effort to get more of us personally involved in our communities, our country, and our collective future, was forced by your misguided fear-mongering to stop what they were doing and address what has otherwise been hailed as a brilliant communication effort.

You sir, are the man.

Seriously people, can we stop worrying about cookies for a little while? Given all the other problems we have as a nation and as a global community, am I alone in thinking that people like Chris and his fellow “privacy advocates” need to find something else to focus their efforts on? Maybe if this community spent more time trying to help the President come up with ideas to put America back to work and less time creating fear, uncertainty, and doubt in the popular media we’d see the kind of change that our President has been talking about.

At this point I’m fairly confident that any person who has any shred of concern about their cookies being scraped, hijacked, poisoned, bombed, or otherwise maliciously used to expose their personal habits or ruin their lives has figured out how to clear or otherwise modify said cookies. Even though I started writing about the profile of the cookie deleter back in 2005, I’m still waiting for someone to give me a good reason to delete said objects that is not A) because you’re a site developer and you need to confirm how cookies are being set, B) you’re a web analytics specialist debugging tracking, C) you gamble a lot online or D) you surf a lot of porn.

If “A” or “B” I understand.  If “C” or “D” … don’t forget to clear your browser history too!

I’m being snarky, I know, and maybe I’m just taking Chris to task since he still has his street-cred inducing ponytail and I cut mine off. But at this point the hand-wringing about cookies in general much less because of the mandate set by OMB M-03-22 has become tedious and needs to stop. President Obama is working to change the way government works and I think his staff deserve some latitude when it comes to the Internet. If we want government sites to work for us, we need to let analytic technology work for them. If we want change, we need to be open to change.

Put another way, if you fear Google, don’t use their products. If you fear cookies, delete them. If you fear for your privacy online, don’t go online. Wear a foil hat. Don’t answer the phone. Don’t open the door. Don’t speak.

But please people let’s decide to take some personal responsibility on this issue and stop bugging an otherwise busy administration–whichever administration that may be. Regardless of how you feel about Barack Obama, let’s all recognize that we are facing substantially bigger challenges today than we have in recent history and since the man was fairly elected he deserves at least a chance to improve the economic conditions in the U.S. without “privacy advocates” forcing his staff to make tedious (and functionally meaningless) changes to the White House web site.

I know I’m going to get slammed for this post, that’s okay. Somebody needs to stand up for cookies and since I already tried “diplomatic” I suppose it’s time to try “direct.” Browser cookies help make it possible for great companies like CNET to provide lots of great content–including Chris’s blog! Browser cookies help justify great technology like Twitter, Facebook, and MySpace. Browser cookies power the Internet and should not be feared, especially not by President Obama.

I look forward to your comments and criticisms of my position.

0
General

Seven Things You May (or May Not) Know about Me

Posted on by Eric Peterson

I got tagged for the “Seven things…” meme by Chris Wilson.

The rules:

  1. Link to your original tagger(s) and list these rules in your post.
  2. Share seven facts about yourself in the post.
  3. Tag seven people at the end of your post by leaving their names and the links to their blogs.
  4. Let them know they’ve been tagged.

So, on to the seven facts:

  1. I cut out of college partway through my last semester to hike 2,142.7 miles from Georgia to Maine on the Appalachian Trail. I proposed to my girlfriend (now wife) when she came out to pick me up at the end of the trail.
  2. I hate talking on the phone and generally won’t answer it at home.
  3. My undergraduate degree is in architecture, although it is officially a Bachelor of Science in Art and Design. Since it is abbreviated BSAD, and due to the financial prospects for those of us who earned it, we pronounced it “Be Sad.” I don’t actually have much artistic talent, and I got out of architecture once I realized how woefully lacking I was on that front. I have an enormous appreciation for people who have visual design skills. 
  4. A play I wrote in college got produced on campus and was later included as an example for the Playwrights in Performance class included in MIT’s Open Courseware project.
  5. I grew up in Sour Lake, Texas, which is where Texaco was founded (originally named Texas Company). The town’s population was at 1,807 and dropping as of the last census before I moved away, but it has been a boomtown twice: first, in the mid-1800s, as a health resort when people (including Sam Houston) came from far and wide to soak in the town’ssulphur springs (long since dried up) that gave it its name, and again in the early 1900s when oil was discovered.
  6. I am a rabid amateur baseball fan — especially focused on the Texas Longhorns and any team or league in which I have a kid participating; if I know I’m going to be able to watch an entire game, I’ll score it if I’ve got a score sheet and a pencil.
  7. My cousin, Chris, authored the first Windows version of NCSA Mosaic (the first real web browser) and later developed the first CSS implementation in Internet Explorer. He’s currently the Platform Architect of the IE Platform team at Microsoft.

And, the seven people I’ll tag to pass this along:

  • Chris Tammen because, like my cousin, he doesn’t post nearly often enough. And, he’s one of the most spontaneously hilarious people I know (for years, his resume included “Able to change many lightbulbs without the aid of a chair or stool” in the “Additional Skills” section, as he is a very tall fellow)
  • Kristin Farwell because I already know that her mother and stepfather are professional skydivers (and she’s got a half-dozen ready tales of record-breaking and noteworthy jumps her stepfather has led or participated in), because she grew up as the daughter of a quintessential free spirit, because she taught me a lot of things about the internet, and because I have two framed photos she took in Austin hanging on the wall of my office…and I suspect her list would turn up more fascinating nuggets
  • Greg Phelps because he’s an English major who has spent the last decade of his career doing hardcore database work. And he knows baseball.
  • Avinash Kaushik because all I know is that he is one of the most brilliant and pragmatic minds in web analytics, that he is extremely personable, that he’s got a degree from The Ohio State University, and that Google probably knows how lucky they are to have him helping drive the evolution of Google Analytics.
  • Amy Bills because she is really good at what she does, but, more importantly, because her perspective on the world tends to be hilariously wry and sardonic
  • Bryan Cristina because he keeps saying he’s going to blog more, and maybe this will give him a nudge to get re-started. And because the filter between what he thinks and what he says is pretty minimal, which makes for entertaining tweets, Yahoo! group responses, and conversations
  • Kevin Sasser because I met him when he was trying to sell me a content management system (and almost succeeded — would have if the product had been a better fit for our needs), and who I have since gotten to know as the author of one of the most entertaining, yet practical, blogs on how to be effective in Sales.

    One person I would have tagged, but she was already tagged last year:

    • Connie Bensen — community strategist, mentor, friend, and former .357 pistol packer; when the meme hit her, it was an “8 things” meme, but substantially the same (and more reasonable than the “25 things” that’s making its way around Facebook!)
    0
    Analytics Strategy, Social Media

    Monish Datta: "It was the best WAW yet!"*

    Posted on by Eric Peterson

    Another month, another Web Analytics Wednesday (WAW) in Columbus. We had two sponsors — both the Web Analytics Wednesday Global Sponsors and Lightbulb Interactive, which was nifty. And, we headed back to O’Shaughnessy’s Public House because, by golly, we just knew if we gave them enough chances they could get up to batting .500 when it came to screwing up our reservations. They succeeded by having no record of our event in “the book.” We made do nonetheless.

    The turnout was slightly below normal — we wound up with eleven people all told — but we tried a new format for the discussion that worked out well! Although the group was small, it was a good mix of people: web analysts from major financial institutions, web designers, SEM and web analytics types in online retail, a horse racing marketer, and a slightly-crazy-but-always-entertaining developer from an interactive agency:

    Columbus Web Analytics Wednesday -- January 2009

    The format for the formal part of the discussion was going around the table and asking everyone (who was willing) to describe the report or type of report that they felt was the most worthless or irrelevant, and then to also describe the type of report that they could not get or that was unduly difficult to get that they felt would be most useful. In other words — cheap and easy blog fodder for me! The results…

    Most Worthless/Irrelevant Reports

    • “Hits” reports — we agreed that two cases where this wasn’t a worthless metric were: 1) error logging (e.g., missing images), and 2) server load monitoring; a late arrival proceeded to state how many hits she had to her company’s web site last year. Doh! She actually had a good recovery by proposing a third valid use: when your site is selling sponsorships and you need the biggest number you can find. Okay, so “valid” is a stretch here. Marketers. Yecchhhh!  🙂
    • Overlay reports — great eye candy for the vendor when they’re selling a web analytics product, but notoriously inaccurate, can’t handle links in Flash, require a lot of very careful link creation on the page that’s going to run the overlay to make sure all links are unique (which hurts SEO), and don’t work for pages that have their content updated with any regularity (when trying to look at an overlay from “two weeks ago;” Bryan provided us with an amusing medley of impersonations of business users asking questions about this sort of report
    • Average time on page — this prompted some debate, but the general agreement, I think, was that the problem with this report is that many, many people use it without understanding its shortcomings (which Avinash covered in detail early last year in a blog post).
    • Path reports — again, we had general agreement that it’s the persistent myth that a significant percentage of visitors to a site will follow the exact same path through the site that is the killer (I call that the “people are cows myth“); we walked through the various alternatives that do have value — single-level paths to/from a page, bucketing of types of pages, looking at combinations of pages visited but not worrying so much about sequence, etc.
    • Geographic overlays — they have their uses in some very specific cases, but they really don’t warrant being on the main page of any tool’s dashboard

    My favorite from the “worst” discussion, though, was this: “Any report provided without context.” That one from the aforementioned slightly-crazy-but-always-entertaining developer. 

    Most Wanted or Wanted-With-Less-Work

    These reports got a bit more philosophical, but it was a good list nonetheless, with some common themes:

    • Several people brought up the need to marry web analytics data to other marketing channels as a biggie; they provided examples of where they had or were in the process of doing this in some fashion, but the beef was with how painful it was; this also headed down a tangential discussion of “attribution” — siloed marketing channels lead to each channel vying for as much credit as possible when they “touched” someone who converted to a sale at some point; I think we all ordered another drink in the midst of this discussion, and the lively discussion took a slightly maudlin turn. But the drinks arrived, and we recovered.
    • Forward attribution combined with segmentation — this was actually related to the prior one, and I scribbled it down as soon as Scott threw it out…but now realize it went totally over my head. Maybe he’ll elaborate in a comment on this post (after he nails down the venue for next month’s WAW, of course).
    • Form abandonment — this was one where it wasn’t that it’s not doable, it’s that it takes a lot of work to pull it off effectively. Well worth the effort, but would get more use if it was easier to set up.
    • Onsite search — this is akin to the form abandonment one, in that it’s a really useful set of data to look at, but, all too often, is tricky to get set up in a way that makes it practical to use
    • Social media integration with web analytics — this one is a result of the decentralized nature of social media, so much of what we’d want to integrate isn’t happening on sites that we “control.”

    Other discussions/topics/mentions of note from my end of the table:

    • Dave went from being a social media skeptic less than a year ago to being an active user and evangelist. He’s even speaking on the subject in Cleveland next month (although he hasn’t yet plugged that on his blog)
    • In that same vein, Dave has also become a Gmail convert. Now…if I can just get him off of Blogspot and on to WordPress, my work here will be complete…
    • I found myself talking up Techrigy’s SM2 in two separate conversations — encouraging people to sign up for a freemium account to explore social media tracking, and plugging Connie Bensen as someone to ping on Twitter with questions.
    • I wound up talking about many of the people I met (in person or via social media) last fall when I moderated a panel on social media for nonprofits
    • We had a few chuckles about the <political> “Leaving Us in Great Pain” video </political> that I helped produce with some friends from Austin

    I almost passed my notepad around asking people to put their Twitter usernames on it…but I decided against it. Feel free to add yours as a comment here whether you were in attendance or not if you’re interested in Columbus Web Analytics Wednesday. And/or, you can join our Facebook group. I was struck by the difference 10 months makes. We talked about Twitter during the first couple of WAWs last year, and the number of users were in the distinct minority. Some people had not even heard of it. Everyone I talked to last night uses Twitter, and uses it enthusiastically. The times they are a’ changin’!

     

    * While quotation marks would ordinarily indicate that this was a direct quote, those in the title of this post more indicate paraphrasing of Monish Datta’s take on the evening. Actually…”paraphrasing” is an overstatement. In other words, I totally made the quote up. But, Monish was smiling and laughing, so I don’t feel too bad about it. I really just needed to get his name in the title for SEO chuckles.

    0
    Adobe Analytics, Analytics Strategy, Conferences/Community

    Great news from Web Analytics Wednesday!

    Posted on by Eric Peterson

    Wow, once and awhile in life you get involved in something that turns out differently than you had planned. I’ve written about this in the past, but for me Web Analytics Wednesday is one of those things

    Web Analytics Wednesday was started by my good friend (and soon-to-be-new-mom!) June Dershewitz and I back in 2005 with little expectation of success. Our goal? To give web analytics professionals around the world the means and motivation to gather locally and get to know each other.

    Now, in 2009, by every measure we have succeeded and even exceeded expectations, having helped create regular Web Analytics Wednesday events in places like New York, San Francisco, Boston, London, Toronto, Paris, Madrid, Copenhagen, Stockholm, Sao Paulo, Sydney and smaller cities like Austin, Texas, Columbus, Ohio, and Nashville, Tennessee. These events were attended by over 4,500 web analytics professionals around the world in 2008 alone and 2009 is off to a great start with over 600 total attendees in January alone.

    In 2008 we decided to do more to help Web Analytics Wednesday events get off the ground in more cities and to provide more financial support to as many groups as possible. We were able to do this with the generous support of Coremetrics and SiteSpect, the Web Analytics Wednesday Global Sponsors. Both of these companies had already been hosting events around the world, and senior management from both turned out to be enthusiastically supportive of Web Analytics Wednesday.

    Today I am incredibly pleased to announce that the Web Analytics Wednesday Global Sponsors have joined Analytics Demystified in making a donation to Operation USA. This donation of roughly $1 per Web Analytics Wednesday participant was made in recognition of the relative success many in the web analytics industry have enjoyed in the context of the challenges faced by our fellow man, woman, and child across the globe.

    Personally I have been blessed with a healthy family, a rich life, and moderate success in business, and thus have frequently been able to make charitable contributions. But as we all know, as the economy worsens, charities are often the first to feel the pinch despite the fact that an increasing number of people around the world need the support of groups like Operation USA. Because of this, I am incredibly grateful to Joe Davis, CEO of Coremetrics, and Eric Hansen, CEO of SiteSpect for their willingness to match my $1,500 contribution on behalf of both of their companies.

    I hope everyone who hosts, sponsors, and participates in Web Analytics Wednesday will take the time to thank Coremetrics and SiteSpect for their generosity, either by commenting on this post or by emailing the companies directly.

    On behalf of Analytics Demystified, June, our Global Sponsors, and all the Web Analytics Wednesday hosts we wish you all the best in 2009 and hope you’re able to make it out to an event near you soon!

    0
    Analytics Strategy

    Columbus Web Analytics Wednesday — Jan 2009 Edition

    Posted on by Eric Peterson

    The first Columbus Web Analytics Wednesday of the year is coming! As always, the meetup is open to anyone and everyone who is interested in web analytics — we just ask that you register so we have a sense of what our headcount will be.

    We are heading back downtown this month, and back to our increasingly regular haunt: O’Shaughnessy’s. We will likely be upstairs, barring any SNAFUs. We are going to try something a little new on the format this month. In our normal presentation slot, we are going to go around the table and ask everyone to answer this question:

    What web analytics report do you think has become the most irrelevant or overrated, and what report would you most want that you do not have?

    We’re not going to require that participants all answer the questions, but hopefully enough people will that we can have a good discussion. 

    Wednesday, January 21, 2009 at 6:30 PM
    O’Shaughnessy’s Public House
    401 N Front St, Columbus OH
    As always, thanks to our sponsors: Web Analytics Wednesday Global Sponsors and Lightbulb Interactive.
    We’re close to having February’s WAW lined up as well, so stay tuned!
    0
    Analytics Strategy, Conferences/Community

    Thoughts on the proposed IAB Guidelines

    Posted on by Eric Peterson

    UPDATE ON JANUARY 19, 2009: Peter Black from BPA Worldwide who was also on the IAB working group with Josh Chasin, wrote in and disagrees with Chasin’s characterization of who the “Unique User” language is targeting. I have email into the IAB and MRC’s George Ivie to clarify the situation. Watch this blog!

    UPDATE ON JANUARY 18, 2009: Josh Chasin from comScore, who was a member of the IAB working group that defined the guidelines described in this post, wrote in to point out that I misinterpreted the IAB’s intent. While their web site clearly says …

    “The IAB believes that all companies involved in audience measurement should be audited for their processes.  These audits are intended to establish the source of any measurement discrepancies and to find potential solutions.

    All measurement companies that report audience metrics have a material impact on interactive marketing and decision-making. Therefore, transparency into these methodologies is critical to maintaining advertisers’ confidence in interactive, particularly now, as marketers allocate more budget to the platform.”

    … according to Chasin the IAB is excluding web analytics vendors from “all companies involved in audience measurement” and the type of companies  that have a material impact on interactive marketing and decision-making. Since this doesn’t sound right to me at all I will warn the reader that some of the questions I raise in the following post may, in fact, be totally irrelevant (at least in the context of the IAB Proposed Measurement Guidelines.

    If nothing else, with two days left in the open comment period, the IAB may want to use my confusion as an example and clarify the target for the recommendations made in the document.

    Reader beware!

    As long as we’re talking about web analytics standards I figured I would take the opportunity to offer up a few thoughts on the Interactive Advertising Bureau’s Audience Reach Measurement Guidelines that are open for public comment until January 20th. If you haven’t had a chance to read these proposed guidelines you should, especially if you have an interest in how we collectively communicate about data.

    At 34 pages the document is certainly a slog to read–and I say this knowing full well that I have a tendency to write 50 page white papers! Since you’re all bright folks I’m just going to address some of the proposed language that stood out to me.  And, as always, if you have any thoughts or positions on the proposed guidelines I’d love to hear from you!

    Starting in Section 1.2 the IAB clarifies the relationship between “Unique Cookies”, “Unique Browsers”, “Unique Devices” and “Unique Users / Visitors”.  The discussion about “Unique Devices” is interesting because this is a clear indication of the impact that mobile devices like the iPhone are having on audience measurement.  Things start to get really interesting, however, in Section 1.2.4 where the IAB says (emphasis mine):

    “However, in order to report a Unique User, the measurement organization must utilize in its identification and attribution processes underlying data that is, at least in a reasonable proportion,, attributed directly to a person. For instance, data collected from registrants is one possible source that can be utilized in creating a Unique Users measure by a census-based measurement organization, if registrants represent a reasonable proportion of the total user-base and when appropriate scientific projection methods are used for non-registrrants (sic).  In no instance may a census measurement organization report Unique Users purely through algorithms or modeling that is not at least partially traceable to information obtained directly from people, as opposed to browsers, computers, or any other non-human element.“

    Did you get that? Keep in mind that while at JupiterResearch I was among the first to publicize the decline in accuracy of visitor counting due to cookie deletion. In fact the report was subtitled “Addressing the Decline in Accuracy of Cookie-Based Measurement.” At the time people called me crazy and Seth Godin even accused me of living in an echo chamber (I have since forgiven Seth.)

    Now, three years later, the IAB is expressly telling measurement vendors to stop reporting a metric called “Unique Visitors” or “Unique Users” unless they have a research-based strategy for determining the correct proportion of cookies to “real people” and have applied that calculation in a transparent way.

    Whoa.

    Think about this for a minute. Every one of the fine census measurement packages (nee web analytics) out there is reporting a Unique Visitor number, but I’ll go out on a limb here and propose that none of them are even vaguely adhering to the IAB proposed definition of a “Unique Visitor.”  I’ll go a step further and postulate that, at least in the base offerings, these vendors don’t currently have the technical capability required to report an estimated/algorithmically derived “Unique Visitor” count based on scientific projection methods.

    If I’m wrong about this I suspect I’ll hear about it, but I don’t think I’m wrong when it comes to the base offerings like SiteCatalyst, WebTrends Web Analyics, Coremetrics 2009, etc.  And yes, I’m aware that end-users can use higher-end products like Discover on Premise and the data warehousing tools to apply a correction factor to UV counts, but that is not what the IAB is saying. This guideline is saying that correcting for cookie, browser, and device-related over-counting of unique visitors is the responsibility of the measurement vendor.

    Again, whoa.

    And as if that’s not a radical enough move, the document goes on to state in Section 2.2 that the vendors need to actually break out these correction factors across three components: first-cookie acceptance, deletion, and browser denial (again, emphasis mine):

    “Cookie deletion rates, calibration methods and sources or estimation methods used to account for first-use, deletion, and non-accepting cookie groups should be disclosed by the audience measurement organization. The audience measurement organization should disclose census-based unique cookie counts and the estimated unique activity from first-use, deletion and non-accepting cookie groups separately and in aggregate. If the measurement organization relies on a unified model that makes reporting among these separate groups impossible, it may report these counts in aggregate only, but should be prepared to demonstrate in an audit the ability of its unified model to address each type of cookie completely.”

    The IAB goes on in Section 2.4 to start to push web analytics into what is an uncomfortable position for some people, the use of algorithms and data models, to better report on unique visitors:

    “As noted above, Publishers and Ad-servers will generally need to rely on algorithms (data models) to estimate the number of users attributable to the counts of Unique Cookies they develop. The underlying basis for this algorithm should be a study of actual users (i.e, people).  Ideally, such a study would be based on direct contact and/or observation of people using the browser at the time of accessing web-site content or ads with the unique cookie, as well as observation of the number of browsers in use by these users.  Additionally, inferences will need to be made about advertising activity of users with non-cookied browsers, so these types of users should also be contacted and observed.  Also, the activity of users who access content from multiple locations (home, work, school) on different browsers should be factored into these algorithms.”

    Finally, the IAB is telling the vendors they need to report the results of their research to their customers, essentially exposing flaws in their technology for all to see:

    “The resulting study should be representative of, and projectable to, the users of the web-site or property, and periodically re-performed to reflect gradual changes in audience.  Known weaknesses in the projection processes should be disclosed to users of Audience Reach Measurements.“

    If  you’re keeping track, the IAB is telling the vendors A) to completely change their definition of “Unique Visitors”, B) to start to actively research sources of inaccuracy on behalf of their customers, and C) pro-actively report known weaknesses in their system to their customers. Anyone want to place any bets on when the vendor community will adopt these recommendations? I’m going to be a little snarky here and put my money on “never in a million years.”

    Seriously you have to love the IAB for putting this out there.  Unlike the Web Analytics Association’s Standards which I believe are an excellent start but are a little soft in areas, the IAB is basically telling the measurement vendor community that they are doing the entire world a disservice by reporting unique visitor counts that are complete bollocks and they need to stop doing that post-haste! Okay, maybe I’m over-reading the document but the scope of changes required for any vendor to become IAB-compliant is dramatic, both technically and psychologically.

    I’m not sure if Brandt Dainow had seen the IAB proposal when he besmirtched the fine work of the Web Analytics Association’s Standards Committee, but if you compare the two proposal documents (the WAA’s proposal can be found here in PDF form) you will detect a noticable difference.  Personally I’m glad that my good friend Judah Phillips bridged the gap between the IAB and WAA and I find myself wondering, at least a little bit, whether the IAB+WAA relationship should be even deeper.

    This all brings me to an excellent point that Bryan Robertson made on my last post on standards regarding how standards are defined and moved into common use. Bryan’s thesis is based on the W3C’s move from HTML 1.0 to XHTML and his point is that this transition to the XHTML standard came about because of A) a powerful standards body, B) a vocal community, and C) passionate thought leaders.  Regarding a powerful standards body, Bryan specifically make a point that other folks have made, usually behind closed doors:

    “Is the WAA powerful enough at this point in time, or do we need to continue to build momentum before the standards can be more bold? For example, is the WAA hand wringing too much over the polite “we’ll share with you if you share with us” arrangement with the IAB over standards definitions? Is the WAA in a tough position in trying to bring practitioners and vendors together at the same table?”

    Bonus points to Bryan for willing to be direct on the conflict of interests arising from having two masters, vendor and practitioner.  Again, I have nothing but profound respect for Angie and all of the other members of the WAA Standards Committee, but since I do know that vendors participated in the definition process I wonder a little bit how much impact they really had.

    Anyway, I’m doing all the talking here and it’s a beautiful day so I will ask what you all think — either about the IAB proposal, Brandt Dainow’s assertion, Bryan’s thesis about the strength of the WAA, or anything else that strikes your fancy. Do you think the IAB standard for “Unique Users” has a snowball’s chance of being widely implemented? Do you think Brandt Dainow makes a good point (even if he does it in a lousy way)? Do you think the WAA may be better off working more closely with the IAB on Standards, given the IAB’s relative might?

    My site host assures me that my comments table will not crash again so I look forward to hearing from you all.

    0
    Analytics Strategy, Conferences/Community

    Sad to say, I partially agree with Brandt Dainow

    Posted on by Eric Peterson

    Readers who are enthusiastic members of the web analytics community are by now familiar with Brandt Dainow and his sometimes antagonistic missives published at iMediaConnection. While I try pretty hard to follow the old “if you can’t say something nice” rule I occasionally fail in my efforts. Perhaps the best evidence of my failing was my calling Brandt Daniow insane when he suggested that Google Analytics version 2.0 was “simply a quantum leap above any other analytics product on the planet.”

    While I firmly believe that Google Analytics is a great, valuable, and appropriate application for a wide range of needs, I think that Dainow’s “quantum leap” claim and statements like “What Google has done is simply take every feature in every product on the market and put them all into one system, and then make it available for free” are so obviously hyperbolic that they beg criticism (which Mr. Dainow got in spades from many within the analytics community.)

    Dainow has since turned on Google Analytics, more recently pointing out what he describes as “disturbing inaccuracies behind Google Analytics” and again getting our  attention with irresponsible statements like “Google Analytics is different from other products in that it has been intentionally designed by Google to be inaccurate over and above the normal inaccuracies that are inevitable.“

    Oddly enough, his rant about Google Analytics included some statements that rubbed members of the Web Analytics Association the wrong way.  When folks like Jodi McDermott commented on the article and questioned some of Dainow’s assertions, Brandt did what any normal person would do …

    … he wrote a nasty follow-up piece critical of the Web Analytics Association and the WAA Standards Committee!

    I will let you read his piece yourself, but the two-sentence summary of Dainow’s opinion is that “the work of the WAA standards committee is a disaster for the web analytics community. It will take years to undo the damage and create proper precise standards that can be implemented in software. The WAA “standard” is not a standard, it’s just second-rate muttering.“

    Clearly Dainow is not worried about making friends in the web analytics industry.

    I personally am a big fan of the Web Analytics Association.  I am pretty loyal to some of the current Board of Directors, I’ve done a bunch in the past to support the WAA and am about to announce more of the same, and I’ve even gone out of my way to help promote the work of the Web Analytics Association Standards Committee.  So it is was with great trepidation I wrote this article’s title … but I find myself agreeing with one small part of Dainow’s otherwise unnecessary rant.

    Towards the end of his article, right before he declares that some pretty nice people’s work has been little more than time wasted, he says this:

    “The WAA should be setting the agenda, not following the crowd. The task of the WAA standards committee should be to determine how web analytics metrics should be calculated in order to achieve the highest degree of precision possible. The WAA should be laying out the roadmap for the way things should be. It then falls to the vendors to bring their software into line.”

    I more or less made this same comment, although I like to believe I used a great deal more tact, when I commented on the original Web Analytics Association Standards published under the direction of former Director Avinash Kaushik back in August 2007.

    At the time I preferred to focus on the reality of the situation–the fact that the WAA had proposed a set of standard definitions that, for good or ill, were better than anything else out there.  Instead of being openly critical of the definitions as written, I preferred to ask the question, “Now that we have these definitions, what are we going to do with them?”

    While my call for a web analytics standards compliance matrix has since been answered by all of the major vendors except for Omniture, I personally don’t believe that the Standards process is serving the needs of our community as best possible.  We all continue to be vexed by a lack of standard definitions, a situation that will likely get worse with the decline of the web analytics economy.

    Not having participated in the process of drafting the WAA Standards I can only express gratitude towards those members of the community who have volunteered  their valuable time for this work.  In my humble opinion, people like Jason Burby and Angie Brown are to be congratulated for their efforts, not denigrated and accused of having set our industry back into the dark ages.

    But, in the spirit of having an open mind and building consensus, I would be interested in hearing my reader’s collective thoughts on Dainow’s point that the WAA should be setting standards without regard to their practicality today. Put another way, should the Association have written definitions that would be robust and useful in an analytics context and then presented that guidance to the entire community–vendor, consultant, and practitioner alike–saying “this is the result we should all be working towards.”

    For example, should the WAA have been more explicit in their definition of a “visit” and proclaim that a visit is terminated after 30 minutes of inactivity, instead of saying “if an individual has not taken another action (typically additional page views) on the site within a specified time period, the visit will terminate by timing out.”  Being explicit about the timeout duration would make a clear statement about our collective expectation for the definition of a visit, and any technology or analysis that choses to use a timeout other than 30 minutes would also need to justify their decision to eschew the WAA Standard for another value.

    I know that the WAA is doing the best they can, and I am enthusiastic about the work Angie, Judith and their fellow volunteers have all been doing.  But I do think Dainow’s assertion that standards should be set based on overall value to the community in the long run, not necessarily the near-term practicality, is worth exploring.  Taking this approach would definitely penalize some vendors and reduce their self-generated “compliance score” but it does kind of make sense to be working collectively towards a more precise set of definitions we can all work from.

    Doesn’t it?

    These are the kinds of conversations that aren’t just magically resolved and so I’m sure we’ll have to add this to the list of issues worthy of discussion the next time we all meet.  I’m sure it will come up at some of the upcoming vendor events, in San Jose at Emetrics, and likely at our own web analytics conference, the X Change (where last year Forrester analyst John Lovett led a conversation on the topic.)

    As always I welcome your thoughts, feedback, open disagreement, pointing out flaws in my logic, etc.  I consider myself fortunate to have such thoughtful and experienced analytics practitioners among my most loyal readers and sincerely hope Dainow’s otherwise disturbing rant will lead to something of value for our community.

    0
    Analytics Strategy

    Getting Started with Master Data Management (MDM)

    Posted on by Eric Peterson

    I stumbled across a March 2008 paper written by Mike Ferguson of Intelligent Business Strategies a couple of weeks ago that looked really interesting, and I finally got around to reading through it this weekend. First looks were not deceiving! The paper is Getting Started with Master Data Management. The link I went through required a two-page registration form, which really is unacceptable, but this post isn’t about web registration form best practices!

    It is worth it to wade through the process to get at the 22-page paper if you:

    • Are struggling with data being managed in disparate repositories at your company and are thinking that this whole “master data” thing might be worth looking into
    • Are new to a master data management initiative and are trying to wrap your head around the myriad issues
    • Are deeply embedded in a master data initiative and would like some assurances that you are on the right track while also uncovering a new nugget or two that you can apply to your work

    Some of the highlights that jumped out for me are below.

    What Is Master Data Management?

    The paper makes a very clear and clean distinction between two broad categories of structured business data:

    • Master data — the data associated with core business entities such as customer, employee, supplier, product, partner, asset, etc. regardless of where that data resides
    • Transaction data — the recording of business transactions such as orders in manufacturing, mortgage, loan, and credit card payments in banking, and premium payments and claims in insurance

    I like the distinction, as it adds some nuance to the tendency to make a knee-jerk definition that master data is “all data that might need to be shared across systems or the enterprise.”

    Early on, the paper also makes a very, very, very, VERY key point about MDM:

    …MDM is not just about data. It is about putting processes and policies in place with respect to governing master data, as well as putting services in place that provide common ways to access, maintain and manage it. MDM is, therefore, about data and the processes associated with it.

    The fundamental principle described in that paragraph is that it is a fatal mistake to assume that MDM is mostly about technology. IBM deserves a negative shout-out for their irresponsibility in perpetuating this misperception by actually naming their MDM-supporting technology “MDM.” Technology vendors in all areas tend to overplay the importance of “the tool” and underestimate the importance of “the process” and “the people.” But, still… shame on you, IBM!

    Why Is Master Data Management Needed?

    Again, highlighting a quote from the paper is useful:

    The problem this causes is that master data is fractured and partially duplicated in many different places. In addition, there is often no complete master data system of record in existence within the enterprise, except perhaps in a data warehouse where its use is restricted to analysis and reporting.

    The paper goes on to include a “spaghetti architecture” picture that illustrates where many companies find themselves when they hit the point that they need to develop a master data management strategy. And, the paper includes several lists of the “…and here’s why it ain’t as easy to solve as you might think” variety. Issues such as:

    • Managing/reconciling data conflicts between different systems — is the first name for this customer “Tim” or “Timothy?” Our web site registration system has captured his name as “Tim,” while our billing system has captured it as “Timothy.” Which one should be recorded in the “master record” for the customer’s first name?
    • How do you know if the data is complete and accurate? — in the registration form I had to fill out on bitpipe.com to download the white paper, I intentionally stated that I was at a company that generates less than $1 million in revenue every year, because I didn’t want their lead management team to pounce on the fact that I’m at a very large company and should be contacted immediately by a salesperson. At the same time, I was honest about where I work and my job title. With a little bit of research, bitpipe.com can figure out what’s accurate…but manual research is expensive to operationalize, and automated research can be expensive to develop and still be less than completely accurate.
    • If changes happen to master data in one system, how long is it before these changes  get to all other parts of the enterprise that need to know about them? — this is another good one. Customers, for instance, typically have multiple entry points to interact with you: through your billing department, through your technical support department, through  your web site, etc. Each department may rely on a different operational system. Customers expect that, once they tell “the company” that they moved, that their name changed, that they had a life event that changed something relevant, that they bought a product or service, etc., that “the company” (and all of its systems) are aware. A key goal of master data management is to make that true…but it can be inordinately complex to actually pull that off across the entire enterprise.

    This is just a subset of the questions and issues the paper lists. There are many more!

    Getting Started with an MDM Project

    The paper devotes several pages to how to build a business case for an MDM initiative. On the one hand, it’s a Business Case 101 summary — know your business, know your business’s competitive environment, build the case around relevant business value. But, it does call out some examples of where and how MDM can roll up to core business objectives.

    The unwritten piece of this section is that MDM is typically expensive, requires a longer view than “this quarter,” and requires a certain degree of underlying business transformation. It’s not a case of “buy some software and install it and then the problem is solved.”

    Scoping the MDM Initiative

    There are lots of good tips about defining the different requirements and the potential scope — short-term and long-term — for the initiative. The paper includes a good explanation for why the goal-state architecture for the MDM system should be developed early on, even if it’s not fully realized in the first iteration. For more on the high-level alternatives on that front, I recommend pages 3-8 of Colin White’s paper that SAP commissioned back in 2007: Using Master Data in Business Intelligence (although his paper is geared a bit more towards the BI benefits of MDM, the different options for handling master data and the system-of-record vs. system-of-reference distinction is laid out really well).

    The Importance of Data Governance in MDM

    The paper explains why MDM should be part of a wider enterprise data governance program, going as far as saying that, “Adoption of data governance standards, policies and technologies are an essential part of any MDM project.” That data governance program needs to include data standards (shared business vocabulary, data integrity rules, taxonomies), common policies for enterprise data management and data integration development, an enterprise data governance technology platform, an enterprise data model, and processes for monitoring and managing enterprise data quality.

    Conclusion

    I’ve touched on some of the highlights from the paper. But, for a 22-page paper, it is packed with more information than can be easily summarized in a single blog post. It’s worth a download!

    0
    Analytics Strategy

    2009 Predictions for CRM (not mine — Forrester's)

    Posted on by Eric Peterson

    There’s an interesting piece over at SearchCRM with three experts’ predictions for 2009. The first expert — Bill Band of Forrester Research — had an intriguing list of six trends he expects to see in 2009 in CRM suites. The three that really jumped out at me (the bold text is Band’s label, and everything else is my description and observations):

    • Trend 1: The emergence of the Social Consumer — okay, sure, you’d have to have had your head buried in a backhoe-assisted hole in the sand to have missed that social media is a big deal. My sister went from telling me a year ago that instant messaging was too intrusive…to being an active user and advocate on Twitter — using it effectively and with success in her role as a manager at a nonprofit. That’s anecdote 1 of 20 that I could rattle off without thinking, and I’m sure you could, too. Web 2.0 is getting steadily adopted by, well, everyone eventually, just like Web 1.0 did. Band’s point, though, is that CRM suites “will be looking to enrich the customer experience through community-based interactions, and architecting solutions that are flexible and foster strong intra-organization and customer collaboration.” That’s an intriguing thought. CRM tie-ins to LinkedIn, Facebook, Twitter, and the like? It’s a recipe for disaster if the capabilities are developed, sold, and put to use with a Web 1.0 mentality. Pulling this sort of thing off effectively is going to require some nuance, both on the part of the CRM suite providers and their customers. Should be interesting.
    • Trend 3: The requirement to fully cost-justify CRM investments — Band boils this down to four questions that every business case will have to answer, and he indicates that his contacts are telling him that every CRM investment will need a rock-solid financial case built for it given the economic environment. I wonder. Certainly, I wouldn’t expect anyone to be out spending money frivolously, but, then, I wouldn’t really expect that to be happening in a robust economic environment, either. What I’ve seen happen time and again is that a company gets along with spreadsheets, a seasoned sales force, and relationships that have been built up over years to drive the business. Then, one day, they realize that they need to actually do some marketing and actually put some processes around how they do demand generation and demand management. Someone does a back-of-the-napkin calculation as to just how much money they’re leaving on the table, and it becomes pretty clear that they need to bring in some talent, some process, and some systems to realize that value. For the companies that have already made it over that hurdle, they tend to have the information they need to see where they are falling short and to articulate — even in the absence of guaranteed immediate financial returns — the value of further investment. For companies that have a strong cash position, downturns are a good opportunity to grow their market share by selectively investing…and CRM enhancements can be a relatively low-cost investment that can pay real dividends down the road.
    • Trend 5: The need to get more value from customer information — well, this is the one that actually hits the closest to my current bailiwick. Band nails it when he says “the right approach to customer data management is elusive.” His point, though, is that companies are going to have to get smarter about how they manage and use customer information. No doubt. I don’t see this nut truly getting cracked in 2009 — more and more companies are starting to realize they need to invest in master data management (MDM) and customer data integration (CDI). That’s easier said than done. For one thing…see Trend 3 above — no company tackles this issue until they’ve painted themselves into a corner that will require a hefty price tag to get out of. There are a handful of different approaches for getting out of that corner. All of them have there pros and cons. None of them are conceptually difficult to understand. ALL of them are wayyyyyy more involved to implement successfully. I’m struggling to envision any sort of step function change in the world of CRM on that front in 2009.

    It will be interesting to look back in a year and see which of these areas really did see seismic shifts. I’m staying tuned!

    0