Capturing Web Traffic Data — Two Methods That Suck

Posted on January 2, 2008October 25, 2019 by Eric Peterson

We’re working with a client who is simultaneously deploying Eloqua for marketing automation while also switching from Urchin 5 to Google Analytics. All three of these tools provide some level of web traffic data. And, right out of the chute, the client was seeing 40% lower traffic being reported by Eloqua than was reported by Urchin 5. That raised questions…as the deployment of concurrent web analytics tools always does! Having put myself through this wringer several times, and having seen it crop up as a recurring theme on the webanalytics Yahoo! group, it seemed worth sharing some material I put together a couple of years ago on the subject.

First off, it is largely a waste of time to try to completely reconcile data from two different web analytics tools. This post really isn’t about that. Mark Twain, Lee Segall, or perhaps someone else coined the saying, “A man with one watch knows what time it is; a man with two watches is never quite sure.” The same is true for web analytics. Thanks to different data capture methods, different data processing algorithms, different data storage schemas, and different definitions, no two tools running concurrently will ever report the same results. The good news, though, is that most tools will show very similar trends. WebTrends preaches, “in web analytics, it’s the trends that matter — that’s why it’s part of our name!” But, even in the broader web analytics community, this is widely accepted. Avinash Kaushik had a great post titled Data Quality Sucks, Let’s Just Get Over It way back in 2006, but it still applies. Read more there!

This post, rather, is more the basics of “log files” versus “page tagging,” which are the two dominant methods of capturing web data. Page tagging has been much more in vogue of late, but its got its drawbacks. In the case of our client, their Urchin 5 implementation is log-based, while Google Analytics and Eloqua are tag based. And, not surprisingly, Google Analytics and Eloqua are providing traffic data that is fairly similar. But, even when two tools use the same basic data capture method, there is no guarantee that they will present identical results.

The following diagram tells the basic story of how the two methods differ (click on the image to see a larger version):

Web Data Capture Methods

“But, wait!” you exclaim! “How come both of these have ‘log file processed’ in them? I thought one method was log file-based and the other was not!” <sigh> As it turns out, both methods are, in the end, parsing log files. With page tag solutions, the log file being parsed/processed is the page tag server’s log file. In theory, your main web server(s) could be the page tag server…but then the tool would be stuck having to sift through a lot more clutter to get to the page tag-generated requests.

I’m getting ahead of myself, but go ahead and file that little bit of information as a handy cocktail party conversation…um…killer (unless the cocktail party is a Web Analytics Wednesday event — it’s all about your target audience, isn’t it?).

In a log file-based solution — the left diagram above — the “hit” is recorded as soon as the user’s browser manages to get a request for a page to your web server. It doesn’t matter if the page is successfully delivered and rendered on the user’s machine. This is good and bad, as we’ll cover in a bit.

In a page tag-based solution — the right diagram above — the “hit” is recorded much, much later in the process. The user’s browser requests the page, the page gets downloaded to the browser, the browser renders the page and, as part of that rendering, executes a bit of Javascript. The Javascript usually picks up some additional information beyond the basic stuff that is recorded in a standard web request (such as screen resolution, maybe some meta tag values from the page, and so on). It then tacks all of that supplemental information onto the end of an image request to the page tag server. The page tag server log file, then, only has those image requests, but it has some really rich information included in them.

Got all that? Well, there are obvious pros and cons to both approaches.

Log File-Based Tools Pros and Cons

The good things about a log file-based approach

They (more) accurately reflect the actual load on your web servers — your IT department probably cares about this a lot more than your Marketing department doe
They captures data very early in the process — as soon as you could possibly know someone is trying to view a page, they record it

But, it’s not all sweetness and light. There are some cons to log files that are nontrivial:

They miss hits to cached pages (by browser, by proxy) — this can make for some rather nonsensical clickstreams
They are limited to data captured in the Web server log file — this can be a fairly severe limitation if, for instance, you have rich meta data in the content of your pages and you want to use that meta data to group your content for analysis
They capture a lot of useless data — I just went to the Microsoft home page, and watched 65 discrete requests hit their web servers to render the page (images, stylesheets, Javascript include files, etc.); this is fairly typical, and means you wind up pre-processing the log file to strip out all of the crud that you don’t really care about
It is difficult for them to filter out spiders/bots — there is a “long tail” of spiders crawling the web, so this is not simply a matter of knocking out Google’s bot, Yahoo’s bot, and Baidu’s bot; there is an unmanageable, constantly changing list of known bots and spiders…and many bots mask themselves, which is extremely difficult to detect (this was actually the far-and-away biggest culprit with the client who spawned this post)

Page Tag-Based Tools Pros and Cons

Alas! Although page tags address the bigger negatives of log file-based solutions, they have their own downsides. But, let’s start with the positives:

Because they are Javascript-based, they are able to capture lots of juicy supplemental data about the visitor and the content
Most (not all, mind you) spiders/bots do not execute Javascript, so they are automatically omitted from the data
The Javascript “forces” the page tag to fire…even on cached pages

There are some downsides, though:

They requires the page tag Javascript to be deployed on every page you want tracked — even if you have a centrally managed footer that gets deployed to all pages…chances are there are still some important corner case pages where this is not the case; and, even if that is not the case now, that could happen in the future; we had a pretty robust system that was undermined when the design of a key landing page was completely overhauled…and the page tag was nuked in the process
They do not record a hit to the page until the page has been at least partly delivered to the client — if you have visitors that bounce off of your site very quickly, you may never see that they hit the site at all
If Javascript is disabled by the client, then you have to put in some sort of clunky workaround to capture the traffic…and what you capture will not be nearly as rich as what you capture for visitors who have Javascript enabled

So, What’s the Answer?

The obvious answer may seem to be to employ both approaches in a hybrid system. And, that is obvious if you or your management is so aggressively compulsive that you are willing to deploy major time and resources to try to pull this off (and very likely fail and create confusion in the process).

Let’s toss the obvious answer out then, shall we?

The answer is more simple, actually:

Understand the pros/cons of both approaches
Be clear on what your objectives are — what do you care about?
Determine which approach will more effectively help you meet your objectives and go with that

Now, if you are a Marketer, there’s a pretty good chance that you’ll wind up settling on a page tag-based solution. If that’s the case, then it might still make sense to figure out where your log files are and to do a little snooping around in them. I’ve found log files to be very handy when the page tags throw some sort of anomaly. If you can narrow down the anomaly, the log file can be a good way to get to the bottom of what is going on. Page tags…with log files to supplement. Does that sound like a tasty recipe or what?

Analytics Strategy

How Many Times Do I Have to Tell You My Name?…Redux

Posted on December 18, 2007October 25, 2019 by Eric Peterson

I seem to be a fan of redux postings. I can’t help it — it so often seems that, within a few days of making one entry, something happens that reinforces or expands the point!

Quick recap on my original post on this subject: Chris Brogan made a short video about how he wished he didn’t have to provide the same information about himself — name, phone number, address, date of birth, etc. again and again and again. Why couldn’t he just have some place — maybe even a social network like Facebook or LinkedIn — be the holder of some sort of master record with all of his contact info? My response was, “Well, it’s a lot trickier than you might think.”

Now, several different things have happened to me in the past couple of weeks that touch on the same basic issue:

When Twitter announced a 12-hour outtage, a group rallied to make a temporary shift to Pownce…and began questioning why they had to re-invite / re-accept all of their friends on each social network they joined
I realized that Jeremiah Owyang has been thinking about OpenID quite a bit
I stumbled upon the Todeka Project and their MyID.is efforts (which I don’t fully understand — they’re a little thin on content)
I stopped by REI while travelling for work to pick up something for the kids

I’ll summarize the first three like this: lots of people are thinking about the problem, some with more depth than others. There’s no easy, obvious solution.

Mainly, though, I’m going to focus on the REI experience.

Here’s a little background:

I’ve been married for 13+ years, and Julie and I have settled into some pretty clear roles and responsibilities
Shopping is not my thing; I avoid it like the loud, drunken co-worker at the company’s annual Halloween party
When I do have to buy things, I try to do it on the internet; if I have to shop in person, the checker could overcharge me by 50% and I’d be so busy mumbling and avoiding eye contact that I might not realize it until an hour or two later, after I’d recovered from the interaction (am I starting to sound like Monk?)
I have a name that falls in the second tier of “commonplace” — the first tier being reserved for the John Smiths, William Joneses, and Tom Johnsons of the world: in my high school of 400 students, there were two Tim Wilsons and two Kim Wilsons — in a school with a crappy intercom, that made for lots of confusion; when I picked up my college ID as a freshman, I wound up with the ID for “Tim A. Wilson” — he was a grad student, so I saw “Tim” and “G” (for grad student — my middle initial is G) and “Wilson.” I took it. You get the idea.
I avoid using my work contact information in any personal situation — I’m not a goodie-goodie on that front; there’s just no real point

And, doggonit , a few more important details:

I changed jobs in April of this year
I moved from Austin, TX, to Dublin, OH, in August of this year
While we have an REI membership, we have not done any shopping there — online or offline, at any location — since before the move

So, there I am at REI in Austin. I’ve pretty much wandered the whole store twice looking for three different things in the $5-7 range for each of my 8-year-old, 6-year-old, and 2-year-old. The 2-year-old was the humdinger, but I eventually settled on a silk sunglasses case with little peppers on the outside (she likes to accessorize, so it seemed like a good mini-handbag — it was a stretch, but I’d been in the store for a good 45 minutes and could feel the hives just waiting to burst out).

It was a slow night at REI. I headed to the cash register to check out, thinking, “Crap! She’s going to ask me if I have a membership and I know I do but I don’t really care and it’s only $20 or so why on earth did I think this was a good place to shop for stuff for the kids crap crap CRAP!”

Sure enough: “Do you have an REI co-op membership, sir?”

“We do, but I don’t have my card, and it’s going to be a bear to try to look it up.”

“Well, let’s at least give it a try!” (rather perkily, I might add)

<sigh>

First attempt: our old (Austin) home phone number. No dice.

Second attempt: wife’s name…in Austin. No dice.

Third attempt: my name…in Texas. No dice.

Fourth attempt: wife’s cell phone number. No dice.

Fifth attempt: my name…in Ohio. Not Cincinnati. Not Lima. Dublin? There’s a Dublin, OH one? At <my new address>??? Bingo!

But then I was curious: “Okay…so what’s the phone number on that?”

The answer: my old work phone number.

To recap: REI had our membership under my name (read background above — the chances that we actually signed up under my name are slim; it’s remotely plausible that Julie signed up under my name for some reason, so I’ll give them that) with my old work phone number (in Austin) and my new home address (in Dublin, Ohio). And, I called my wife to confirm — we had not conducted any transactions with REI since the move that would have prompted them to update our information.

So, how did they do it?

Well, I think they tapped into one of several databases that are maintained by companies whose sole value proposition is that they keep lots of information on people, and they keep that information current. Not just their addresses and phone numbers, but also demographic information. REI, despite my rather generic name, used one of those services, and the service (or REI) performed matching logic and decided that it was sufficiently confident that I was the Tim Wilson formerly of Austin who now resided in Dublin that it could update my address. But, it was not sufficiently confident to update my phone number — or it didn’t have a phone number to do an update. How on earth my old work number ever got into the system is a mystery.

All that is to say that this sort of thing is complicated. Over the course of 2007, I have had six different phone numbers that I might have provided in the course of various transactions:

My old home phone number
My new home phone number
My cell phone number
My wife’s cell phone number (see the roles above — there are times when I may initiate something, but she’s agreed to take follow-up action)
My old work number
My new work number

To further complicate things, of these six phone numbers, only one has an Ohio area code. My new work phone number is an Austin area code, and we haven’t changed our cell phones (there’s no need).

It really does make my head hurt to think about it. The Google toolbar’s “Autofill” option is a neat idea, but it only fills in the information I want it to maybe 1 in 3 times.

Interesting stuff. I’ll keep my fingers crossed.

Analytics Strategy, Reporting

Four simple rules for identifying a good metric

Posted on December 12, 2007October 25, 2019 by Eric Peterson

Avinash Kaushik — the man, the myth, the legend — had another excellent post yesterday. He titled it Web Metrics Demystified, which is a take on Eric Peterson’s Analytics Demystified (book, brand, catchphrase). Avinash has a background in data that extends into the broader world of BI and data warehousing. So, typically, his posts talk about “web” metrics and “web” data…but the “web” can be removed and you’ve got insightful thinking that is much broader than the world of web analytics.

In this post, Avinash laid out four attributes by which any metric should be judged. The metric must meet all four criteria to be a good metric — no ORs in this evaluation:

Uncomplex (or…um…Simple…but Avinash feels like the term “simple” has a “semantic implication” that he wanted to avoid) — all too often, we head down a road of trying to limit the number of metrics we’re looking at, so we combine multiple metrics into a single metric (I just saw one today: “annualized revenue by role based on current month’s revenue divided by number of people in the role and multiplied by 12″…OUCH!); or, we feel like a metric is too simple and doesn’t sufficiently reflect the nuances of our business…so we add in adjustments and tweaks that, in the end, just make the metric much harder to understand while only getting incrementally closer to an accurate reflection of reality
Relevant — it seems like this would go without saying…but it’s critical; have you ever found yourself or your company reporting on something simply because “we’ve always reported that?” It brings to mind a case at my last company where new functionality had been rolled out on the web site that was expected to offload some of the work that CSRs were doing with repeat customers; a report was established and distributed to a broad group on a weekly basis to monitor the global adoption of that feature; 3 or 4 years later, that feature was pretty much defunct…but I’ll be damned if we didn’t have someone still spending 15 minutes every Monday morning putting together that report and blasting it out to the masses! Relevancy is a slippery slope — it’s easy to think relevant means “directly links to the bottom line,” which it doesn’t necessarily need to do (see my last post).
Timely — Avinash has a great example of a company that had a query that took 3 months to run. That’s an extreme. He is also an anti-“real-time” guy, which I wholeheartedly support. Timeliness is indeed key. Somehow, I’d like to work Frequency in there, too, though. BI vendors often talk about having data that is real-time or near real-time…and then start pitching how you can check your dashboard “every morning.” This is misguided. The reason to have data near-real-time is so that whenever a user looks at it, it is as current as possible. Businesses are like boats — the bigger they get, the more room they need to turn. If you plan your Marketing campaigns on a 2-month horizon, then it doesn’t make a whole heckuva lot of sense to check your results every day! As a matter of fact, if you start making changes before you’ve let your last set of changes play out, you’re headed for a heap of trouble! But, Frequency is more a business usage of the metric than an attribute of the metric itself, so I’ll call this a side note to Timeliness.
Instantly Useful — I love this one as much as Avinash does. The challenge, in my experience, is that, when someone looks at data that is not instantly useful (read: actionable…but I suspect Avinash steered clear of that term due to its overuse), he almost never says, “I guess I shouldn’t be looking at that.” Rather, he says, “It’s not useful now…but it will be if we keep reporting it for the next few months,” or “It’s not useful now, but it’s important for me to see it.” That’s why I’m a major proponent of probing for actionability when establishing the metrics, rather than waiting until after they’ve been delivered and then seeing if they drive action. And, to be clear, “no action” is a valid action in my book, as long as it’s a conscious decision to take no action (as in, “we are hitting our target for this metric, so our ‘action’ is to maintain the status quo).

Good, good stuff that!

Analytics Strategy, General

Web Analytics Salaries 2007: Insights and Observations

Posted on December 12, 2007April 8, 2017 by Eric Peterson

A lot has been written recently on the hot job market for web analytics positions. Perhaps because there is such a profound gap between the number of open positions and the number of experienced practitioners looking for jobs, web analytics salaries run pretty high. We collected a bunch of data on salaries earlier this year and our report on the subject is available now.

Some of the key insights:

The average U.S. web analytics practitioner, across all job types (end-users, vendors, consultants, agencies) is reported to be earning $86,883 annually in salary and bonus
Those of us working for vendors are the best paid, with the average vendor practitioner earning $105,192 in salary and bonus
Unless you’re working for a very large company (5000+ employees), the difference in average salary is negligable
There appear to be three “tenure groups” for salary, which may be as much a function that people get raises by switching jobs as they do waiting patiently for a substantial salary increase
As we’ve published previously, companies relying on defined business processes are paying far better than those relying on employees or those lacking any formal approach towards web analytics
There may be hope on the horizon for those companies actively seeking experienced practitioners

Because I’m trying to write shorter reports, I’ve also created a companion web page that will allow readers to examine all of the salary data we used to produce this report, segmented over 30 different ways. Hopefully everyone will find this data interesting, regardless of whether you’re looking to hire or looking for a raise. The report and companion data are available here:

Web Analytics Salaries 2007: Insights and Observations (PDF format)
Web Analytics Salary Data

I welcome your feedback on this research — your reaction to the data as well as your opinion about the value of our publishing this research on an annual basis. I have to admit I rather enjoy being able to publish this research so hopefully everyone out there is finding it valuable.

UPDATE: I am lame for forgetting to thank June Dershewitz, James Gardner, and Dylan Lewis for spending the time to review this report prior to its publication. Thanks June, James, and Dylan!

Analytics Strategy, General

Please attend my webinar on Web Analytics 2.0 and the Web Site Optimization Ecosystem

Posted on November 28, 2007 by Eric Peterson

Thanks to Tealeaf I’m excited to be able to present a free webinar on December 11th titled “Who, What, Where, When, and Why: Understanding Visitor Interactions on the Internet.” I’ll be presenting my thoughts on Web Analytics 2.0 and discussing the Web Site Optimization Ecosystem fundamental to helping companies effectively measure and manage visitor and customer experiences in a Web 2.0 world. Plus, everyone who registers will get copy of a whitepaper I recently published sponsored by Tealeaf titled Customer Experience Management and Web Analytics: From KPIs to Customer Transactions.

When: December 11th at 9 AM Pacific / Noon Eastern
Register at: The Tealeaf web site

If you’ve ever wondered about Tealeaf and how their technology is best integrated with your existing web analytics practice I’d encourage you to attend this free seminar.

Analytics Strategy

How Many Times Do I Have to Tell You My Name?

Posted on November 26, 2007October 25, 2019 by Eric Peterson

Chris Brogan, one of the leading minds in the world of social media, just started a new job. And, he’s going through the new employee paperwork hassle (I think), which prompted this mini-rant:

http://attentionupgrade.magnify.net/item/BRCM0MCXMK2C3JG9

It’s interesting, as Chris is expressing the same frustration that we’ve all faced. WHY do I have to write in my name, date of birth, spouse’s name, etc. a gazillion times to do one simple thing?

He acknowledges that part of the reason that there isn’t a centralized, global ID that has gained widespread adoption is security concerns. And, that’s certainly part of it. The University of Texas managed to give my social security number up to some hackers several years after I graduated. They’ve now implemented much more stringent security…to the point that it’s a royal pain for me any time I need to conduct a transaction online with them.

Still, I firmly believe that people are overly paranoid on the security front. It’s a valid concern, though.

Another reason to push for a universal ID is from a data maintenance perspective. In the last six months, I’ve changed my home phone number and my home address. My job title, my name, my SSN, my mobile phone, and my work address have not changed (actually — that’s not entirely true; in some cases, my work address has changed). Various studies (none of which are close enough at hand for me to quickly track down), have shown how frequently individual and corporate data changes. Even doing the most simple of analyses of a company’s internal prospect lists will turn up duplication rates that are eye-popping (20% or higher), with no easy/obvious de-duplication options. And this is perfectly valid!

To have a universal ID, you need a universal key, and that is hard. SSN meets the criteria of being unique and unchanging…but only for US citizens. So, hardly universal.

A lot of applications try to use e-mail address, but that is wildly flawed. Chances are, you have at least two e-mail addresses — a personal one and a work one. But, there’s a good chance you have others: email forwarding addresses from your college or professional organization, “junk mail” e-mail addresses for online registrations that you think may spawn a lot of spam, or a gmail address that you need to get to your Blogger or Google Analytics account. And, e-mail addresses can change over time.

You can get people to pick a single e-mail address or ID, which, if memory serves, is what Microsoft tried to do with Passport. But, this was requiring people to remember that ID…and Microsoft was pretty weak when it came to controlling access levels for third parties to use that data. Not to mention, paranoia kicks up a notch in people who have decided that Microsoft is inherently evil (simply because they are large and successful).

In short, it’s a lot more complicated than, “I’m looking at five forms that all want me to fill out the same basic data!” Now, absolutely, an employee onboarding application should be able to make this electronic so, at a minimum, you fill in your name and phone number once. And, maybe that application could have a hook into LinkedIn to do a one-time prepopulation of the data for you to review and confirm. But, as much as companies would like it for that to be a permanent link to LinkedIn…it’s a messy, messy proposition.

And, things get even messier on the company front. Mergers, acquisitions, and spinoffs happen constantly. It is incredibly hard to track a single person through those changes. But, that’s where companies like D&B and Hoover’s, combined with customer data integration companies like Initiate Systems come into play.

Okay, so I’m rambling all over the place. But, I’ve been in this area for long enough and had enough people oversimplify the challenges of maintaining data integrity, that it seemed worth scratching the surface with a quick blog post.

Does all of this say, “There’s got to be a business opportunity here?” Well, yes and no. Anyone who tries to solve the whole problem is going to require a ridiculous level of funding and time…with a limited chance of success. Lots of companies already exist that are trying to carve out pieces of the problem and solve them.

In my mind, the best bet is to focus on standards for storing and transferring that data. I’ve heard rumblings of some attempts to come up with XML standards on that front. But, the W3C is a good example of where global standards for something “simple”…generally aren’t.

Analytics Strategy, General

The bleeding at WebTrends continues …

Posted on November 15, 2007 by Eric Peterson

Apparently statements made earlier this month after the ouster of senior management at WebTrends were not entirely true and the management shakeup is continuing. The latest casualty is the company’s Chief Marketing Officer, Tim Kopp, who according to very reliable sources has “decided to move on” from WebTrends as well, as well as several of Tim’s senior lieutenants.

Having met Tim on a few occasions I have to say this is a huge loss for a company that just sustained (or perhaps inflicted) huge losses. Having been on the company’s board of directors, Tim had an understanding of the space that was rare in this industry. His hand in the MarketingLab 2 release was apparent, and his enthusiasm for the sector will surely be missed.

Perhaps living here in Portland and knowing some of these folks personally I’m too close to the situation but I just don’t understand the logic behind getting rid of this much senior management and talent all at once. This is a tough market and starting over from scratch will be painful for WebTrends.

Regardless, it will be interesting to see what all the anonymous commenter’s who didn’t like Greg, Jason, Tore, and Hamid have to say about Tim’s departure. And perhaps more interesting to see how WebTrends spins this news for the rest of the world …

Analytics Strategy

What's Going on at WebTrends?

Posted on November 7, 2007October 25, 2019 by Eric Peterson

I got way backlogged (over a week — 150+ messages) on the webanalytics Yahoo! group. I started catching up last night, and then hit it again this morning.

This was the first I heard that WebTrends has had a major management shakeup (there’s another good article at MSN), with the CEO and a couple of other executives being removed from the company by the investors. The timing of this is really interesting, since it’s very fresh news that Omniture is acquiring Visual Sciences. And, Omniture acquired HBX/WebSideStory several years ago.

So, major consolidation in the space…and then a major shakeup at one of the few unconsolidated remaining players.

Anil Batra’s blog seems to have one of the most active discussions going on the subject. What’s interesting is that, so far, there is no emerging consensus as to what’s going on.

I was a driver behind switching from SPSS NetGenesis (groan!) to WebTrends On Demand three years ago at my previous employer. We didn’t even look at Omniture because we intentionally narrowed our scope to vendors that had the same product available in both an ASP and software model. It’s been hard to ignore the buzz about Omniture, though.

When we went with WebTrends, it was just as WebTrends was being spun off from NetIQ. I’m inclined to believe the speculation that there was one type of management needed to manage that transition — NetIQ was really weighing WebTrends down innovation-wise. And, the landscape has changed sufficiently that it may be that they simply need a different set of skills to continue.

As “Steve,” my favorite webanalytics poster put it — just because WebTrends still has the largest installed base of any major WA vendor, that’s not a guarantee that they’re here to stay. He cited WordPerfect and Lotus 1-2-3 as examples!

In my opinion — supported by various case studies and discussions in business school — mergers and acquisitions generally don’t add much, if any value. That doesn’t speak well of the acquisition frenzy Omniture has been on for the past few years. But…I still hear a lot more of interesting stuff being pulled off successfully with Omniture than I hear from WebTrends.

It will be interesting to see how things play out.

Analytics Strategy, General

My thoughts about Omniture and WebTrends

Posted on November 5, 2007 by Eric Peterson

A number of you have commented that I have been oddly quiet on the subject of Omniture planning to acquire Visual Sciences and then the news that four senior-most managers at WebTrends were let go. It’s not that I don’t have an opinion — I can assure you that I do — but I wanted to take a little time to clarify my thoughts on these subjects before blogging about it.

On the Omniture/Visual Sciences deal, I sincerely do congratulate Josh James and the entire team at Omniture on building a company capable of completely taking out their biggest competitor. Over the years I have found myself having a somewhat topsy-turvy relationship with Mr. James and his organization: First I had to compete with them while at WebSideStory, winning some deals and losing others. Then I worked directly with them while at JupiterResearch, spending time both in their offices and also on their behalf through online seminars and client events. Finally I spent a little over a year competing with them again, this time at Visual Sciences, again winning some deals and losing others.

Regardless of where I worked, it was impossible to not develop a healthy respect for Omniture and their success. It pained me to watch deals like HP, AOL, CBS Sportsline, USAtoday, Overstock.com and others go their way, despite hard work from a talented group of individuals, and I absolutely hated going up against their particular version of salesmanship. But as an analyst it was encouraging to see Josh and John Pestana build a company that understood the underlying technology but also how that technology could make their customers more successful.

Their customers responded to this, and still do. It is not uncommon to meet Omniture customers who have “drank the kool aid” and for whom their customer status is very much a badge of honor. Hopefully Mr. James et al. will deliver the same Omniture experience for the 1,500-odd companies they’re purchasing from Visual Sciences/WebSideStory, because that is where I see the inherent risk in this deal.

All week last week people with a lot of money under their management asked me “what is the upside and what is the risk in this acquisition?” I’m not a financial analyst (disclosure: I don’t have any holdings in OMTR or VSCN) so all I could comment on was what I hoped the combined company would do and not do. And while nobody from Omniture has asked me — not that it would be particularly appropriate anyway — here are a few thoughts on what the new company needs to do to make this acquisition successful:

Suck it up and start the migration from HBX to SiteCatalyst immediately. I haven’t read all of the various transcripts on this deal, but nobody I am talking to expects HBX to survive the balance of 2008 if the deal is approved (which I sincerely believe it will be.) Omniture should smooth the transition path by splitting data collection at the gateways now and simultaneously loading whatever HBX-collected data into the SiteCatalyst data collectors, thusly giving HBX customers the easiest possible transition from one technology to the other. And while if I were on HBX I would be aggressively thinking about migrating to the SiteCatalyst code base, this transition is far from a slam-dunk at the customer-level. Splitting the data will give marketing something to show I.T. if they complain about needing to replace the JavaScript (again), and getting started on data collection now will potentially ease some of the pain associated with not being able to migrate years of HBX data that some customers might not want to lose (if that is the final assessment.)
Admit that Visual Workstation is the right interface for serious analysts. Again, I have not read the transcripts, but comments I have read are unclear about whether Discover 2 or Visual Workstation will live past the acquisition point. And while I have spent much time looking at Discover 2, I can assure you that of the two, Visual Workstation is the technology to keep (disclosure: I recently entered into a licensing agreement with Visual Sciences to use Visual Workstation at Analytics Demystified.) No disrespect to Omniture’s fine product team, but Visual Workstation is unparalleled for sheer analyst-class power, and I’m fairly sure that without modification Visual Workstaion can leverage whatever format Omniture stores visitor-level data to get up and running quickly. This may cause problems from a pure SaaS-perspective, and I could be wrong, but I suspect that most analysts wouldn’t actually mind having to run the software locally in exchange for having the robust data manipulation capabilities that Workstation provides.
As painful as it will be, resolve the internal stuff quickly. A huge potential pitfall in this deal is that it has tremendous potential to create confusion regarding who is managing what, when, where, and how all of these technologies are presented in sales and support situations. Not that this will be easy, any M&A transaction has the potential to be messy, but Josh and Jim MacIntyre won’t be doing anyone favors by sugarcoating what this deal is or being vague about who might be reassigned and who might be let go (keep in mind that these companies that were bitter enemies in the marketplace up until two weeks ago.) Any internal confusion about the transition will inevitably impact customers in the form of unclear deadlines, changing account managers, and other miscommunication that will only open the door for other vendors …

Which brings me to the other change in the web analytics market last week: WebTrends announcing that Greg Drew, Jason Palmer, Tore Steen, and Hamid Bahadori had all been asked to leave the company. I have to admit, I was more-or-less shocked by this announcement, specially given that I have been saying to folks since mid-July that I believe, at least from a software perspective, that WebTrends is finally getting back on track. I really do believe that WebTrends Score is one of the few true innovations we’ve seen in the web analytics marketplace recently, and learned WebTrends users far and wide have commented that they really like the stuff in the MarketingLab2 release.

I should also say that I personally really like Greg Drew and Jason Palmer. Now, I say that not having worked with or for them in any role other than that of an industry analyst, and anecdotally some of the recent flight from the company can be tied back to their leadership. But Greg has always struck me as one of the nicest guys in the entire industry and someone who was willing to do what it took to get the job done.

Regardless of Greg’s personal disposition, I again find myself nearly flabbergasted that the folks at Francisco Partners who are calling the shots would give up Greg and Jason’s experience in the field and knowledge about web analytics in general. I mean, it’s not like Eli Shapira is going to come back and run the company, or that it will be easy to find someone else to run the ship as experienced with web analytics as Josh James from Omniture, Joe Davis at Coremetrics, or Dennis Mortensen at IndexTools. Especially on the heels of the Omniture/Visual Sciences announcement, this whole thing sounds so fishy it’s almost unbelievable, but I have to believe that these four guys will be harder to replace than people think.

Case-in-point: when Jeff Lunsford showed up at WebSideStory sans web analytics experience, some of us were worried. But Jeff was a natural born-leader, and given time it was clear that Jeff had what it took to get the job done. Unfortunately, in retrospect, it is no longer clear exactly what that job was aside from making a small number of people a huge sum of money, and Jeff has moved on to even bigger deals. I liked working for Jeff tremendously, but I’m not 100 percent sure he left WebSideStory in better shape than he found it.

I’ll admit, I don’t have the experience that these guys have … I’ve been running a company of two people for seven months. But just as I felt like WebTrends was well positioned (along with Coremetrics) to be a strong solution with a great customer base, a good set of features, that was incidentally “not Omniture”, I now find myself questioning how strong the organization will really be when run by folks largely new to web analytics. No disrespect to Tim, John, Leo or Bruce, but web analytics is hard, the competition is big and about to get bigger, and sophisticated web analytics buyers will easily differentiate between passion and experience.

Trust me, I want to be wrong about this. I would like nothing more than to have someone clarify what happened at WebTrends and detail how the company is going to accelerate growth against Omniture given their recent momentum. I think despite Omniture’s strength and Google Analytics widespread deployment that the “web analytics wars” are far from over. Like others, I worry that a two horse race isn’t very exciting to watch, and despite believing that “it’s not the technology, it’s how you use it” that it’s nice to see innovation from time to time. For this to happen, I believe we need a strong WebTrends, a strong Coremetrics, and at least a small handful of smaller innovators out there in the world (Nedstat, IndexTools, Clicktracks, etc.) nipping at everyone else’s heels.

What do you think? Am I crazy? Am I just missing the most obvious thing? Am I too close to the situation, having worked with or for all of the companies involved in the past few weeks insanity? Or do you share some of the same concerns I do? Either way, I’d love to hear what you have to say.

Adobe Analytics, Analytics Strategy

Example uses of the visitor engagement metric

Posted on October 29, 2007April 8, 2017 by Eric Peterson

My post last week on measuring visitor engagement was pretty long by the time I outlined the calculation, so I put off publishing examples of how the metric could be used until now. I’m excited to see that this topic has generated so much interest, both in terms of comments and emails sent to me directly.

My goal for this post is to provide a few examples and explanations to show how the metric can be used to supplement our otherwise already-rich set of web analytics data. Since so many folks have been willing to explore the engagement metric, I have embedded a bunch of questions in this post in italics that I’d love your feedback on.

Distribution of engagement scores and segmentation. Here is the distribution of engagement scores for about six months at Analytics Demystified by percent of visitors. As you can see, these scores are left-skewed and tail off as the score increases, showing that nearly half (47.6%) of visitors to my site are “poorly engaged”. When I look at this distribution it makes perfect sense to me — what do you think?

I have created segments to group visitors by their engagement score: “Well engaged” visitors have engagement scores over 30%, “moderately engaged” visitors are those between 10% and 30%, and”poorly engaged” visitors score less than 10%. These segments can then be used to explore how the behavior of visitors in each engagement group differs by looking at my page and referring source dimensions (page, content group, referring domain, campaign, search phrase, etc.)

Identify relationships that might otherwise not be found. At the top of this report you can see the pronounced difference in visitor engagement (and traditional metrics) for “branded” and unbranded searches (“None”) bringing visitors to my site. Now, because branded searches are a component of the calculation (Brand Index), you definitely expect to see a difference between the two engagement scores. What is interesting is that while other metrics (duration, sessions per visitor, page views per session) show a slight difference, visitor engagement and conversion are all three times higher for branded searches. I think this difference observed in all the metrics is further evidence that brand-driven searches are bringing more engaged visitors — what do you think?

In the middle table you can see search phrases bringing visitor to my site, showing visitor engagement, page views per session, and sessions per visitor. Here three phrases stand out to me:

“web analytics book” and “web analytics process”, neither of which are particularly distinguished from other search phrases based on page views per session or sessions per visitor but both of which have visitor engagement scores over double my site-wide average of 8.8%. This is important to me because these are un-branded search terms that are critically important to my business.
“vendor discovery tool” which would appear to be pretty important based on traditional metrics but only stands out slightly using the visitor engagement score (at 13.6%) I spend a lot of time trying to figure out how to drive folks using the vendor discovery tool to take other actions (buy books, inquire about consulting) and this data suggests that there is an unrealized opportunity.
“performance indicators” which shows that the visitor engagement metric is useful to identify terms that you’d think are important to the site but aren’t attracting the right audience (average engagement score for these visitors is only 5.6%)

I think this level of information is actually pretty helpful for identifying search marketing opportunities — what do you think?

Engagement-derivative metrics like “Percent Highly Engaged Visitors” are useful. Here you can see a select group of referring domains showing the percent of highly and percent moderately engaged visitors they’re sending my way (with conversion to show that engagement and conversion are in fact different!) Avinash Kaushik is sending me a few (0.2%) highly engaged visitors (thanks!) but Ian Thomas is sending me a bunch (70.4%) of moderately engaged visitors, many of whom are purchasing books (1.2% conversion rate.)

By looking at traffic from Avinash’s site over time (bar graph) I can see peaks and valleys in overall engagement from folks coming from his site, which would be useful to back into those peaks to try and determine what other blogger’s readers might be reacting to when they’re exhibiting highly-engaged behavior on my site (see late August and early September.) Given that Clint proved that conversion is a poor measure of success when trying to evaluate traffic from other bloggers, I think visitor engagement is useful for examining the non-revenue value of referring sources — what do you think?

Those of you who are looking for correlation between engagement and conversion, have a look at the data for Mr. Jim Sterne’s wonderful site emetrics.org — 5.6% of the folks coming from Jim’s site are highly engaged, 66.2% moderately engaged, and man-oh-man does Jim help sell some copies of Analytics Demystified. You’re the man, Jim!

Visitor engagement is globally useful. At least in Visual Sciences Visual Site you can apply engagement metrics and segments to pretty much any dimension tracked. Here I’m looking at the percentage of “highly” engaged visitors (50% or more) in my “well engaged” segment broken down by country. Now, this is certainly more interesting in light of the total volume of traffic coming from each geographic location, and as I think about localizing my books and planning future trips around the world this information becomes very helpful.

There is more, including some of the more granular visitor-level stuff I talked about in the first series of posts on the subject, but I want to be sensitive to protecting the identity of individual users on my site. If you’re interested in helping me collect some “ground truth” regarding the engagement calculation, write me and I’ll explain how you can help.

So what do you think? Do the screen-shots help you understand the calculation better? Or do they still make it look super-complicated and scary? Is there something specific you’d like to see me demonstrate with the calculation? Or do you think you could come up with these same insights using more traditional metrics?

Adobe Analytics, Analytics Strategy

Nick Arnett challenges my visitor engagement calculation

Posted on October 25, 2007 by Eric Peterson

Nick Arnett from MCC Media (and one of the creators of Buzzmetrics) posted a very well though-out and moderately critical assessment of the visitor engagement calculation I wrote about earlier this week. Nick makes some great points and I thought it was worth addressing them while I prepare the follow-up post that shows off some of what the metric can do. My comments are preceded by ETP and Nick’s statements are in italics.

Definitely thought-provoking, Eric… I’m deep into this issue, although focused specifically on community sites.Overall, your approach doesn’t work for me on two main counts — it is too complicated (and thus unlikely to become any sort of standard) and doesn’t generate a metric that allows different sites to be compared. The latter is arguable, since standardized weightings could yield comparable numbers, but I think that’s excess complication also.

ETP: I’m sorry the calculation doesn’t work for you but I do appreciate your thoughts on the subject. Regarding it being too complicated, compared to what? Compared to “simple” metrics like bounce rate and average page views per session? Or compared to the technology you built to power Buzzmetrics? I guess I separate the complexity of making the calculation from one’s ability to actually explain the calculation.

ETP: Regarding using this metric to compare different sites … as I mentioned in the post, I don’t think there is “one” measure of visitor engagement and thusly trying to compare sites is probably a futile effort at best. I suppose you could remove the Brand, Feedback, Subscription and Interaction indices and come up with a standard set of threshold values for specific vertical markets, but I’m not sure that is really the best use of this calculation.

Is there any ground truth behind this? In case that isn’t clear, do you have any sort of primary market data for engagement that correlates with the output of your engagement metric?

ETP: Hmmm, here I’m not sure what you mean. What kind of primary market data is actually able to identify “engaged” visitors? Because I am able to see individuals interacting with my web site, I did talk to a handful of people based on their engagement scores when I was doing the original work on this metric, and some of their feedback was critical to tweaking the metric and inputs to its current state. But other than that I’d love to see the primary data you’re talking about if you’re able to share it!

As I’ve dug into the issues and our data (about five dozen communities ranging from very large to very small), I keep coming back to two main indicators of engagement — return rates and proactive behavior. If visitors don’t visit regularly and do something other than passive page viewing, I have a tough time including them in any measurement of community engagement.

ETP: Exactly why the Recency Index and Interaction Index are included in the calculation, but I disagree with your assessment that these are the only measures of engagement. I’m not sure exactly how I would determine that someone was only “passively” viewing pages, and again this metric is not designed to be a measure of “community engagement” but rather visitor engagement more broadly considered.

Some point-by-point thoughts…

Click-depth index — this is a place where ground truth really matters, I think. I’m not comfortable with the assumption that more clicks per session means greater engagement. Do we know enough about browsing behavior to know that this is true? And of course there’s the old problem of bad design resulting in more clicks… but when I consider that issue, I tend to think that if people show willingness to click through a bad design, maybe that means they really are engaged! Perhaps we should all include some known bad design… 😉

ETP: I haven’t seen anything that says that more clicks means less engagement but I agree that confused people might generate more clicks. But I think it’s unlikely that confused and frustrated people would return, complete defined events, subscribe to blogs, etc. so despite your assertion that the metric is complex, multiple inputs are designed to mitigate those that may be confusing.

ETP: You do, however, make an excellent argument for not using something as simple as “click-depth” or “average depth of visit” as your sole measure of engagement.

I have pretty much the same questions about duration. Is there good, objective evidence that session duration correlates to engagement? There are visitors with long-duration visits who don’t visit regularly and don’t do anything proactive… I can’t see including them in any measurement of engagement.

ETP: It sorta depends on your definition of engagement, doesn’t it? But see my comment above about why a single measure like duration (as in Nielsen’s Time Spent ranking system) is perhaps inappropriate on its own to determine engagement.

Recency makes perfect sense to me — the fact that engaged visitors return often is practically a tautology. I would be very skeptical of calling anybody engaged if they aren’t returning regularly.

ETP: What about first time visitor? Are you saying you can’t be engaged on the first visit to a site? I agree that regular returns are a good indicator of engagement, but in my analysis the metric I’ve defined is able to resolve first time visitors into several engagement segments which I personally have found quite useful.

Your Brand Index is a great piece of data, but I don’t believe it works in a metric intended to compare sites. Language is too subtle and ambiguous to infer engagement from search terms. I spent years in the search engine and related markets, which gave me a great appreciation for the fact that what sometimes seems obvious about language isn’t. When people search on brand-related terms, it indicates *reach* to me, not engagement. I’m unwilling to assume anything more than brand awareness. People search on things they dislike, but that doesn’t mean they’re engaged with the subject they’re searching. And my data shows that visitors who show many other indications of engagement actually search *less* often.

ETP: Same comment about this metric not being specifically designed for comparing sites. I know that is the uber-goal for lots of folks in the world, it’s just not necessarily my goal or the best use for my engagement calculation.

ETP: Doesn’t “reach” plus “action” equal engagement? I haven’t spent years in search and related markets, but I struggle to believe that people searching on brands they dislike are not somehow engaged. Again, maybe this is a semantic issue arising from conflicting definitions of engagement.

ETP: Because the calculation is designed to be made over the lifetime of visitor sessions, searching less often is not a problem. I guess I more-or-less expect that the “direct” component of the Brand Index will be more important over time with truly engaged visitors (who wouldn’t be as likely to go back to Google and search on a branded term.)

Counting brand-related searches makes sense if we’re measuring *brand* engagement.
Counting direct (non-referred) visits makes sense if we’re measuring *site* engagement.

Counting both in the same metric doesn’t make sense to me. I don’t think we should even be talking here about ways to measure brand engagement… because I believe that’s well beyond the scope of site analytics. It requires massive monitoring systems along the lines of BuzzMetrics. (I’m the primary inventor of one of their systems.)

ETP: I’m not differentiating *brand* and *site* engagement since I’m trying to calculate an operational measure of ongoing *visitor* engagement. Brand is just a component, and the site is the measurement point. I think I understand your desire to differentiate the two given your background with Nielsen but I’m not trying to do the same thing.

One more problem with the Brand Index — people will argue all day long about what terms are appropriate to include… and there’s a strong incentive for site owners to err on the side of too many terms if their success is being measured by this metric. For example, you included “web site measurement hacks” in your list… but that could be a generic term. Is “Web Analytics Wednesday” really your brand? Or is it the WAA’s? I don’t want to argue which it is, just point out the kind of ambiguity that is inevitable.

ETP: Here I agree with you, coming up with a reasonable list is not easy, but web analytics is hard so at some point you have to make some tough decisions. “Web Site Measurement Hacks” is a book title and a branded term but could be a generic phrase. “Web Analytics Wednesday” is a branded Analytics Demystified term and has nothing to do with the Web Analytics Association. I don’t think there is that much ambiguity at the site level, at least in my experience.

Your Feedback Index is a specific instance of what I think of as the general principle of tracking proactive behaviors — what you seem to be getting at in your Interaction Index. In communities, visitors have many such opportunities — posting, editing, tagging, voting and so forth. I decided very early in this work to just give people one point for each such proactive action, despite the temptation to weight them (which would violate the need to keep things simple). These are the behaviors that make a community work; sites that aren’t based on user-generated content can exist without them.

ETP: Same comment about this calculation perhaps not being what you’re looking for vis-a-vis communities.

Your session focus really got me thinking. Does it make more sense to count the number of sessions in which visitors signal engagement or the number of actual such signals? I think it’s close to a toss-up, but so far, our ground truth suggests the latter — the number of proactive actions correlates better to our subjective estimates of engagement… but among our future tasks is to establish better ground truth. So far, I’m just using our community manager’s collective subjective scoring… but it correlates quite well to all but our smallest communities.

ETP: I agree, it’s probably a toss-up but if you think about the calculation all it does is count the number of signals. Long sessions are a signal, deep sessions are a signal, frequent sessions are signal, etc. I know you don’t like anything but recency and interaction but we can agree to disagree on this point. I’d love to hear about your “ground truthing” efforts and I’ll try and keep you appraised of mine.

The subscriber index doesn’t work for me because we want to be able to compare communities regardless of whether or not visitors are able to subscribe, join, become members or what-have-you. Some of our clients — e.g., a large professional sports organization — allow full participation without any need to sign up. Also, as I’ll explain below, I’ve found a strong negative correlation between highly active visitors and RSS subscribers.

ETP: Again, not designed for comparison (and at this point no wonder you don’t like my calculation!) I’d love to see the negative correlation data for RSS and yes, if you don’t have a subscription it doesn’t make sense to assign a negative penalty.

Finally, I guess I’ll toss out one of the ideas that I’m working with — segmenting visitors by proactivity.

In several ways, communities (and most web sites, I suspect) have a bimodal distribution of users. There’s typically a relatively large “Core” group that visits often, looks at lots of pages and does a lot of proactive stuff. There’s a middle ground, which I’m calling “Lingerers,” of people who fall into the 10th to 80th percentiles of such activities. Third and last, there’s a large contingent in the 0th percentile, people who might have one or two activities in a given time period, which I call the “Drive-bys.” In our communities, the Drive-bys are the largest group, but the Core usually is a bigger group than the Lingerers. What this says to me is that people tend to engage a lot or hardly at all — there isn’t much middle ground. I’ve been focusing on the Core’s relationship to the whole community for my engagement measurements. That’s what seems to correlate best to what little ground truth we have.

ETP: I am seeing a more normal distribution, especially as visitors return a third time, but it is definitely left-skewed towards lower levels of engagement. I’ll try and highlight this when I show some data that highlights the calculation in action. And since I’m not working on a community proper, I’ve found myself focusing on my middle group (“Moderately Engaged”) and trying to determine what I might be able to do to shift them up to “Highly Engaged”.

Overall, I’ve found that the Drive-bys and Lingers exhibit fairly similar behavior, but the Core is different. The Core visitors post more, search less and use RSS far less (so much for “subscribing” to RSS as a positive indicator of engagement!)

ETP: Your assessment of RSS being a poor indicator of engagement runs contrary to popular opinion (why would you subscribe to a RSS feed or email newsletter if you weren’t engaged??!) Perhaps this result is uncovering a flaw in your engagement calculation?

This post is getting long… so I’ll wrap it up (but ready to discuss further, of course) by repeating myself. I think any sort of engagement metric has to be backed up by demonstrating correlation to some kind of ground truth. Otherwise, it’s a mental exercise that runs the risk of having little relevance to the marketplace.

ETP: You keep coming back to the notion of “ground truth” but surely you recognize that this is A) extraordinarily difficult to come by and B) if we had it easily available we wouldn’t need a measure of engagement. I would love to see your “ground truth” data and talk about how you’re generating that, but unless I’m missing something it sounds a little impractical for widespread use. Still, I appreciate your feedback and very thoughtful comments and will endeavor to demonstrate the correlation between my calculation and “truly engaged” visitors.

Man, talk about a long post! What do you think? Is Nick more right than wrong? Are you focusing on communities and have the same concerns? Do you have similar concerns about your site? The conversation is almost as interesting as the metric and resulting analysis in my opinion so please, comment away!

Analytics Strategy

OmniVisualWebSideSciences-ature!

Posted on October 25, 2007 by Eric Peterson

Omniture just bought Visual Sciences for $394M. Congratulations to all involved. It will be very interesting to see how this plays out for clients, prospects, and the entire industry but exciting times indeed.

I sure am glad I predicted industry consolidation by the end of the decade back when I worked for JupiterResearch.

Adobe Analytics, Analytics Strategy, General, Reporting

How to measure visitor engagement, redux

Posted on October 22, 2007April 8, 2017 by Eric Peterson

Back in December of last year when I first posted on measuring visitor engagement, I hardly imagined how much interest the topic would generate. Shortly after the first post, I commented that my definition of engagement was as follows:

Engagement is an estimate of the degree and depth of visitor interaction on the site against a clearly defined set of goals.

I then went and wrote over a dozen posts, publishing feedback from some incredibly bright people and demonstrating the utility of a well-defined measure for engagement. Since that time, however, some have questioned the value of such a metric and thusly prompted me to update and publish the following calculation for visitor engagement:

I presented this calculation to a completely full room last week at Emetrics but wanted to provide an update to all my patient readers who were not able to make the event. You can download my entire Emetrics on “Web Analytics 2.0” which includes the slides on measuring visitor engagement from the White Papers and Presentations section of my site.

I very much believe that engagement is a metric, not an excuse, and that the metric described in this post provides a powerful measurement framework for sites looking for new ways to examine and evaluate visitor interaction. I know that for my own site, the use of simple measures like “bounce rate”, “conversion rate” and “average time spent” is simply insufficient for selling anything other than my books. But I’m now in the business of selling consulting, a complex and sometimes time-consuming sale, and so I’m always on the hunt for any web analytics measure that will give me an edge and help identify truly qualified opportunities.

I believe this metric is exactly that.

This post is an extension of the work I did in late 2006 and early 2007 and was written to clarify my position, update my thinking in the context of “Web Analytics 2.0”, and reiterate my desire to have an open and honest conversation with my peers and other interested parties regarding the measurement of visitor engagement. Web analytics is hard but not impossible; the same is true regarding the calculation and use of robust measures of visitor behavior.

I believe the visitor engagement measurement to be perhaps the most important of all “Web Analytics 2.0” measurements. Given that this model fully supports both quantitative and qualitative data, and given that the model is build as much around the measurement of “events” as much as page views, sessions, and visitors, I (perhaps haughtily) believe this calculation to be prototypical of the types of measurements we will see as we continue to explore the boundaries of “Web Analytics 2.0” (download my presentation from SEMphonic X Change).

The Analytics Demystified Visitor Engagement Calculation

The latest version of my visitor engagement metric, with notes about its calculation and use, are as follows. If you’re too busy to read this entire post but would like to learn more about this measure, please write me directly and we can set up a time to discuss it.

This is a model, not an absolute calculation for all sites. I agree with other analysts and bloggers who insightfully say that there is no single calculation of engagement useful for all sites, but I do believe my model is robust and useful with only slight modification across a wide range of sites. The modification comes in the thresholds for individual indices, the qualitative component, and the measured events (see below); otherwise I believe that any site capable of making this calculation can do so without having to rethink the entire model.

The calculation needs to be made over the lifetime of visitor sessions to the site and also accommodate different time spans. This means that to calculate “percent of sessions having more than 5 page views” you need to examine all of the visitor’s sessions during the time-frame under examination and determine which had more than five page views. If the calculation is unbounded by time, you would examine all of the visitor’s sessions in the available dataset; if the calculation was bounded by the last 90 days, you would only examine sessions during the past 90 days.

The individual session-based indices are defined as follows (and these are slightly updated from past posts on the subject):

Click-Depth Index (Ci) is the percent of sessions having more than “n” page views divided by all sessions.
Recency Index (Ri) is the percent of sessions having more than “n” page views that occurred in the past “n” weeks divided by all sessions. The Recency Index captures recent sessions that were also deep enough to be measured in the Click-Depth Index.
Duration Index (Di) is the percent of sessions longer than “n” minutes divided by all sessions.
Brand Index (Bi) is the percent of sessions that either begin directly (i.e., have no referring URL) or are initiated by an external search for a “branded” term divided by all sessions (see additional explanation below)
Feedback Index (Fi) is the percent of sessions where the visitor gave direct feedback via a Voice of Customer technology like ForeSee Results or OpinionLab divided by all sessions (see additional explanation below)
Interaction Index (Ii) is the percent of sessions where the visitor completed one of any specific, tracked events divided by all sessions (see additional explanation below)

In addition to the session-based indices, I have added two small, binary weighting factors based on visitor behavior:

Loyalty Index (Li) is scored as “1” if the visitor has come to the site more than “n” times during the time-frame under examination (and otherwise scored “0”)
Subscription Index (Si) is scored as “1” if the visitor is a known content subscriber (i.e., subscribed to my blog) during the time-frame under examination (and otherwise scored “0”)

You take the value of each of the component indices, sum them, and then divide by “8” (the total number of indices in my model) to get a very clean value between “0” and “1” that is easily converted to a percentage. Given sufficient robust technology, you can then segment against the calculated value, build super-useful KPIs like “percent highly-engaged visitors” and add the engagement metric to the reports you’re already running.

The Visitor Engagement Calculation in Detail

The Click-Depth, Recency, and Duration indices are all pretty straight forward and are more-or-less the traditional indicators that most people (incorrectly) call “measures of engagement”. Each of these are very important to the overall calculation, but none of these alone are sufficiently robust to describe “engaged” visitors. I set the “n” values for my site’s calculation based on the average value for each and this seems to work pretty well (meaning my Ci looks for sessions more than “5 page views” in depth, my Ri looks for sessions more than “5 page views” that occurred in the “past three weeks” and my Di is looking for sessions longer than about “5 minutes” in length.)

Brand Index is a little more complicated. Here I have made a list of all the terms I believe to be “branded” for my site and business, terms like eric t. peterson, web analytics demystified, web site measurement hacks, web analytics wednesday, and the big book of key performance indicators. Whenever a session begins either with no referring domain or comes from a search engine with one of these terms attached, I count this as a “branded session” and score appropriately. While this index perhaps unfairly weights towards search engines, I firmly believe that if you’re starting your session with either my branded URL, my name, or the name of one of my books that you are already engaged.

Feedback Index is the sole qualitative input to this model but it can easily be expanded if necessary. Here I am simply scoring sessions based on whether visitors are providing qualitative feedback via the OpinionLab “O” present throughout my web site or writing me directly by clicking a “mailto:” link. I’m not looking at whether the feedback is positive or negative, only whether feedback was given, operating under the belief that anyone willing to provide direct feedback is engaged.

The Feedback Index could easily be expanded by scoring based on the answer to direct questions posed to the visitor, questions like “do you find the content on this site valuable?”, “do you plan on calling Analytics Demystified about consulting?” and “would you described yourself as engaged with this site?” Given a sufficiently robust mechanism for making the calculation, the Feedback Index can provide a tremendously powerful input to the visitor engagement model.

The Interaction Index captures sessions in which specific “engaged events” occur other than the site’s primary conversion event — events like downloading a white paper, providing an email address, requesting a presentation or PDF, commenting on a blog post, Digging a post, emailing content to a friend, printing a page, etc. The Interaction Index is designed to capture a small weighting from those measurable goals on your site you believe to be indicative of engagement.

The Interaction Index specifically does not examine commerce transactions and other conversion events of fundamental import to the site. While I have debated this in the past, here is the rationale for recommending the exclusion of primary conversion events:

These events already have their own key performance indicator: conversion. Given that conversion is likely already defined for most transactional sites and tracked in great detail, adding conversion to the visitor engagement calculation is superfluous in my opinion.
The visitor engagement metric is designed to provide information about the large number of visitors who do not convert. Given relatively low conversion rates online, having visitor engagement be decoupled from conversion provides a cleaner measure for use in exploring non-purchaser behavior, including looking for independent correlation between the two measures.
By excluding conversion, the two metrics can be used side-by-side to look for visitor behaviors may not be obvious otherwise. Given the lifetime of possible visitor behaviors, having a way to look for well-engaged visitors who have not completed a transaction online or have completed a transaction outside of the available data set provides a critical view not otherwise readily attained.

The Loyalty Index is a reflection of my belief that repeat visitation behavior is perhaps the best measure of engagement available. Based on the distribution of visitor loyalty data at Analytics Demystified, I score “1” when visitors have come to the site more than five times in the past 12 months.

The Subscription Index is a reflection that truly engaged visitors are able to self-identify by subscribing to our blogs or newsletters; if you have taken the time to subscribe to one of the Analytics Demystified blogs I believe you to be engaged. If your site does not have some type of XML-based content subscription you can either drop this index or (perhaps better) look for an opportunity to develop a subscription service, thusly giving your visitors another good engagement point.

How Does This All Work in Practice?

Careful readers will likely have already figured out that as visitors come to your site over time, their cumulative “lifetime engagement score” changes as they satisfy the criteria of each individual index. So someone coming from a Google search for “web analytics demystified” who looks at 10 pages over the course of 7 minutes, downloads a white paper and then returns to my site the next day will have a higher visitor engagement value than someone coming from a blog post who looks at 2 pages and leaves 2 minutes later, never to return.

If you think about it for just a bit, and consider the components in the full calculation, the visitor engagement metric starts to make an awful lot of sense. Consider the following:

A visitor can quickly move through a lot of pages, getting exactly what they need, and still be scored usefully through the Click-Depth Index
A visitor can slowly and methodically read a few pages and be scored usefully through the Duration Index
A visitor can come to the site frequently and do little more than read a single page of content and be usefully scored through the Recency and Loyalty Indices
A visitor can come to the site once, subscribe to the blog, return later and download a presentation, and be usefully scored through the Subscription and Interaction Indices
A visitor can come to the site, click on dozens of pages but fail to find what they are looking for, then tell me so using my feedback mechanisms and be usefully scored through the Click-Depth and Feedback Indices

The power of the metric is appreciated when you apply it to the commonly measured dimensions found in web analytics: referring domain/URL, search engine/phrase, campaign/placement/creative, content group and page, browser/operating system, etc. Suddenly instead of looking at simple measures, you’re examining the potential of visitors coming from or going to each element in the dimension. To see the metric in action, I encourage you to read my post on the gradual building of context, at least until I’m able to publish new screenshots later this week.

Some Parting Thoughts about Measuring Visitor Engagement

Some folks have complained that this metric is “not immediately useful”, that nobody will understand it, and that it is impossible to calculate. Perhaps, but I would argue that A) no metric is truly immediately useful and B) most people don’t understand web analytics because web analytics is hard. The assumption that a diverse organization is going to be more successful using “bounce rate” because it can be glibly explained by saying “your content sucks” is just wrong — all of this stuff needs to be explained regardless of the complexity of the metrics involved.

Regarding the metric being impossible to calculate, it fully depends on which application you’re using. If you’re trying to get by using free tools then yes, you’re out of luck. But if you’re using robust tools like the high-end offerings from Unica, IndexTools, Visual Sciences, and WebTrends then you should have little trouble using the metric I describe in this post.

I personally believe that Web Analytics 2.0 both requires and allows us to be more creative and thoughtful in our use of metrics. Why not use a robust indicator if one is warranted? Especially if you’re not selling anything online, or if you’re selling high-consideration items, my visitor engagement metric can be shown to be an extremely powerful measurement.

Given the assertion that some consultants are apparently charging $200,000 USD for complex “engagement index” work, and given that someone working for Google is in the process of trying to patent a much simpler version of this equation, I am happy to give my work away to the entire industry in an effort to promote the use of more meaningful metrics to be brought to bear on increasingly complex measurement problems.

What do you think? Did you see my Emetrics presentation and still have questions? Did you read every word of my series on engagement and still not believe me? Do you need to see engagement in action before you’re willing to say it’s not just an excuse? Or are you chomping at the bit to have a robust measure like this for use on your own site?

Especially on this subject I relish your feedback, either via comments or via email — your choice! I find the subject fascinating and welcome the opportunity to discuss it you, my (hopefully) engaged readers.

Analytics Strategy

Forrester is calling all web analytics consultants

Posted on October 18, 2007 by Eric Peterson

Megan Burns from Forrester Research wrote me and asked if I would pass along the following URL to all of my readers who are doing web analytics consulting:

http://globaltestmarket.com/survey/s.phtml?sn=86482&lang=E&secid=0fd357

I took the survey myself and it only took about 10 minutes. Megan will be publishing a list of web analytics consulting firms later this year and she wants to make the list as complete as possible.

Analytics Strategy, General

Welcome Daniel Shields!

Posted on October 15, 2007April 8, 2017 by Eric Peterson

I am hugely excited to be able to announce the addition of Daniel Shields from CableOrganizer.com to the Analytics Demystified weblogs. Daniel works for Paul Holstein at Cable Organizer and is an exceptionally bright analytics practitioner, something he has demonstrated time-and-time again in the Web Analytics Forum.

Daniel joins Judah Phillips and I here in the Analytics Demystified weblogs. You can read Daniel’s introductory post and I encourage all of you to subscribe to Daniel’s feed today.

Welcome Daniel!

Adobe Analytics, Analytics Strategy, General, Reporting

Is engagement an excuse?

Posted on October 13, 2007April 8, 2017 by Eric Peterson

Blogger Avinash Kaushik kicked off a little debate in the blogosphere a few weeks when he declared:

“Engagement is not a metric that anyone understands and even when used it rarely drives the action / improvement on the website.

Why?

Because it is not really a metric, it is an excuse.”

Suffice to say, some pretty bright folks disagreed with Avinash, openly and vocally. Anil Jasra has a good summary of a panel from WebTrends Engage where Gary Angel, Andy Beal, Manoj Jasra, Jim Novo and Jim Sterne all apparently voiced their opinion that engagement is a metric, not an excuse.

Perhaps ironically, in an interview with Eric Enge from February of this year, Enge asked Kaushilk about my long series of posts on measuring engagement (emphasis mine)

Eric Enge: Another thing I read about recently was Eric Peterson’s notion of an engagement metric. Can you comment on that?

Avinash Kaushik: Sure. You know that Eric is obviously a leader in the industry. We are all following the trail that Eric has blazed. He is just an awesome guy and a really great thinker. And, in terms of the specific post that you are referring for engagement, I think Eric’s initial proposal for the methodology is a very good one, and it does extend the conversation in terms of what it is possible for us to measure, because Eric obviously has access to some pretty good tools that allow for deeper analysis. But my preference is to ask a random sampling of people, or every single person who comes to website, are you engaged, here is my definition of engagement, do you like this site or product, are you going to recommend it, or whatever is the case.

Now, to be fair, I agree with part of Avinash’s argument — qualitative data is a valuable input into measuring visitor engagement — I just don’t think qualitative data is the only input. Nor do I think that it is “nearly impossible to define engagement”. For over a year I have been calculating visitor engagement on my site using the following equation:

Looks complicated, huh? It is. But if you’re running a site like mine where the major outcome you’re trying to create is simply not measurable online, wouldn’t you like to have some reasonable proxy that would help you identify where your best leads are coming from, what those leads are looking at, and who your highest quality leads actually are?!

I know I do.

Obviously the equation above doesn’t tell you very much. If you want to hear the rest of the story, you have two options:

Come to my Web Analytics 2.0 presentation next Wednesday at 1:30 PM in the Blue Ballroom at Emetrics
Wait until next Thursday and download my updated Web Analytics 2.0 presentation from my web site

Ironically this little debate prompted me to stick the long-awaited explanation of how to measure and use visitor engagement into my Web Analytics 2.0 presentation. Thanks to Avinash for kicking off a nice (if a bit lopsided) debate!

See you in Washington!

Analytics Strategy, General

Just as I got caught up on my web analytics blog reading …

Posted on October 1, 2007April 8, 2017 by Eric Peterson

Gary Angel and the folks at SEMphonic have just launched five more must read web analytics blogs. Check out Gary’s post and subscribe to new web analytics blogs from great people like Joel Hadary, Paul Legutko, Phil Kemelor, June Dershewitz and Jesse Gross. While it’s no secret I am a huge fan of June Dershewitz, I’m very excited after the recent X Change conference to read Joel, Paul, Phil, and Jesse’s work as well.

Props to Gary for getting this uniquely talented crew to share their ideas and experiences. Even if he did call me a monolith, I still appreciate what Gary and Joel are trying to do for the entire web analytics community.

Analytics Strategy, Reporting

Inventing a Metric — Redux

Posted on October 1, 2007October 25, 2019 by Eric Peterson

Shortly after posting my last entry, there were a couple of responses:

Wandering Dave, the poster whose entry sparked the thought, complimented me on the idea (and made some additional comments/cautions regarding the topic).

And, ultimately, the original poster wrote: “The Herfindahl-Hirschman Index is very promising and I am working onhow this behaves and reflects different scenarios. I’ll report back when I have tried it out on a few more sets of data.”

I hope it works out. I continue to be fairly tickled with the idea.

Analytics Strategy, General

Stephane Hamel on Web Analytics 2.0 and 3.0

Posted on September 26, 2007April 8, 2017 by Eric Peterson

Stephane at immeria has a blurb about Avinash Kaushik’s video on Web Analytics 2.0 and my post this week on Web Analytics 3.0 that I started responding to in a comment. But as typical of me the comment got really long so I will just publish it here and link it to Hamel’s blog.

Stephane, good point that I didn’t explicitly define Web Analytics 3.0 … something for a follow-up post to be sure.

To your point:

“The Web and Internet ecosystem encompass quantitative and qualitative elements, physical and virtual organisms, online and offline interactions that are functioning together within legal, ethical and technological constraints. From that angle, things like a website, competition or location can’t, by themselves, explain the complexity of what’s going on. They can merely improve the science of analysis that will eventually lead to better insight.”

While it is difficult to disagree with you, I think you’re making the same argument that Charlene Li of Forrester made regarding her definition of engagement — she commented that engagement can be indicated at a minute level, such as when a flashy print ad catches your eye. Sure, but how the hell do you MEASURE someone noticing Charlene’s flashy print ad? And how do you MEASURE your legal, ethical, and technological constraints?

Kaushik and I are in near complete agreement about Web Analytics 2.0, and I thought he did a pretty good job explaining it. A lot of people have been saying the same thing as Avinash and I for over a year (Larry Freed pops to mind). An important distinction is that both the Web Analytics 2.0 and Web Analytics 3.0 paradigms are focused on tangible, measurable aspects of our (online) lives. And, in my humble opinion, the measures we take should be practical to make.
So I agree with you, it’s not about “e” business but rather about simply doing business, you’re spot on there. But here is the problem:

Web Analytics 1.0 was a full-on after-thought … not just for companies like yours but for the entire Internet. First we had web sites then later (more or less in 1995 if you believe most time-lines) we had measurement tools built to hack web server log files (poorly) and to try and cobble together some semblance of visitor behavior. A ton of R&D and money has gone into refining Web Analytics 1.0 and today we have JavaScript page tags and sophisticated applications that are basically still an after-thought for most companies.

Web Analytics 2.0 is also an after-thought, at least for the most part. I mean, we’ve had the qualitative data in systems like ForeSee Results and Tealeaf for years, so why is it only now that we’re actively talking about combining these data into a more holistic view of the visitor? We’ve had multivariate testing systems like Offermatica and SiteSpect for years, so why is it only now that we’re actively talking about using the combination of qualitative and quantitative data to drive action? (FYI, you can download my Web Analytics 2.0 presentation from my web site if you’re interested in more of my views on the subject …)

So I guess what I’m getting at by talking about Web Analytics 3.0 at this early stage is this:

Wouldn’t it be nice if the global solution to measuring the inevitable state of “digital ubiquity” wasn’t another after-thought?

Wouldn’t it be sweet if the platform providers and device manufacturers, the standards bodies and the compliance police, all came together now instead of 10 years from now and asked “How in the world will we measure all of this?” Personally, I think so, that’s why I’m starting the conversation more-or-less five years ahead of time, so that this time we’re not all standing around trying to figure out how to answer good business questions using incomplete and inaccurate data.

Call me crazy …

So yeah, I am probably still right and wrong. And yes, you make a good point — Kaushik and I were both caught navel-gazing (again!) But if in 5 years you and I are banging around in the Yahoo! group asking people whether the “Nokia X5150J Revolution” accepts cookies and JavaScript I am going to be awfully put out, aren’t you?

Thanks very much Stephane for offering up an opinion other than “Eric and Avinash are both brilliant!” The ego stroking is great but this kind of stuff needs to be debated, openly and honestly in my humble opinion. Beers are on me in D.C.

Analytics Strategy, Reporting

Inventing a Metric

Posted on September 25, 2007October 25, 2019 by Eric Peterson

As I’ve written about on many occasions, I’m a huge proponent of establishing clear objectives before trying to come up with metrics. I’ve always claimed that, with this approach, you really get freed up to come up with truly relevant metrics, rather than settling for metrics that are what you’ve always measured or metrics that are easy to get at.

An interesting example came up on the webanalytics Yahoo! group yesterday. The original post:

I use web analytics mainly in the context of natural search. One thing that I always want to measure is `landing page diversity’ and would like to know if anyone can help me with a metric, or metrics, tomeasure it.

To explain a little more fully: if a site has 10 pages, then at one extreme all visitors enter through the home page (zero diversity) andat the other extreme 10% of visitors enter through each page (100%diversity).

But how do I put a number to all the variations in between, in effect differentiating between sites in which a few landing pages attract most of the traffic and ones where a higher proportion of internal pages attract traffic.

Any help greatly appreciated.

While this post doesn’t explicitly state objectives, there is clearly some real thought and rationale behind what the poster is trying to look at.

A very frequent (and high quality) member of the group, “Wandering” Dave Rhee, responded, and his response triggered a thought that I posted (Dave coined an acronym — SMP, for “Smart Math Person,” which I then referenced in my response). Frankly, I was pretty tickled with the idea. My response:

I am NOT one of the SMPs on this list. But, WDave’s second, simpler thought somehow knocked down a small, distant, cobweb-encrusted door inthe remote reaches of my brain.

It got me to thinking of the Herfindahl-Hirschman Index (HHI), which many economists use as a way to measure competition in an industry (aswell as to quantitatively determine when a monopoly exists). The formula is pretty simple: you take the market share of each of the companies in the market, square it, and then add them all together. Actually, you usually sort in decending order of market share and then square and sum the top n companies’ market shares. An HHI of 1 is a perfect monopoly. An HHI that approaches zero has extreme diversity/competition.

It seems like this approach might work as a simple, yet valid, landing page diversity metric. Each landing page has a % of the “market” of entry to the site. Square that percent for each page and then add them together (within reason – the impact on the HHI steadily decreases…maybe even exponentially or logarithmically, but I’m no SMP- the farther you go down your list).

More details on HHI at: http://en.wikipedia.org/wiki/Herfindahl_index

There has not been any further discussion of the topic since my post, which can be interpreted any number of ways. My reason for reposting it here is really to illustrate how open you can and should be to establishing metrics once you know what it is you care about. The webanalytics group is a great forum that many members use for just this purpose. For instance, there’s a very active thread going on right now debating how best to measure “proactivity” in social media.

I’m a fan of the approach!

Analytics Strategy, General

Web Analytics 2.0? I am more worried about Web Analytics 3.0!

Posted on September 25, 2007April 8, 2017 by Eric Peterson

If you’re reading the web analytics blogs, you’ve probably already heard about the recent presentations I’ve given on the subject of “Web Analytics 2.0”. The future of web analytics and the relationship between Web 2.0 technology and measurement is something I’ve been talking about for over six months — I actually have a Web Analytics 2.0 workshop that I regularly give that you can read about under Analytics Consulting on my site — but given that it is “conference season” it is no wonder that this subject is getting attention from other folks in the industry. I have given my presentation at Web Analytics Day in Brussels, SEMphonic X Change in Napa, and will be giving a variation on same at Jim Sterne’s Marketing Optimization Summit in October.

Due to demand, you can download a PDF of the presentation from the white papers section of my site. If you’re interested in learning more about Web Analytics 2.0, please give me a call and I’d be happy to discuss it with you.

Strangely enough, the slides that are generating the most interest and commentary are not those about the Web Site Optimization Ecosystem, the integration of quantitative and qualitative data, or the Analytics Demystified RAMP, but rather the few slides I included outlining my thoughts about Web 3.0 and what I am calling Web Analytics 3.0.

What the heck is Web Analytics 3.0?!

Before I can tell you what Web Analytics 3.0 is, I need to tell you what I think Web 3.0 is going to be. The good old Wikipedia basically dodges this by saying:

Web 3.0 is a term that has been coined with different meanings to describe the evolution of Web usage and interaction along several separate paths. These include transforming the Web into a database, a move towards making content accessible by multiple non-browser applications, the leveraging of artificial intelligence technologies, the Semantic web, the Geospatial Web, or the 3D web.

While I know that Judah is all hopped up on the notion of the semantic web, after having traveled to Tokyo and Europe in the past month, I find myself absolutely convinced that the next technology era will be characterized by our collective ability to access the Internet anyplace, anytime, using so many devices we begin to look back on computers much the same way young people do television today — as something nice to use when YouTube is unavailable. Rolf Skyberg, a disruptive innovator from eBay who I met in Rotterdam a few weeks back, called it “digital ubiquity” — the point where we forget that the Internet actually exists and take our ability to access information completely for granted.

Given so many sexy alternatives — 3D web, transforming the Internet into a database, artificial intelligence, and the such — why am I so convinced that in the next three years we’ll be talking about Web 3.0 when we talk about mobile phones and non-traditional browsers?

Easy. The financial opportunity available via the mobile Internet makes the billions transacted today look like pocket change.

Think about it:

Most people in the U.S. haven’t seen QR codes in advertising yet, but they will.
Most mobile users around the globe don’t have access to a brilliant browsing experience through their phones, but they will.
Most marketers aren’t advertising on mobile platforms yet, but they will.
The most-loved company in the world isn’t in the mobile market yet, but they will be.
Most mobile platforms aren’t passing the phone number (or a derivative) along with HTTP requests, but they certainly could.
Most mobile platforms aren’t passing along GPS coordinates along with HTTP requests, but they certainly could.

Just think for a minute about how your browsing experience might change if the web sites you visited remembered you and delivered a tailored experience based on your demographic profile (theoretically available via your phone number), your browsing history (accurate because you’re not deleting your phone number) and your specific geographic location when you make the request?

Now think about how the advertising buying experience would change if the same were true, not to mention behavioral targeting. I mean, given GPS and demographic data, the behavior being tracked could be “works downtown during the day, checks Facebook on his phone often, lives in the suburbs, surfs sports scores from his neighborhood bar.” The Starbucks web site could have a link at the top with a coupon to save $1 on my double-tall non-fat latte in stores 1 block, 2 blocks, and 5 blocks from my current location; the Best Buy web site could have an in-store promotion for the store I am standing in, targeted to my age and gender; and my search engine could disambiguate my searches based on my demographic profile, my geographic location, and my recent search history to serve me paid search ads designed to influence my geo-spatial movement, not just my likelihood to click.

Jeepers, huh?

Sure there are privacy issues, but given the intensely personal relationship most people have with their cell phones, and the fact that far more people in the world have mobile phones than computers (Gartner estimates 271 million units sold to end-users by Q2 2007) it is easy to make a convincing case for mobile computing and digital ubiquity defining the next technology era, much like social networking, AJAX, XML, and mashed-up business models define the current Web 2.0 era we’re living in today.

Okay, mobile is the future. So what the heck is Web Analytics 3.0?

If Web Analytics 1.0 was all about measuring page views to generate reports and define key performance indicators, and if Web Analytics 2.0 is about measuring events and integrating qualitative and quantitative data, then Web Analytics 3.0 is about measuring real people and optimizing the flow of information to individuals as they interact with the world around them.

Your log file analyzer can do that, right?

The current state of mobile measurement isn’t about Omniture and Visual Sciences, it isn’t about JavaScript and cookies, and it isn’t about page views, visits, and visitors. Web Analytics 3.0 is going to be something completely different, and it will depend on completely new technology. Anil Batra and I talked about a project he did a few years back while he was at digiMine — he hacked together WAP gateway logs into a pseduo-log file, using the phone number in place of a cookie. Brilliant, and the fact that Anil has this experience propels him to very near the head of the class for Web Analytics 3.0 analysts.

In theory, the mobile Internet has many of the same measurements as the hard-wired Internet. But as the information the platform and device providers make available changes, something I very much believe will happen, the quality and volume of information at our disposal will increase and improve. The W3C document on “Mobile Best Practices 1.0” already exists but surprisingly enough don’t have a section about logging requests or measuring user interaction. M:Metrics is out there providing analyst reports, but the service is more similar to comScore and Nielsen than WebTrends and ClickTracks.

This post is already extremely long but I wanted to start the conversation. In future posts, as time allows, I’ll expand on some of what I believe is possible and how. In the interim, let me know what you think! Am I wrong? Is Web 3.0 bigger than mobile? Or do you already have a handle on measuring your mobile content, even without GPS and phone numbers as unique IDs? Do you personally have experience doing analysis on mobile content? If so, I’d love to hear about your experience.

As usual, I very much welcome your comments but am happy to receive your comments directly via email. Also, if you’re a mobile service provider or device manufacturer concerned with how advertisers and marketers will measure their success through your platform, application, or device, I would love to talk to you about the Analytics Demystified vision for Web Analytics 3.0.

Adobe Analytics, Analytics Strategy, General

World Tour Day Five: Brussels

Posted on September 17, 2007April 24, 2017 by Eric Peterson

On my fifth and final (working) day in Europe I had the distinct pleasure of presenting at OX2’s Web Analytics Day in Brussels, Belgium. I had been working directly with Rene and Aurelie to promote this event as our companies have a business partnership and I am very happy to say the event was a tremendous success with over 100 people in attendance.

I presented three times covering my now well documented “Web Analytics is Easy?” presentation, the subject of key performance indicators, and my newest presentation covering what I am calling “Web Analytics 2.0” This third presentation worried me a bit since it is more technical than the stuff I usually present but it was very well received according to everyone I talked to. Aside from my work, there were great presentations from Aurelie, David Rhee, and a handful of vendors including WebTrends, Omniture, NedStat, IndexTools, and of course the now famous Microsoft Gatineau.

Continuing my theme of finally getting to meet some very bright folks in Europe, here are a few I was smart enough to get photos of while at Web Analytics Day.

Rene Otamendi from OX2, Marianina Chapin (the Web Analytics Princess) and “Wanderin” David Rhee, super-moderator of the Web Analytics Forum, blogger, and the newest OX2 employee in Germany.

Marianina, her friend from Great Britain (whose name I am forgetting, forgive me!) and Michael Notte from Toyota Europe who gave a nice presentation on behalf of WebTrends.

Luc Peeters, a consultant for Digitmedia in Belgium, who was very nice to talk to and paid me a very high compliment, saying my presentation was better than one he had seen from Nicholas Negroponte of the the MIT Media Labs. Very kind, indeed, if perhaps a wee stretch of the truth …

My best friend in Italy, Giovanni Lorenzoni, who writes an Italian web analytics blog at blog.webanalytics.it. Giovanni has a lot of enthusiasm for web analytics and is excited about how things are shaping up in Europe of late.

I was also sent a handful of photos from my presentation in Rotterdam at eDay. Now, I’m a bit camera shy normally but you can get an idea of what eDay was like via this photo of me (probably asking the audience if they thought web analytics was easy or not)

The behind-the-ears microphone was classic and made me feel like I was on American Idol.

All in all I have to say I very much enjoyed my European trip. Thanks to great people like the folks at SATAMA and OX2, the nice people at eDay, and all the great fans of my blog and my writing that I met all across Europe proving positive feedback about the work I’ve been doing since I left Visual Sciences four months ago.

I hope to get back to Europe this coming Spring, and hopefully I’ll be able to bring my wife and spend a few days actually relaxing and enjoying all the beautiful scenery that I had to rush by this time. Until then I’m off this afternoon to sunny Las Vegas for the Advanced Analytics session at Shop.ORG then on to SEMphonic X Change where I will be giving the keynote speech. If you’re at either of these events please come and introduce yourself!

Analysis, Analytics Strategy

A GIANT in web analytics says, "Don't get your hopes up…"

Posted on September 17, 2007October 25, 2019 by Eric Peterson

Avinash Kaushik has a great blog post about trying to do predictive analytics with web data:
“Data Mining And Predictive Analytics on Web Data Works? Nyet!”

Avinash is one of the truly brilliant minds in web analytics, so it’s great to see him put his brainpower behind explaining this assertion. And, it’s timely, in light of the new book by Ian Ayres, a Yale Law School professor and econometrician. I really need to order the book and read it, as I’ve got preconceived notions based on watching an interview with Ayres. Fortunately, I’ve got a B&N gift card and we’re B&N members, so get an additional discount. Hmmm…my link above is to Amazon…yet I’m going to buy through B&N. Why is that? Topic for another post…on someone else’s blog, I suspect!

For now, I’m going to view these two sources as representing two schools of thought / approaches to data. On the one hand, we actually have, oddly enough, highly trained statisticians, academics, AND casual users of business data. This is a group that, overly simplistically, sees “more data is better,” albeit for different reasons. The statisticians want more data so they can find increasingly subtle correlations in the data. the casual users want more data because, like revenue, more data is better, right?

The other school of thought is a bit more grounded in reality. As I’m prone to do on this blog, I’m once again blatantly showing where I fit. This school of thought recognizes that more data brings along the need for more discipline — more *business* discipline — to actually get actionable information from the data.

Now, I’ve got to go and see if I can figure out how to use a BN gift card AND get my discount through their online site.

Analytics Strategy, General

EXCLUSIVE Microsoft Gatineau presentation and screen shots!

Posted on September 14, 2007April 10, 2017 by Eric Peterson

UPDATED: Apologies to Ian Thomas but I forgot to indicate where BETA 1 screen shots stopped and where future releases began as he related this information during his presentation. Please see the note inline below and again, apologies to Ian!

Ian Thomas from Microsoft was in Brussels, Belgium today showing off some screen shots of the upcoming Microsoft Gatineau product. The following are exclusive screen shots of the product, sans commentary per my agreement with the ever-so-nice Mr. Thomas. If you’d like to be considered for the Gatineau BETA, please either visit Ian’s blog at http://www.liesdamnedlies.com or fill out the form at http://advertising.microsoft.com/gatineau

Thanks again to Ian for letting me bring these to light and best of luck with the BETA launch!

UPDATED: The following screen shots include visualizations that Mr. Thomas indicated would be in future Gatineau releases.

Again, if you’re interested in signing up for the Gatineau BETA, please go to http://advertising.microsoft.com/gatineau. Any questions about these screen shots or the product images should be addressed to Ian Thomas via his weblog (http://www.liesdamnedlies.com)

Adobe Analytics, Analytics Strategy, General

World Tour Days Three and Four: Rotterdam and Brussels

Posted on September 13, 2007April 8, 2017 by Eric Peterson

I finally got a little time to relax on Wednesday when I arrived in Rotterdam, Holland, the site of Emerce’s eDay conference where I presented my “Web Analytics is Easy” presentation. The event planners and the City Council of Rotterdam invited the event speakers out on a nice boat trip. I was lucky to connect with serial entrepreneurs Mark Fletcher (Bloglines, Startupping.com) and Biz Stone (Xanga, Google, Twitter), two tremendously nice guys and both serious movers in the Bay Area Internet scene.

Biz and I had a pretty good conversation over dinner about “Web 3.0” and I was also lucky enough to catch up with Rolf Skyberg, Disruptive Innovator at Ebay, on the same subject. What is “Web 3.0” you ask? Come to SEMphonic X Change and find out!

The only web analytics vendor at the conference was Holland’s Nedstat but it was nice as I got to chat with the company’s CEO and VP of Marketing. I was also surprised when one of their newest consultants produced a well-used copy of Analytics Demystified which he asked me to sign.

My presentation at eDay went well I think. The Dutch people in attendance seemed very engaged despite (or perhaps because of my very non-European presentation style) and many stuck around after the session to ask questions. The woman in the back (standing) told me that she believed me when I said that web analytics was hard. I think I said something like “Hallelujah!”

After a quick Internet radio interview, my good friend Aurelie Pols and I had a nice drive to Brussels to the Global Headquarters of OX2. It was my first visit to Brussels and to meet many of the employees of my partner in Belgium and it was quite fun. Here is Aurelie with her newest web analytics specialist (whose name I didn’t write down, please forgive me!)

Here is Aurelie’s husband and the President of OX2, Rene Otamendi (right) with the infamous W. David Rhee, OX2’s newest employee, who is perhaps best known for his incredibly valuable contribution to the Web Analytics Forum.

I am hugely excited about Web Analytics Day here in Brussels. Not only will Ian Thomas be on hand to show off Microsoft’s Gatineau application for the first time in public, but after nearly 150 requests to attend Rene had to stop registration due to space limitations. While we’re obviously disappointed that everyone cannot join us, it is very encouraging that so many people are interested in measurement here in central Europe.

For those of you who have grown tired of reading about my jaunt about Europe, fear not! Tomorrow will be my last post on the tour and I promise to not blog next week from Las Vegas.

Analytics Strategy, Conferences/Community, General

World Tour Day Two: Stockolm, Sweden

Posted on September 11, 2007April 8, 2017 by Eric Peterson

I just got back from the special Web Analytics Wednesday event in Stockholm, Sweden and wanted to post some thoughts and a few of the pictures I took.

When I first founded Web Analytics Wednesday I wasn’t really sure what to expect. Like I’ve posted in the past, I started the event simply because it seemed like a really good idea — not to make money, not even necessarily to sell books, really just to bring the entire web analytics community together.

Now that I’ve seen what Lars Johanssen with help from IndexTools, WebTrends, and Omniture have built here in Sweden I have to say I am completely blown away!

Lars, with the help of his sponsors providing some food, some drink, and a nice place to gather and present, manages to draw the largest Web Analytics Wednesday group in the entire world almost every single time they get together. And while Lars does this for the betterment of the Web Analytics Association, when he asked the group, less than half of the 70 or so attendees were even WAA members! Lars has tapped into a rich community of very interesting (and interested) business people, all of whom appear quite passionate about learning more about web analytics.

It would be great if more cities around the world could reach out, get sponsors, plan activities, and really push to make their local Web Analytics Wednesday events something that people are excited to attend. I’ll ask Lars next time we talk, but I suspect it doesn’t really take that much effort on the part of the local organizer!

Let me ask you this: What would make you personally more willing to host, sponsor, or attend a local Web Analytics Wednesday event? Is it more support, more recognition, more formal education, or simply more free food and drink? Please leave me your thoughts in the comments or email them to me directly and I’ll summarize later this week (from Holland or Belgium!)

After seeing what Lars has put together, I am personally more motivated than ever to help the entire web analytics community benefit from Web Analytics Wednesday, regardless of your affiliation, location, or motivation! Please do let me know!

Okay, here are the pictures from Day Two of my web analytics world tour …

About half of the attendees, including Mr. Fulton Yancy from Visual Sciences (front left) and Lars Johansson waving in the back (red blob, sorry about the pictures this time!) I’m not entirely sure why Fulton didn’t join the panel and why Visual Sciences declined to sponsor this event despite generous funding from WebTrends, Omniture, and IndexTools.

Google’s Dr. Brian Clifton (left), who flew in from London for the event and three unnamed Swedes.

The vendor and consultant panel. From the left, Dennis Mortenson of IndexTools, Steve Jackson from Satama, the guy from Omniture whose name I cannot begin to pronounce, Ian Tickle from Webtrends, and Per Strid of Sweden (not on the panel.)

Ian Tickle and Per Strid. I had heard a great deal about Mr. Tickle but had never met him. He turned out to be incredibly nice, thoughtful, and polite. He actually even cited an Omniture TouchClarity case study when an audience member asked about the value of optimization technology!

Anyway, despite having had almost no sleep since I got to Europe, and it being 3 AM as I finish this post, everyone seemed to enjoy my “web Analytics is Easy” presentation and I heard several people talking about the Analytics Demystified RAMP over beers. Thanks again to Lars Johansson for inviting me to Sweden and further motivating me to help foster WAW events around the globe!!!

Analytics Strategy, Conferences/Community, General

World Tour Day One: Helsinki, Finland

Posted on September 10, 2007April 8, 2017 by Eric Peterson

After a long flight from Portland > Minneapolis > Amsterdam > Helsinki I finally arrived at my first destination in Europe: Global headquarters of Satama, a leading European digital services company with a global clientèle and an energetic, insight-oriented crew and authors of Captain Blackbeak’s Blog (arrrrr!) I didn’t mention Satama in my last post because the event tomorrow is private for a select list of roughly 70 Satama clients and prospects.

I’ve known some of the Satama crew for awhile now online but it was nice to put names with faces! Here are a few of those faces so you can play along at home:

Mikko Isoniemi, Business Unit Director for Analytics, and Steve Jackson, Senior Consultant for Analytics

Janne Korpi, Project Manager and Team Leader for Analytics

Many of the Satama crew, including Mia Luostarinen (front left) who provided invaluable help getting me to Helsinki!

I’m very much looking forward to meeting Satama clients tomorrow at the presentation, after which Steve, Mikko, and I will be heading directly to Stockholm, Sweden for Lars Johansson’s totally sold-out Web Analytics Wednesday event!

Analytics Strategy, General

One of the best decisions I made this year …

Posted on September 3, 2007April 8, 2017 by Eric Peterson

(I wrote this post a week or two ago and had WordPress issues so am re-posting it. Sorry about that Judah!!!)

One of the smartest things I did in 2007 so far was not quitting my job (although I do very much enjoy the independent life and my clients are all great!) but rather asking Judah Phillips of Reed Business Interactive to blog here at Analytics Demystified.

I met Judah just after I left JupiterResearch in Boston and was immediately impressed with his expansive knowledge of the web analytics sector, his candor, and his communication skills. We didn’t talk for awhile but reconnected while I was at Visual Sciences, picking up the conversation and getting to know each other better. This time he blew me away with his ability to sift through the noise and make an incredibly complicated software buying decision, all while being exceptionally honest and candid about his process.
After his selection was made, he and I kept up the conversation and were talking with some regularity at the same time I was thinking about expanding on Analytics Demystified’s blogs and adding other voices to my site. I was especially looking for practitioner voices. Judah was perfect, and he was open to the idea.

The rest, they say, is history!

Judah is a big thinker and a ton of fun to spend time with (alas, he lives in Boston, although Alaska Airlines is starting to fly direct to/from Boston later this month!) If you’re like me and not lucky enough to be part of his Boston Web Analytics Wednesday group, I strongly recommend that you start reading and subscribe to Judah Phillips today!

I just wanted to publicly say “thanks” to Judah for exceeding any expectations I had about his writing and his commitment to sharing his knowledge and experience with his readers and mine. Now if only all the decisions I had to make as a small business owner were that easy!

Analytics Strategy, Conferences/Community, Reporting

Congratulations to the WAA Standards Committee!

Posted on August 28, 2007April 8, 2017 by Eric Peterson

I wanted to say congratulations to Jason Burby, Angie Brown, and everyone on the Web Analytics Association’s Standards Committee for publishing their standards document last week. Given the number of web analytics terms they defined (26) and the somewhat slow process the Association has for getting documents approved, this effort is a huge milestone for the organization, one that Jason and Angie deserve great praise for indeed!

If you haven’t already downloaded and read the definitions, check them out here (PDF download).

While the PDF document says that the final product is “Web Analytics Definitions – Version 4.0” this is clearly a “Web Analytics 1.0” document. The committee relegated all of the really wonderful Web 2.0 stuff like AJAX, RSS, XML, and the such to the same confusing obscurity they exist in today with the comment “certain technologies including (but not limited to) Flash, AJAX, media files, downloads, documents, and PDFs do not follow the typical page paradigm but may be definable as pages in specific tools.”

Given the last year’s push towards measuring Web 2.0 the right way and some great, insightful work from folks like Ian Houston and Judah Phillips it is kind of a shame that this document doesn’t address event-based measurement architecture more directly. The group does define “event” but only does so under the header of “Conversion Metrics” stating that an event is “any logged or recorded action that has a specific date and time assigned to it by either the browser or server.

Sounds like the definition of a Web 2.0 event to me, but I’m not sure why this is relegated to conversion metrics.

Regardless, this is great and valuable and useful work on the part of these hard-working volunteers. But the definition of standards raises one particularly important question: Given the definition of standards, what the hell do web analytics practitioners do with them?

The Fundamental Problem

The fundamental problem with these definitions (and any standard definitions IMHO) is that without an enforcement mechanism they are unlikely to provide any real benefit to the folks in the trenches. As long as smart folks like Eric Enge at Stone Temple Consulting continue to uncover as much as a 154% difference in the measured number of visitors and a 161% difference in the measured number of page views between concurrently deployed solutions, the average web analytics end user should not be comforted by the existence of standards.

Put another way, it is not the definition of standards that makes a difference, it is the adherence to standards by technology vendors that will provide the portability of skills, knowledge, and solutions so desired by many in our industry. Jason Burby sagely points this out in his Clickz article on his volunteer work when he says:

“Companies often switch metrics tools and subsequently change the terms they use to discuss analytics. One tool will call something one name, while another tool calls it by a different name or applies different meanings to a very similar name. When people switch tools and bring data with them, they don’t get an apples-to-apples comparisons. As a result, companies lose the important year-over-year view.

Though the new standards won’t instantly take care of that issue, they provide a step in the right direction.”

The Barrier to the Adoption of Standards

The problem as I see it is this: For many web analytics vendors, the way they calculate some of the critical metrics in web analytics is the “secret sauce” in their solution. Consider the WAA’s definition of unique visitors which states that unique visitors are:

“The number of inferred individual people (filtered for spiders and robots), with a designated reporting timeframe, with activity consisting of one or more visits to a site. Each individual is counted only once in the unique visitor measure for the reporting period.”

This is perfectly reasonable, but the definition goes on to say that “a unique visitor count is always associated with a time period (most often a day, week, or month), and it is a non-additive metric.”

Do you wonder what the folks at Visual Sciences who have spent millions to perfect their “data wheels” technology that effectively removes the “time period” requirement would say to this? One of the major value propositions at Visual Sciences (at least during my brief tenure) was that time was irrelevant — if you wanted the number of unique visitors for the football season, you dragged your mouse across the calendar; if you wanted the number of unique visitors for a few hours during the day, you dragged your mouse; if you wanted the number of unique visitors to your site since recording began, you dragged your mouse.

You can make the case that this example more or less removes the time dependence associated with the WAA definition. But should all the vendors who don’t have this capability (anywhere you are forced to use metrics like “Daily Unique Visitors”) spend the R&D money necessary to eliminate the dependence on time? Or should Visual back this functionality out of their application?

When you start to think about these kinds of things, much less issues associated with data sampling and data roll-off that occurs for a litany of reasons, you can start to understand why I made this somewhat snide comment in a MediaShift article awhile back:

“A friend of mine described it as the most beautiful fantasy…but it would never happen,” consultant Peterson said. “Omniture has a $1 billion market cap, and I don’t see Omniture tearing apart their technology to calculate unique visitors and page views differently because all their competitors have decided there’s a different way to do it. It’s hard to imagine. Not impossible. Fantasies sometimes come true.”

Ironically the cost isn’t the main problem: The impact on existing customers who would be forced to learn new definitions and suffer from potentially dramatic changes in data collection and reporting is the main problem. Do you want to be the person who has to tell a Fortune 500 customer that because you’re adopting more standard definitions that their page view count will suddenly drop by 35% month-over-month?

I had to do that once. Trust me here, it wasn’t a fun conversation to have.

An Idea in the Absence of a Solution

Given that I think that the WAA has produced some incredibly valuable work, despite some potential barriers to the work’s adoption, I do have an idea that I would love to see the Association follow-up on, one that would add a tremendous amount of value to this already great work.

I would love to see the Standards Committee create a matrix of standards compliance for each of the vendors in the marketplace today. Basically a checklist that details on a term-by-term basis which vendors are currently using the WAA definitions that would let companies looking for a solution to include that criteria in their assessment. Something that would let everyone quickly determine:

How standards compliant a given solution is (and which solution today is “most compliant”)
Which standard definitions are calculated out-of-box in each solution (for example, “Original Referrer” and “Bounce Rate”)
Which currently available solutions dramatically differ from the norm in their use of standard terms

Something like this would probably have to be backed up with some documentation or examples as proof points, just for reference. And yeah, this is kind of a lot of work, but if you think about it all you really need is for one WAA member per solution to poke around in their documentation and then someone (Jason and Angie maybe) to collate the results and write it up. I would be happy to contribute the matrix assessment for the web analytics solution I’m using now if that would up!

Who knows, maybe we’d discover that all the vendors are already standards compliant and there really isn’t a problem with definitions!

What Do You Think?
I’d love to hear what all of you think about the new standards and my concerns about how they’ll be used (or not used.) Am I missing something? Were you disappointed to not see something that spoke more clearly to your concerns about Web 2.0 technology? Or are you just pleased that the WAA published these definitions and see them as a small-but-important first step?

Analytics Strategy, Conferences/Community, General

Congratulations to Jennifer Veesenmeyer and June Dershewitz!

Posted on August 23, 2007April 8, 2017 by Eric Peterson

I have been patiently and politely holding my tongue for quite awhile now waiting to say CONGRATULATIONS to both Jennifer Veesenmeyer and June Dershewitz on the career changes they recently made!

Jennifer has joined the already great team at Stratigent as a Senior Web Analytics Consultant, bringing her experience and very well-liked presentation technique (highest scoring presentation at Emetrics, ever, I read!) to Stratigent and their clients. I missed Jennifer’s PIMP my Reports presentation (which apparently she doesn’t own the rights to, too bad!) but heard only amazing things about it. She and I are presenting together at Emetrics so that should be fun!
June has joined Gary Angel and the team at SEMphonic as Vice President of Analytics! I’ve written about June in the past, citing her as one of my personal heroes and the inspiration for Web Analytics Wednesday. June and I even talked about her joining me as a partner at Analytics Demystified but alas, the timing was not quite right. Thusly I am thrilled that she has joined another of my favorite people, Gary Angel, bringing her significant expertise to SEMphonic clients.

Both Stratigent and SEMphonic are Analytics Demystified business partners so this news is even better since my clients will eventually be able to benefit from Jennifer and June’s greatness as well. Win-win-win!

Congrats to Josh Manion and Gary Angel on adding these very talented professionals to their teams.

Analysis, Analytics Strategy, Reporting

One more reason why you CAN'T just start with the data

Posted on August 8, 2007October 25, 2019 by Eric Peterson

My boss mentioned Parkinson’s Law to me this morning in reference to a discussion we were having about sales and marketing process efficiency. I was familiar with the concept, but not with the actual law. If you didn’t follow the link, and you don’t know what it is, it’s the principle that “work expands so as to fill the time available for its completion.” This is so true in the business world that it’s well, kinda sad.

The part of the write-up that jumped out at me, though, was the statement that, “It has been observed over the last 10 years that the memory usage of evolving systems tends to double roughly once every 18 months.” Poor form on the passive voice usage, but that’s a tangent that is not related to this post (or this blog at all, for that matter). I need to do some digging to find the source of this stat. It sounds right, but I did some digging several years ago for this sort of information, and I didn’t find this. What I did find were two different studies by Gartner — performed several years apart — that predicted that there would be a 30x increase in the total volume of enterprise data in the next seven years (I think the studies were done five years apart, and both had a similar projection). I have a clipping somewhere with one of the studies, but it’s in a box en route to Ohio, so I can’t nail the specifics.

These two estimates are so eerily similar that they sort of smell like they came from the same study. Doubling every 18 months would mean you had a 32x increase (2^5) in 7.5 years.

As usual, I’m spending way too damn long on the preamble and not getting to the point, which is this:

Rewind seven years and let’s come up with a hypothetical situation whereby you have just started in a new position. In order to get the lay of the land and figure out what you should do first, you ask for a dump of all data that could possibly be related to your domain of responsibility. For chuckles, let’s say that came out to 3 pages of raw data (not realistic, but making it ridiculously small still supports my point). So, you could take that data, print it out, spread it out on your desk, and pore over it for a couple of hours. Make it a day. You could become so intimate with that data that you would feel like plopping back on a big fluffy pillow and smoking a cigarette. If you did plop back on a pillow and take a drag on a smoke, you could then stare up at the ceiling and wait for your brain to work it’s magic. If there were any interesting, useful insights in that data, your brain would likely find them (assuming your boss doesn’t interrupt your thoughts and want to know: 1) what you’re doing with a big fluffy pillow in your office, or 2) why you’re smoking). That’s one of those really cool things about the brain.

So, in that case, you could start with the data: “Give me the data, I’ll ‘analyze’ it, and then I’ll figure out what action I should take.”

Fast forward seven years. Same situation. Except, there’s been a 30x increase in what you get when you ask for “all the data that could possibly be relevant.” That’s 90 pages of data. You’re brain isn’t going to be able to work it’s magic with that. You could spend 3 weeks looking at the data without feeling like you truly had your head wrapped around it. What most people would do with 90 pages of data would be to start charting it. A picture is worth a 1,000 words, right? That’s one way to get 90 pages of data summarized into something that the brain might be able to handle. Of course, with 90 pages of data, you could produce 900 pages of graphs. Obviously, you would have to pick and choose what you would graph and how. Then, you would keep generating one graph at a time until you saw something that showed either an “interesting” trend or a spike somewhere. At that point, you would be so relieved that you had found something, that you would quickly copy the chart and paste it into PowerPoint so you could show it to a group in a meeting and prove that you were, by golly, doing stuff (um…see Parkinson’s Law!).

If asked by an anal BI-oriented stickler, “Did you take action on the data?” you would respond, “Absolutely! I charted it, put it in PowerPoint, and showed it in a meeting, where everyone agreed that it was interesting!”

EGAD!

Point made?

Analytics Strategy

Sometimes, the best plan is to just ask

Posted on August 7, 2007October 25, 2019 by Eric Peterson

A fairly common web site registration form checkbox is something along the lines of: “Please have a sales representative from this company contact me.” It’s an easy question, very clearly understood…and almost no one checks it.

One way to identify more of these people is to be more subversive. Ask a bunch of questions to try to assess the person’s general fit for the product/service the company is selling. The problem with this approach is that, all too often, the company winds up asking these questions entirely in the company’s terms. Something like: “Check which of the following products you’re interested in.” Or: “Which of the following four items describes your situation” (followed by a list that does not capture the entire universe of possibilities — just the universe that the company has a product or service that is relevant. And, because this information is important, these questions become required fields. The problem is, they require the visitor to think too hard — to map their reality to the company’s terminology.

We tried an experiment recently. We left the “sales representative” checkbox. We removed the confusing, dancing-around-the-issue questions. We replaced it with: “Do you need help with ?” We debated how many people we expected to answer “Yes.” The debate was over where in the spectrum of “not very many” we thought the final count would fall. I was the most optimistic, but I still didn’t expect very many. After all, isn’t it totally obvious why we’re asking?

So far, we’re at ~50% of the people choosing “Yes.” It’s a required field, but the default is “Unselected.”

My theory: it’s a simple question with only two options for the answer. The visitor doesn’t have to think much at all to determine what their honest answer is. And, it’s a question that is sufficiently unambiguous that Yes/No covers the universe of possible answers. Sure, there are some people who think, “My answer is ‘Yes’ but I don’t want to be called yet, so I’ll enter ‘No.'” But that’s a damn sight better than, “What? I’ve read the question three times and still don’t know exactly what my answer should be. I’ll pick something at random.”

Analysis, Analytics Strategy

Are you data-oriented or process-oriented?

Posted on August 4, 2007October 25, 2019 by Eric Peterson

Yet another entry on a topic that’s been kicked around in my head for several years, but which seems to come up every week or two. It’s the recognition that there are two different types of people: data-oriented people and process-oriented people. Okay, that’s a gross oversimplification. There are loads of people who are neither. And, there are people who can manage to approach a problem from both perspectives, but those people tend to have a bias one way or the other.

This isn’t an idea I came up with. A business analyst at National Instruments named Drake Botello actually first explained the concept to me. Drake started his career at National Instruments working on various internal applications. When I got to know him well, he had long since switched over to supporting the company’s data warehouse. On the app side, he had to be more process-focused. On the data warehouse side, he had to be more data-oriented, and what he found was that he often ran into challenges when working with business analysts that supported transactional apps that generated data that wound up in the DW. Drake formed the “data vs. process” theory from those experiences.

So, what does it actually mean?

Well, a process-oriented person expects any system, first and foremost, to support the process it was designed to support in the most efficient manner possible. He views data as a natural, incidental byproduct of the process — it doesn’t need to be thought of beyond what specific reporting capabilities are needed to support the process: all of the data comes from the process, so if the process is good, then the data will be good.

A data-oriented person, on the other hand, sees data design, data capture, data integrity, and data analysis as a critical part of any process. He feels it is perfectly acceptable to incorporate additional steps in a process as required to ensure the integrity of the data that gets generated.

Both of these are perfectly valid perspectives. While I don’t have any way to prove it, my sense is the process-oriented perspective outnumbers the data-oriented perspective by 5-to-1 or so.

This perspective can and does cause problems. My earlier post about operational reporting (not the title, but that’s the meat of the content) comes at this issue from a different angle by pointing out that marketing automation and sales force automation tools, which are geared towards driving very efficient, repeatable processes have a bias towards operational reporting. A data warehouse is just the opposite — it’s geared toward metrics reporting and analysis.

These different perspectives are spawned from different legitimate needs. Still, you don’t want a data-oriented person designing a transactional system any more than you want a process-oriented person designing a data warehouse — they’re both liable to see their assignments as relatively straightforward…and then thoroughly muck them up!

I’ll give one simple example of how two people might approach a contact management system. Contacts’ job titles change over time, right? So, when that happens, how should the system handle it? A process-oriented person may say the job title in the system should be overwritten with the newest job title. He may even pat himself on the back by recording a timestamp as to when the title was last updated (“for analysis purposes.”) But, he will likely say there is no need to retain the previous job title. What use is that? A contact management system is for managing contacts and communication with someone. Since you only really care about the current job title — that’s what you would want to use in any correspondence — the old data is old and not of any use.

A data-oriented person, on the other hand, would demand that a historical record of the person’s job title over time must be maintained and accessible. What if you want to analyze which job titles you are most effective selling to? if you sold to the contact before his promotion, but you don’t know what his title was at that point, your analysis will be severely hampered!

I’ve got a definite data bias, as you can probably tell from my tone. But, I do realize that data requirements can add some very real complexities and difficulties to transactional processes. The best situation is to have both perspectives at the table and collaborating.

Of course, there are also lots of people who don’t really think “data” or “process” at all. This is probably the overwhelming majority of the human population. But if you’re still reading this entry, you’re not one of them!

Analytics Strategy, General, Industry Analysis

Is Google Analytics the Killer App? No.

Posted on August 2, 2007April 11, 2017 by Eric Peterson

For the past two days readers and friends have been writing me asking my opinion of Brandt Dainow’s recent iMediaConnection piece on “Google’s Killer App.” Most of the questions run along these lines:

“Is this guy insane, or did I completely miss the revolution?

http://www.imediaconnection.com/content/15823.asp

I am so impressed with Omniture I have a hard time believing that Google Analytics beats it. I have been in a bit of a bunker trying to keep the magic going here, but can you hook me up with a reality check?”

The reality check is this: Yes, Brent Dainow has apparently gone completely insane. Too bad too, since I kinda liked some of the stuff he’s written in the past.

Let’s consider some of the bizarre statements he makes in his article:

Google has killed the web analytics software industry with the release of the new version of Google Analytics. The new version was released just under two months ago and is simply a quantum leap above any other analytics product on the planet.

This is his opening statement, and I don’t know where to begin. Statements like “killed the web analytics software industry” and “simply a quantum leap above any other analytics product on the planet” are bizarre. Is Dainow paying any attention to the web analytics market? Omniture continues to accelerate, WebTrends has released a great new version of their application, Microsoft is about to release their own free offering, …

And don’t get me wrong: I really do like Google Analytics, and I use it regularly, but there is absolutely no possible justification for saying that Google Analytics is a quantum leap better than other available applications. Google Analytics has some pretty visualizations, a slick UI, and does a good job of integrating with Google’s search marketing products, but a “quantum leap?” I think not.

“Google Analytics version 2 is not revolutionary. It does not extend web analytics software by providing new forms of analysis. Neither does it extend our understanding of websites by offering new approaches. What Google has done is simply take every feature in every product on the market and put them all into one system, and then make it available for free.”

Google has “every feature in every product on the market”? Really? Are you sure? Because I can think of dozens and dozens of useful features that I’ve seen in solutions like ClickTracks, Visual Sciences, Omniture, WebTrends, Coremetrics, Unica, … basically every other solution on the market today that aren’t in the version of Google Analytics I’m using. Features like:

Real visitor segmentation (multidimensional, ad hoc, etc.)
Custom variables at the visitor, session, and page view level
The ability to produce custom reports for automated delivery
The ability to define custom metrics and customize reports in the interface
The ability to import metadata as an input for analysis
Commerce-related reports like browse-to-buy ratios
A browser-overlay that can be customized

(This list goes on and on and on, and has been discussed a great deal by folks like Judah Phillips and Phil Kemelor.)

Dainow continues:

“I am surprised by the range of features Google has added. I would have assumed some had been patented by the companies that created them. I can only conclude this is not the case. The range of features Google has borrowed from other products suggests the web analytics software industry managed to do 10 years of research and development without registering even one patent. This must be unique in the history of computing. If Google has stolen patented ideas, then I can only conclude they simply don’t care and will rely on their massive cash reserves to sort it out later.”

I suspect that Google does not own the patent for the browser overlay, for path analysis, and the JavaScript page tag. I would not assume that Google believes they have “stolen patented ideas” but you can be sure that some lawyer, somewhere, probably does. Maybe the companies that own these patents are pissed at Google but are hesitant to sue a company with the financial resources of GOOG?

Daniow then gets a little more personal:

“I say this as someone who, until this month, ran a company that produced web analytics software and directly competed with Google Analytics. No more. There is simply no way my organization can produce the range of features Google offers and make them available for nothing. We will keep the consulting arm going but use Google Analytics as the reporting system.”

This is perhaps both the most confusing and most telling statement in the entire article. His statement is confusing because one would have thought that as the CEO of a web analytics software company Dainow would have had a more refined understanding of the features available in the market today, the patent market, and the overall utility of free software.

His statement is telling because it sounds like ThinkMetrics is about to become a GAAC (Google Analytics Authorized Consulting) partner, in which case the bizarre pro-Google rhetoric in this article begins to make sense.

[UPDATED: Brett Crosby from Google wrote me and said that ThinkMetrics was not currently nor was about to become a GAAC partner. Which really only makes Dainow’s post that much more bizarre!]
At least, it makes sense that Dainow would want to write a bizarre cheerleader piece like this, I still cannot come up with any justification for iMediaConnection to publish something so strangely biased, poorly researched, and obviously wrong. Perhaps they too have decided that the rest of the vendors are dead and thusly unlikely to buy advertising on their site. I know I wouldn’t be sending a check to iMediaConnection anytime soon if I were Tim Kopp at WebTrends or Gail Ennis at Omniture.

Dainow then makes an even more confusing comment:

“I have been converted to Google Analytics version 2 purely by the strength of the product. It is not just the range of features that is impressive, it is the integration and flexibility.”

If by “integration” Dainow means “with Google’s products only”, and by flexibility he means “a total lack of flexibility” then I suppose I agree. Call me crazy, but I think integration means the ability to pass a variety of data automatically into and out-of the application using defined APIs, not just being able to see Google AdWords impressions and costs. And I think flexibility means the ability to collect multiple custom data, to define new data schemas, and to reprocess data if necessary.
I guess we just have different definitions.
Dainow continues to blather on and on (his “Blather Index” is very high in this article!) about the greatness and wonderfulness and amazing beauty of Google Analytics. For example:

“All the tables are clickable so that I can instantly drill down on the elements that stand out. For example, I recently analyzed the performance of a tourist site’s listings in travel directories. I was able to drill down on specific directories and see which pages and descriptions were working and which were not. Within the same directory, I could see some listings that had a bounce rate of 9 percent and others with a bounce rate of 70 percent.”

Well no wonder Brandt’s so in love with Google Analytics: The tables are clickable and he can instantly drill down on elements that stand out! That is certainly a feature not found elsewhere in web analytics …
I’m getting snarky so I’ll wrap this up. Dainow concludes with the following:

“But despite its failings, the overall range and flexibility of Google Analytics, combined with the price (free), leads me to expect the new version to totally dominate the market and drive most competitors out of business. You need an extremely good reason, or three, to continue staying with any other product.

If there is to be any future in web analytics software for any competitor, that company will need to rapidly expand the scope of reporting available and seriously enhance flexibility and drill-down capabilities.

Industry consolidation is sure to follow, and I expect WebTrends to be one of the few companies with the pockets to pursue such a strategy. It is surprising that Microsoft has not produced a product to compete.”

In these three short paragraphs, Dainow demonstrates a near complete lack of understanding of web analytics and the web analytics marketplace. Google Analytics already dominates the market in terms of total domains coded, but dominance isn’t defined by the breadth of your coding, it’s defined by the success your customers have using your application!

I’m not saying that GA customers aren’t able to be successful, but the data suggests that they still have a long way to go before the value of Google Analytics, or any free analytics application (sorry Ian!), can be assumed. Web analytics is hard and “pretty”, “free”, and “Googly” don’t make it any easier. Dedication and commitment make web analytics easier, not free and click-able.
There are hundreds of good reasons for any company to continue to use an alternative to Google Analytics: Dedicated support, “Enterprise-class” (sic) product features, and a company-wide commitment to customer success, not just to gathering all the world’s data, are three that come to mind.
Most of the licensed solutions on the market today have significantly greater reporting, flexibility, and drill-down capabilities than exist in Google Analytics. Visual Sciences, Omniture, WebTrends, ClickTracks, Coremetrics and others have all spent years working on these kinds of issues, and I think their customers largely agree that they’ve done a pretty good job. Visitor segmentation, custom reporting, and data warehouse analysis are all fundamentally important to “real” web analytics and are all basically absent in Google Analytics.

No disrespect to the management team at WebTrends, but don’t you think that Omniture and their $1.16B USD market cap would qualify as “having deep pockets”? Not that Omniture is likely worried at all about the competitive threat described in your article — plus they’re going to save a bundle by not advertising at iMediaConnection!

Regarding Microsoft and Daniow’s desire to sell Bill Gates his discarded web analytics solution … I think Dainow is the only person in the world who didn’t read last week about Microsoft Gatineau!
The comments at iMediaConnection basically all ask Daniow the same question–What planet are you from, dude?–and are best summarized by this comment:

“oh please….. it is lousy — we are now going to move to Omniture because of all the deficiencies in 2.0 — this kind of post must be paid by Google because people who use it for major adspends (Over 1m for us) know what a lousy move this was for us.. hey but I know the bloggers are excited.. while it has a few nice additions the removal of so many key features and the inability to see metrics together that previously were easy to compares are serious detriments.. plus it is not nearly as sophisticated as it once was.. stop drinking the kool-aid ..” (Elxiabeth schachin)

Well put, Elxiabeth.

In summary: I’m sorry to hear that things didn’t work out for Daniow’s company, especially with the great success that almost everyone else in this industry has been having for the last 24 months. And I wish him all the best as a GAAC partner — the world definitely needs more GAAC partners and smart people able to provide technical support for Google’s wonderful and amazing free application. But the kind of biased, self-serving, and poorly researched rhetoric published in Daniow’s piece has no place in the market today, at least in my humble opinion.
What do you think? Is Google Analytics going to destroy the web analytics marketplace? Is GA2 the best web analytics application in the entire universe? Are you calling your licensed vendor today to cancel your contract, and calling your broker to divest your holdings in OMTR and VSCN now that Daniow has made such a compelling case? I’d love to hear what you all have to think.

Analytics Strategy, General

Technorati is a machine, Gary Angel is a man …

Posted on August 1, 2007 by Eric Peterson

Last night in Tokyo I was griping about how there is very little open debate and disagreement in the web analytics blogging community and how interesting I found the response to my post about Technorati being a poor source of data for any blog ranking methodology.

Then I read Gary Angel’s post “Never Send a Machine to do a Man’s Job” …

I fully retract any complaint about a lack of debate and disagreement, especially the disagreement part. And while I appreciate Gary’s sentiment a great deal, I’m almost more interested in seeing what everyone else thinks about his post. Gary doesn’t pull any punches.
Anyway, thanks again to everyone who commented on the Technorati study. It was a nice reminder about how many great, bright, and thoughtful people read this blog all around the world!

Analytics Strategy, General, Industry Analysis

I am heading to Tokyo but a few thoughts before I go …

Posted on July 29, 2007April 8, 2017 by Eric Peterson

Thanks to everyone who has been so engaged in the debate over Technorati’s utility as a data source for ranking blogs. I guess I opened a can of worms with that post but the debate has been just great! But on to bigger and better things …I’m just about to board my flight to Tokyo, Japan to give the keynote presentation at Digital Forest’s Marketing ROI Day conference on August 1st. I’m very excited about this opportunity and to finally meeting my generous hosts at Digital Forest. If you are reading this and live in-or-near Tokyo, please come to Marketing ROI Day and meet me in person!

Incidentally, I’ve finally updated my presentation schedule. You can learn where I’ll be presenting pretty much through the end of the year at this URL:

http://www.analyticsdemystified.com/link_list.asp?l=Presentation

Finally, I think I mentioned that I’m writing for DM News now. My second article, titled “Hiring Myths for Web Data Talent” is now available online (and theoretically in print as well!) In this article I address four key issues that anyone looking to hire experienced talent — regardless of which analytics platform they’ll be using — needs to consider. I encourage each of you to read the article, but here is the top-line summary of hiring myths:

That an analyst is always the most important “first hire”
That a mathematics background is a must
That web analysts salaries can be easily compared to common IT functions
That a good hire guarantees positive return on investment

I welcome your comments and feedback regarding my presentation calendar, the DM News article, and pretty much anything else you’re interested in chatting about.

Analytics Strategy, General

Technorati is a poor source of blog ranking data …

Posted on July 26, 2007April 10, 2017 by Eric Peterson

A few days ago my friend Avinash Kaushik wrote and asked me for my Feedburner subscriber numbers as an input into his very popular ranking of web analytics blogs. It gave me an opportunity to do something I’d meant to do for awhile — request that Avinash drop my site from his ranking system (which he agreed to do, thanks Avinash!)

I asked Avinash to drop me from the rankings for two reasons:

I’m pretty well established as a web analytics blogger, having done this for a pretty long time. I started blogging about the topic while I was at JupiterResearch back in 2004 and have been writing basically the same blog at a variety of URLs continually since that time. I figure it’s far better for Avinash’s list to highlight some newer folks in the web analytics blogosphere — great writers like Ian Thomas, Gary Angel, Judah Phillips, and Aurelie Pols!
Technorati, which Avinash uses as the basis for his ranking system, is an extremely poor data source for ranking weblogs.

While it may be a unique source of this type of data, Technorati provides a lousy basis for accurately ranking blogs and appears to be very easily fooled by anyone actively working to increase their Technorati ranking.

Why would I say such a thing, you ask? An excellent question, but I have what I think is a pretty good answer (especially if you’ve ever had any concerns about the quality of data you use in your analysis …)

First, and maybe this is something I’m just being dumb about and is easily corrected, if you’ve had a blog for any amount of time and have moved URLs for any reason, Technorati seems incapable of re-grouping URLs for a single blog. Have a look at the following:

As you can see here, based on a search for “web analytics” grouped using the “Blogs” tab in Technorati, my blog shows up as two distinct entries from two slightly different URLs. Both have slightly different levels of authority. When you drill down into “authority” which seems to provide at least partial basis for the ranking system Technorati uses, you’ll see slightly different results for each of these URLs:

The first blog URL lists “437 blog reactions to Analytics Demystified”

The second blog URL lists “461 blog reactions to Analytics Demystified”

What’s worse is that the exact same blog and the exact same content appear further down the same page of results:

Here the blog URL is http://www.analyticsdemystified.com/weblog which was the original blog URL back when I was on the Blogger platform. Perhaps because this is the oldest URL Technorati has in the system, this URL has the greatest reaction:

Now, I wondered if perhaps each of these listings were de-duplicated and could perhaps be added up or something — no such luck it appears. There is a ton of duplication and thusly my blog is pretty much just broken up into three pieces which certainly must make it hard for anyone trying to assess the overall reaction to my writing over the past 3.5 years.

I asked around a bit to see if anyone knew why this happens, and Judah Phillips (who admits he’s been watching his Technorati ranking lately given his relative newness to the blogosphere) pointed me to this entry in the Technotati FAQ:

http://support.technorati.com/faq/topic/56

Just in case you don’t want to read the FAQ entry, I will summarize: You are more or less out of luck. According to the FAQ “we are unable transfer or combine links from different URLs at this time.” I suppose their answer makes sense, but it doesn’t make how Technorati treats blogs that have moved any more useful or appropriate.

Oh well.

The second problem I have with Technorati is that it is either not paying very close attention to where these “blog reactions” are coming from or the system is very easily gamed. Consider the blogs in the number 2, 4, and 5 slots when you search for “web analytics” blogs at Technorati:

There is my friend Avinash, Mr. Marshall Sponder from the Web Analytics Association and KnowMoreMedia, and the entire team at FutureNow, Inc. This is what you expect to see based on Avinash’s ranking system (although I think he might be excluding the guys from FutureNow, I’m not sure …) given that, according to Avinash:

“The evolution of the ranking system continues with a couple of tweaks to the ranking this time around. The primary determinant of the rank in the list below is still Technorati (click here).”

The problem arises when you start to examine the sites that make up the Authority calculation:

Here you can see that Avinash (who is widely loved, I love Avinash too!) has done a great job at generating reaction to his blog, getting 6,013 sites to link back to his content and having a Techorati “authority score” of 948. Very cool … that is until you start to examine the actual sites and blogs linking back to Avinash, at which point you notice something like this entry (#10 on the first page of results when I snapped this screenshot):

Hmmm, that is from Avinash’s own site. That’s odd, isn’t it, that Avinash’s own site would be included in his authority calculation? I thought so, so I quickly looked at the first ten pages of results:

What I found was that 44 percent of the top 50 sites listed as providing “blog reaction” to Occam’s Razor were Avinash’s own (albeit slightly different) URLs.

I’m not sure why that is, do you know?

I figured this might just be some strange anomaly so I took a look at the same thing for Marshall Sponder’s blog, WebMetricsGuru. Marshall didn’t seem to have the same problem, fortunately, but of the 52,152 reactions to Marshall’s blog contained in Technorati, it appears that the dramatic majority come from un-targeted links to his site from other KnowMoreMedia properties:

Similar to the problem with Avinash’s listing, 80 percent of the top 50 sites listed as providing blog reaction to WebMetricsGuru were from KnowMoreMedia. When I continued looking at the results, this percentage actually went up to 83 percent of the top 100 sites “reacting” to Marshall.

I might be thinking about this the wrong way, but that hardly seems like the kind of “reaction” most bloggers are looking for.

Well, at this point I had to look at the Eisenberg’s blog which is a little newer in the blogosphere. Here I saw the exact same problem I found in Avinash’s listing, 34% of the top 50 sites listed as providing blog reaction to GrokDotCom are from, yep, you guessed it, GrokDotCom.

I also noticed that in the GrokDotCom authority listings that some sites appeared again and again and again:

Again, I don’t know what’s going on here but as the basis of “blog popularity” this data seems pretty suspect to me.

Perhaps I’m naive, or perhaps I’m just plain confused here, but the Technorati ranking system doesn’t seem to provide very useful results based on the inconsistency I am describing. Maybe I just happened to stumble on three anomalies — other blogs listed in Avinash’s ranking don’t seem to have the same problems but some surely do. For what it’s worth, none of my three blogs (?!?) listed in Technorati appear to have the problem described above — maybe that’s what I’m doing wrong!

Now perhaps I am thinking about the authority calculation incorrectly. According to Technorati:

“Technorati Authority is the number of blogs linking to a website in the last six months. The higher the number, the more Technorati Authority the blog has.

It is important to note that we measure the number of blogs, rather than the number of links. So, if a blog links to your blog many times, it still only count as +1 toward your authority. Of course, new links mean the +1 will last another 180 days.”

This sounds good and kind of makes sense, except that you can see where KnowMoreMedia is kind of cheating Marshall by having all those completely irrelevant links back to his blog that artificially run up the number of “blog reactions” and likely his authority score.

Also, I kept finding examples of blogs that when I looked at the blogs linking to them, I kept finding the same problem described above — the blog being assessed linking back to itself. Here’s an example from one of the posts/pages on Avinash’s site that has an authority ranking of “12”:

You can’t see all of it but there are 13 reactions to Avinash’s post, and given Technorati’s ranking you would expect 12 different blogs, one of which would have two posts linking to the page, right? Wrong. What you get is five different blogs, two of which are Avinash’s own work (albeit in two different domains) and three of which (the “SEO, SEM, Social Media, web analytics” listings in the image above) appear to be the exact same content in different domains.

If you examine the URLs in the authority listings that come from Avinash’s site, you’ll see that they all have slightly different URLs. But clearly they are all from the exact same blog. If, based on the FAQ answer I gave above about why my blog is listed three times, this is how Technorati is calculating authority … essentially Technorati is saying that every distinct URL is a distinct blog.

Huh?

Seems like a pretty easy system to “game” to me, or one that is easily fooled and mostly useless. At this point I’m even more confused about Technorati’s ability to de-duplicate blogs as an input to their authority ranking.

Please don’t get me wrong, I think Avinash is brilliant for publishing a list of popular blogs (especially one that ranks his own site as #1, how amazing and magnificent is that!) I have learned to respect Marshall Sponder’s ability to write (and write, and write, and write) and I obviously get on well with Bryan and Jeffery Eisenberg (Bryan is one of Analytics Demystified, Inc.’s trusted advisors.) And I sincerely, sincerely hope that each of these fine gentlemen will see that this blog post is far from a criticism of their talents and passions.

But given that I’ve always worked with my clients to make sure they had the best data possible to serve as inputs for their analysis, something about the data reported by Technorati just doesn’t pass the “old smell test”.

Honestly, if anyone out there can help me understand what Technorati is doing and why a ranking system that is apparently so easily corrupted by self-reference and link farming is useful, I’m more than happy to hear from you! Feel free to email me directly or simply comment on this post. Until that time, I will view any ranking system based on Technorati data as quite suspect.

Perhaps you will as well …

I welcome your comments, criticisms, insights and feedback. If someone from Technorati wants to email or call me and explain what the heck I’m doing wrong and why everything I’ve written in this post is incorrect, I’ll gladly listen. And if the answer makes sense to me, I’ll even more gladly apologize for being so confused! If you think I’m carping, whining, or just being critical of Technorati’s data for no good reason, let me hear it! Frankly I sometimes worry that we don’t have enough engaged, thoughtful debate in the web analytics blogosphere …

Analytics Strategy

The Web 2.0 Measurement Group has moved to Facebook

Posted on July 21, 2007 by Eric Peterson

Jeremiah Owyang from PodTech has been bugging me to start a web analytics group at Facebook. I initially resisted but upon kicking the tires at Facebook a bit it seemed like a good idea. If you’re already in the Web 2.0 Measurement Working Group you should have seen an email from me about shutting it down at Google Groups. Either way, the group is now totally open to everyone and can be joined here:

http://www.facebook.com/group.php?gid=2668335473

My hope is that by putting the Web 2.0 Measurement Group on the premier “Web 2.0” platform that it will drive interest and promote conversation. If nothing else it is a reflection that sometimes you can bring audience to you and sometimes you have to go where the audience is.

I hope you’ll join me at Facebook.

Analytics Strategy, General

Why I like Rand Schulman …

Posted on July 16, 2007April 8, 2017 by Eric Peterson

Five little words: Rand don’t pull no punches.

Don’t believe me? Check out some of the commentary Rand offers up about his former employer (and mine, Visual Sciences/WebSideStory) in an interview just published with Eric Enge of Stone Temple Consulting. I only wish Eric would have conducted the interview in the last few days following Visual Sciences announcement that they are going to miss Q2 earnings by a few million dollars (OUCH!) and they’ve retained Goldman Sachs to help them shop the company, just to see what Rand would say.

Rand and Eric even discussed me of all people, with Rand offering up:

“Eric Peterson and I have been in two companies together. WebTrends, and WebSideStory, and we certainly worked together very closely when he was an analyst at Jupiter. We know each other pretty well. Eric and I don’t always agree on everything. I come from a pure solutions point of view, because my DNA is in marketing. I know what I want to achieve with these applicaitons, and Eric has an approach to these things more from a technology point of view. But, more and more, his point of view and mine are becoming very similar. I respect him, because he will tell you what he thinks. We’ve had our differences, but he and I are good friends, and I think it’s good to have differences. It makes the world go round. I think he is going to do really well as an independent by the way.”

While our time together at WebTrends was non-overlapping, and I would more or less describe our close work together while I was at Jupiter as, um, confrontational, I more or less agree with everything else he says in this paragraph (not the entire interview!) And yeah, we drink beer together and have funny conversations about industry folks and he has a unique sense of perspective that most people don’t have, so yeah he and I are friends.

Oh, and I appreciate his vote of confidence regarding the formation of Analytics Demystified.

Wait, what? You don’t know who Rand Schulman is?! You’re kidding me! Rand is an icon in the web analytics industry and, if I may say so, has one butt-kicking web site for his private venture, Rand Schulman Partners.

Anyway, thanks Rand for the nice comments and I certainly wish you and Unica all the best out there. Judah loves your stuff, that’s for sure.

Adobe Analytics, Analytics Strategy, Industry Analysis, Reporting

On NetRatings and time spent on site

Posted on July 12, 2007 by Eric Peterson

In all of the fuss about NetRatings dropping page views as a metric used to calculate site popularity is the fact that the company actually did a pretty smart thing: they took my advice from February 15th of this year and rolled in a very valuable and useful “sessions” metric. Well, maybe it wasn’t my advice they took, but I think it was a great idea either way to drop page views since they’ve become increasingly inconsistent to instead focus on the one metric that is consistently applied and well defined, sessions.

Unfortunately NetRatings chose to focus their announcement on “total minutes” saying that time was a better measure of engagement. Personally I’ve never been a very big fan of the time spent metrics — I guess I’ve just looked too long and too hard at all the problems associated with how time is collected and recorded in the web analytics realm.

There is a really engaged thread at the Web Analytics Forum at Yahoo! Groups on this subject that is definitely worth a read if you’re interested.

And I’ll admit, I don’t have all the details associated with how panel-based services like Neilsen and comScore track time spent. If they’re actively tracking the user and only counting time when the browser window is active and the mouse is moving, well that would be a good use of the panel. My suspicion is that, like in web analytics, they’re simply recording the delta between the first and last request for a page in the domain — a strategy that suffers from a litany of well-described problems.

The two I see as most problematic are:

Single page visits are either difficult to count or not counted in time spent calculations
The amount of time a web page is open is likely only poorly correlated to their actual engagement with the page

Some have already noted that the fact that very popular sites like Google will do poorly in time spent on site because one of the dominant use cases involves only a single page (I search and I go.) Conversely, depending on how time spent on site is calculated, the search engines may have inordinately long times spent based on a search leading to a long browse time on a discovered site, leading back to the search results (same session, clock is presumably still ticking), leading to the next discovered site, etc.

I for one use iGoogle in exactly this way: I load the page frequently throughout the day and do nothing more than look at a single page view. In fact, unless Nielsen is either tracking the AJAX-interaction with the iGoogle interface, or counting single page view sessions, it is likely that my interaction with iGoogle is not counted at all. But let me assure you, I am quite engaged with the content in my Google portal (something that would be well evidenced by the total session count I generate at the site each day.)

As I looked back through the plethora of comments that my original post on using sessions to compare sites I noticed that I had made this statement in response to a comment from Jacques Warren:

If you want to compare two or more web sites, use sessions because of the reasons I outlined in my original post.
If you’re interested in the number of people coming to one web site (presumably yours), use de-duplicated unique visitors but be mindful of cookie deletion.
If you’re interested in the activity of people on your web site, and if you have a “Web 1.0″ web site, use page views but be mindful of issues like code coverage, proxies, robots, etc.
If you’re interested in the activity of people on your web site, and if you have a “Web 2.0″ web site built around RIAs, etc., use some form of event model.

I’ll stand by this. Until I know more about how N/NR and comScore calculate their time spent on site metrics it’s hard to believe their numbers to be any more useful or accurate than those provided by direct measurement systems. That said, I’d welcome a briefing on the subject from either company if they’re reading this and are interested in ~~having me pick apart their methodology~~ spending some time with me.

If companies really need to use time spent on site, they should consider using better key performance indicators for time such as Percent Low/Medium/High Time Spent on Site categories (something I talk about at length in The Big Book of Key Performance Indicators.) That way N/NR could report on the percent of all tracked sessions that were “30 seconds or less”, “31 seconds to 5 minutes”, and “More than 5 minutes” (as an example) which would give us a more powerful view into the relationship between visitors and the time they spend on site.
At the end of the day I like that N/NR has provided a consistent and easily compared metric to their customers in “total sessions” which is what I will inevitably focus on as a measure of site popularity. Having devoted quite a bit of time to describing what I believe to be a solid measure of visitor engagement, it’s difficult for me to think about “time spent on site” (or even “total sessions”) as a good proxy. Time spent, recency, depth of session, session number, etc. are all components of engagement, not direct measures.

What do you think? Is Nielsen right and I’m crazy? Have you been looking closely at your time spent on site metric for years and are delighted that the rest of the world has finally caught up? Or are you like me and spend far too much time browsing from site to site, flipping from task to task, and thusly confounding clocks and counters on every site you visit?

I welcome your comments.

Analysis, Analytics Strategy

In Search of the Mythical Step Function

Posted on July 5, 2007October 25, 2019 by Eric Peterson

One thing I’ve learned over the years is that the real world of data is a lot messier than a whole range of information outlets imply. Whether it’s web data, CRM data, or ERP data, it is very, very seldom that there is something going on that, once discovered, can have an immediate and dramatic positive impact with little effort. The reality is, it takes some up front discipline to prepare to conduct an analysis, then, often, a not insignificant effort to get the data needed for the analysis pulled and prepped. And, at the end of the day, the best results pass the hurdle of “statistically significant.” What that means is that a set of variables may be found that have a slight-but-real impact on something you care about. Now, hopefully, those are variables that you can influence, and that you can influence without too much investment.

A classic examples with web data is what I call “the myth that people are cows.” Anyone who has ever been in a pasture with a herd of cattle knows that it takes the average bovine somewhere between 1 and 3 nanoseconds to settle into an unwavering pattern. Grove of trees to watering hole at 8:07 AM. Watering hole to grassy knoll at 1:53 PM. Grassy knoll back to grove of trees at 6:32 PM. More than that, the entire herd follows the exact same 12-inch wide path unerringly from point to point. It takes almost no time for that path to be a well-worn, dirt, 12-inch wide trail.

(For an absolutely wonderful poem on the subject of such paths, and how Sam Walter Foss imagined such a path driving urban development of a major city, check out “The Calf-Path.” at Public Radio International’s The Writer’s Almanac.)

The problem is that, all too often, Marketeers assume that people are like a herd of cattle. They know that, if they can just find the most common paths through their web site, they can take advantage of it in huge and profitable ways! “We’ll know exactly where to put a billboard for Maisy’s Magnificent Udder Moisterizer that will attract the most eyeballs!”

Unfortunately, visitors to web sites are not cows. Not even close. Try some simple math. How many unique links do you have on your home page? 10? 20? 100? I’d bet good money that, if you count them, you’ll realize it’s more than you thought. That’s the beauty of drop-down menus in that a clean and simple design can still present the visitor with a lot of options. For chuckles, let’s say there are only 10 links on each page on your site, including the home page. Let’s also say that the site does not have a search box that persists on every page (poor form, that). And, let’s go ahead and say visitors’ browsers don’t have a Back button. Given all of those unrealistic constraints, the number of unique possible paths from the main page of the site five levels deep is 10^5, or 100,000. With 100,000 options, the most popular path is going to be, at best a percentage point or two of the overall traffic. Now, factor in a Back button and a search box…and the math got wayyyyy to complicated for this blogger.

The point? It’s a waste of time to try to hone in on the “most popular paths from the main page of our site.” At best, it makes sense to look one or two levels deep…and the most likely insight you will ge there is that there is a noticeable chunk of people who are clicking on a link on your main page and then clicking the Back button, which should make you question if the page they clicked to is delivering what the link implies it will.

Web analytics vendors don’t really help things. Clickstreams are so popular among the underinformed that they have to bake clickstream functionality into their tools, and they need to do so in a way that makes for a slick demo. Of course, in the case of the demo, they have control over the data, so they can show a cow-like clickstream! Reality…just isn’t that simple!

I really can’t seem to right a short entry. I’ll try again next time!

Analytics Strategy, Conferences/Community, General

Lars gathers some of the best and brightest in Europe to chat

Posted on June 29, 2007April 8, 2017 by Eric Peterson

I’ve never been much for listening to podcasts for some reason but all the sudden I seem to be listening to them all the time. Last week it was Bryan’s interview with Avinash Kaushik from ZQInsights. This week it’s Lars Johansson, the Swedish coordinator for WAA, talking to seven of the brightest minds in web analytics in Europe.

Having recently been to Europe, I was delighted to listen to this conversation. While in Holland I presented data (links to a PDF) showing that European web analytics practitioners are not far behind their U.S. counterparts. When you listen to the podcast you’ll hear the participant’s talking about nearly the exact same challenges we all face here in the U.S. Concerns about process, distribution of decision making (Aurelie talks about pan-European companies, essentially multiple divisions but that speak different languages, have different values and expectations, etc.)

This is a long podcast but well worth a listen if you have time. Great work, Lars, bringing these bright minds together for the conversation.

Adobe Analytics, Analytics Strategy, Conferences/Community, General

Bryan, Avinash, and Judah …

Posted on June 18, 2007April 8, 2017 by Eric Peterson

I am just finishing up listening to Bryan Eisenberg’s interview with Avinash Kaushik, author of Web Analyics: An Hour a Day (which was in my office when I returned from San Francisco last week, thanks Avinash!) Even if you’re not much for podcasts you should spend the time to listen to these guys talk. Bryan asks Avinash some great questions and Avinash, as usual, provides some really great answers.

Also, Judah (who is not afraid to take on hard subjects) has opened his own can of worms with a post on selecting a web analytics vendor. I personally know that Judah went through the selection process with two top vendors recently so it will be interesting to see both what he comes up with and also what comments he gets from his readers.

I’m off to Boston tomorrow for a variety of meetings and presentations. If you’re in the Boston area, please join me at a special Web Analytics Wednesday on TUESDAY at Aquent headquarters (711 Boylston Street).

Adobe Analytics, Analytics Strategy, Conferences/Community, General

Video from Jeremiah Owyang and the WAW Guru breakfast

Posted on June 11, 2007 by Eric Peterson

About a month ago, just before I started Analytics Demystified, I had the pleasure of sitting down for an interview with Jeremiah Owyang of PodTech.net. Clint first introduced me to Jeremiah when I was talking about measuring visitor engagement and how social media might be best measured. Jeremiah is very much connected in the Bay Area and I though the interview went really well (but you can judge for yourself by watching the interview at Jeremiah’s web site.)

A number of folks have commented on the interview at Jeremiah’s site and the comments are well worth a read.

More recently I wrote a post on the 10/20/70 Rule for Achievable Web Analytics Success in which I outlined the importance of process to web analytics. A number of folks have since commented on the post but Rene Dechamps from OX2 was kind enough to post a video from the conversation that got me thinking about 10/20/70 (thanks Rene!)

Since Rene was about as tired as I was at 7:00 AM local time, and he’d been kind enough to bring me a coffee, I recommend ** not ** trying to watch the video and just listening instead.

What do you think? Should I stick to writing and stay off the tele? As always, I welcome your comments.

Analytics Strategy, General

The comScore study on cookie deletion is finally out

Posted on June 6, 2007 by Eric Peterson

I just happened to write my contact at comScore today asking about their follow-up report on cookie deletion. He said it would be out today and here it is:

http://www.comscore.com/request/cookie_deletion.asp

This report does a good job of providing additional data and information about the comScore methodology in this report, something missing from the press release and critical to our collective understanding of cookie deletion. This report explicitly addresses anti-spyware and the differences in third- and first-party cookie deletion, essentially showing that there is an anti-spyware effect but it is minimal compared to manual cookie deletion which appears to be the primary culprit.

comScore also presents some of the attitudinal data they alluded to in their press release, essentially confirming what I first reported at JupiterResearch in 2005 … that most consumers aren’t really sure what cookies do.

Since I last saw the report they added a few sections — one on international traffic and one on cookie blocking. While the section on international doesn’t add much to the conversation other than to explain why panel-based and log-based systems numbers differ (something that should be fairly obvious), the cookie blocking data is pretty interesting.

According to comScore, if your web analytics application falls-back to an IP-based value for unique visitor identification in the absence of a cookie being successfully set, you’re likely worse off than you are simply dropping those visitors. Their table on page 15 shows that due to dynamic IP assignment that the average home computer has 10.5 different IP addresses in a month. Yikes!

If you’re into this stuff, or if you’re interested in how much cookie deletion might be impacting your own audience measurement, you should download the report and give it a careful read. It certainly doesn’t provide a solution to the problem, but often times knowing is half the battle.

http://www.comscore.com/request/cookie_deletion.asp

I welcome your feedback on the report and the usual comments and criticism.

Analytics Strategy, Conferences/Community, General

Research summary from our March 2007 survey now available

Posted on June 5, 2007April 8, 2017 by Eric Peterson

The overview document I promised when we conducted our survey in March and April is finally available and can be downloaded here:

http://www.analyticsdemystified.com/research/

You can read the press release that was written about the research here:

http://new.marketwire.com/2.0/rel.jsp?id=739164

There is also a very nice write-up on the research written by Jennifer LeClaire called “Process-Driven Analytics or Bust” published on the NewsFactor Network. In the article Jennifer explores the data and captures good insights from both Megan Burns at Forrester and John Lovett at Aberdeen.

From Megan Burns:

Web analytics is so complex, Burns added, and there is so much analysis that organizations could do. Day-to-day reports need to be institutionalized, she explained, by putting in systems and processes that function smoothly.

“With a process-driven approach, your Web analytics analysts have the time to do higher level activities, advanced analysis, support multivariate testing, and other activities that deliver additional incremental value to an organization,” Burns said.

From John Lovett at Aberdeen:

“The next level [in web analytics] is establishing business processes so you can use analytics to measure results,” said Aberdeen’s Lovett. “That is the best way for companies to leverage the analytics platform.”

Thanks to Zori Bayriamova (my research partner), the Web Analytics Association (our research distribution partner) and over 1,000 people around the world who responded to our request for help with this report. We will be publishing follow-up reports over the next few months so definitely keep in touch.

Again, you can download this research at:

http://www.analyticsdemystified.com/research/

Analytics Strategy, General

The 10/20/70 rule for Achievable Web Analytics Success

Posted on June 3, 2007April 8, 2017 by Eric Peterson

In San Francisco during the “Guru Breakfast” event Rene Dechamps asked a question about the importance of process to web analytics. This is clearly something I believe to be tremendously important (quit my job, printed business cards, etc.) and Bryan Eisenberg commented that “web analytics was 10 percent technology, 20 percent people, and 70 percent process …”

Rene said he would post the video he took of this conversation soon, but suffice to say Avinash Kaushik, Jim Sterne and I all agreed with Bryan. Process is very important to web analytics, but the importance of process is often overlooked.

Recently a reporter got me thinking about these numbers, so I would like to formally propose an update to beloved guru Kaushik’s widely quoted 10/90 rule. I call it the 10/20/70 Rule for Achievable Web Analytics Success. Here is what it says …

Our Goal: Highest value from our investment in web analytics
Percent of time and effort spent on the selection and deployment of their technology platform: 10%
Percent of time and effort spent on the hiring and allocation of really smart people: 20%
Percent of time and effort spent on the process of actually “doing” web analytics, leveraging both technology and people: 70%
Bottom line for Achievable Success: It’s the process

The explicit recognition of the value of process resolves some of the issues people have with Kaushik’s original proposal, two of which include:

It is very difficult to spend $90 of every $100 on “intelligent resources/analysts” given the extreme dearth of available talent relative to the number of jobs currently open in the market today. Even Kaushik’s former organization (Intuit) is, to the best of my knowledge still looking for his replacement, several months later, ironically highlighting the difficulty of finding good talent.
Unless you’re gonna go the Google Analytics route (spending $0 on technology) and hire inexpensive resources to install the software (likely not one of the GAAC partners, although I’m not entirely sure what they charge) you’ll be hard pressed to spend $10 of every $100 on software license and implementation.

Now, I obviously agree with Avinash’s emphatic call to hire smart people. I’m a huge fan of dedicating resources to web analytics projects and have been since 2004 when JupiterResearch published my report Web Analytics: Spending, Staffing, and Vendor Selection. You need bright people to run your web analytics applications and to analyze data (although you may not need the people you think you need … more on that in another post at another time!)

But I think that the right way to frame the right approach to web analytics is not in terms of how you spend your valuable money, it’s how you spend your valuable time. So the 10/20/70 rule updates Kaushik’s rule by applying the appropriate emphasis squarely on the processes involved in “doing” web analytics.

Remember, you can always make more money, but it’s hard to make more time. Fortunately, some pretty bright people seem to agree with me.

Consider this: The technology involved is largely the same, especially at the level of need that most companies currently have from their web analytics solution. And while people are a good proxy for true process, in my experience too great of a dependence on people can cause two substantial problems:

If the people are not the right people, the organization may not realize there is a problem until a great deal of money has been spent and a great deal of time has been wasted
In my recent web analytics survey (results coming very soon!) we found that HALF of all respondents having web analytics experience had considered taking a new job in the last six months

So in the absence of process, some companies end up hiring unqualified people, hiring the wrong people, or hiring people who jump ship when the next best offer comes along. Certainly this is not the case with all companies, but until your organization has clear expectations about the goals for your investment in web analytics and how you plan to achieve those goals, technology and people will only get you so far.

I know, I know, I said to hire people and everything would be fine. It will be fine, but with process, you can be better than fine. You can make money. Piles of it.

So if you think your company is not following the 10/20/70 rule, here is my humble recommendation for you to consider:

Take whatever technology you have already deployed, until you’re good at web analytics the technology doesn’t really matter
Gather your key site stakeholders together
Ask them to share their experience and understanding of web analytics thus far
Document the gaps, looking for statements like “concerns about data accuracy”, “problems with data collection”, “not getting the right reports”, “reports are not actionable” and “concerns about how effectively we’re using web analytics tools”
Pick any typical site process such as launching a new campaign or deploying a new page or micro-site
Diagram the process you picked in step #5, highlighting decision points, tasks, and sub-tasks
Determine where measurement fits in the diagram you produce
Ask yourself if measurement, reporting, and/or analysis always happens in the places you’ve identified
If not, ask yourself if the lack of measurement, reporting, and/or analysis results in the stakeholder concerns discovered in step #4
If so, add measurement, reporting, and analysis to your diagram and make sure to follow the new diagram/process/checklist every time!

Still on the fence? Here are some questions for you to consider:

If you have spent $10 of every $100 on technology, are you successful in getting most of your questions answered?
If you answered “yes” to question #1, are you sure you’re asking the right questions?
If you’re trying to spend $90 of every $100 on people, how exactly is that going for you?
If you answered “great” to question #3, are you sure you’re not paying too much?
If you have great technology and great people, what is your web analytics ROI?
If you don’t know the answer to question #5, why not?
If you have dedicated analysts on your staff, what percentage of their time do they spend generating reports and attending meetings vs. producing analysis and managing experiments?
If you answered “too much reporting and meetings” to question #7, why do you think that is?
If you have thought about the process of doing web analytics, do you have a checklist or business process diagram for the core processes?
If you answered “huh?!” to question #9, call me.

Obviously my clear bias is for companies to invest in the process of doing web analytics. But professionally I have spent a great deal of time looking at this problem from all possible angles. And every time the answer is the same: The companies that invest their time refining how they actually “do” web analytics get more out of their efforts than companies who simply invest their money.

As always I welcome your comments and criticism.

Analytics Strategy

Web analytics in Holland

Posted on June 3, 2007April 8, 2017 by Eric Peterson

On Thursday of last week I had the great pleasure of traveling to Holland to give a keynote presentation at the first Webanalytics Congres in Zeist. By all measures the event was a great success:

Roughly 180 people attended this first time event (congratulations to BBP!)
(Reportedly) very good presentations, including one from Postbank.NL that was very well received and is reviewed by my friend Aurelie Pols. I say “reportedly” very good since I understand very little Dutch and the conference is, well, almost entirely in Dutch
The vendors at the event reported good conversations and good traffic for a crowd this size

I presented research from my recent web analytics survey looking at the differences in attitude and opinions regarding web analytics use in Europe and in the U.S. While survey data can sometimes be quite dry, the information I presented elicited several very good questions, the best of which was “Can you please give me a list of good analysis questions that web analytics can answer?”

I also had the chance to meet in person some tremendously nice people including Marco Derksen of Marketingfact.NL and Upstream.NL, Neil Morgan from Omniture, Dennis Mortensen from Indextools, Bram van Essen from Paul Postma Marketing Consultancy, Eelco van Kuik from the Conversion Company, Nicole Niemann from Satama, Geert-Jan Smits from Jungle Rating. In fact, Geert-Jan Smits signed a copy of his new book on Balanced Scorecards, De Internet Scorecard, which is entirely in Dutch but that Aurelie assures me is very good.

You can download a PDF of my presentation here.

On Friday night, Analytics Demystified and OX2 sponsored a little dinner in Amsterdam which was also very much a success. A number of companies participated and I believe a very good time was had by all.

I want to personally thank Rene, Aurelie, Marco, Paul, Heidi, and everyone else who made this trip (and my little holiday) possible. I am very much looking forward to my return to Europe in September for e.Day and other web analytics events currently in planning (yes Lars, I’m coming to Scandanavia!)

Here are some photos:

Steven Krall from Leads2Business, Rene, Dennis Mortensen from IndexTools, Aurelie in Zeist, Holland.

Heidi Bouhuijzen from BBP. Heidi made this trip an absolute dream by coordinating every aspect of my travel. If you’re coming to Europe, I hope you’re lucky enough to have someone as wonderful as Heidi to help you!

Marco Derksen from Marketingfacts.NL. Marco was the driving force behind bringing me to the Webanalytics Congres and has the most popular marketing blog in the Netherlands.

The two Renes, Dechamps and Nijhuis. The latter Rene is from Google Analytics in Amsterdam and works for my friend Dr. Brian Clifton.

Analytics Strategy, General

Avinash on the definition of Enterprise Class analytics

Posted on May 31, 2007April 8, 2017 by Eric Peterson

Avinash Kaushik recently wrote an interesting response to my original post and request for comment from Google. I asked Avinash or Brett Crosby if they wanted to comment on the “Enterprise” nature of Google Analytics in one of the comments. Avinash didn’t want to answer that question, but he did offer some interesting thoughts on the definition of “Enterprise Class”.

In his post, Avinash says:

“Definition of an Enterprise Class vendor :

1) The Vendor has been around for more than 18 months, the longer the duration the better but beyond 18 months you the client decide what you are comfortable with.

2) The Vendor can scale its ASP infrastructure (or in house software solution) to A] capture the number of page views required by the client and B] process that data on a nightly basis.

{I am biased, I believe that most people don’t need real-time data even if they think they do. Here’s a check-list to figure out if you really need real time data: Is Real-Time Really Relevant?}

3) They have a support infrastructure to assist the client in need at reasonable price. If you are willing to pay for support, you should have to pay a reasonable price and expect solid support from the Vendor or their Partners.

That’s it. Nothing else matters. You need to know it has been around and that it’ll be there. No other golden rules.”

While I like Avinash a lot, I think this definition is way off the mark for what most companies are looking for when they say “We need an Enterprise-class analytics solution.” I think what Avinash has described is the core qualities of every analytics vendor in the market today (at least when he includes “or their Partners” in item #3 which picks up Google Analytics and a few others.)

The problem with this definition is that it does nothing to differentiate hundreds of vendors from one another. Kind of like a vendor constellation where every company is stacked on top of one another … So I propose that Avinash’s definition of “Enterprise Class” is incorrect, and instead point to the definition of “Enterprise software” found in the Wikipedia. Some relevant excerpts:

Enterprise software is software that solves an enterprise problem (rather than a departmental problem …
Enterprise software is often available as a suite of programs that have attached development tools to modify the common programs for the specific enterprise.
Mostly [enterprise software] development tools are complex programming tools that require specialist capabilities.

I personally can only think of one currently available web analytics solutions that would qualify as “Enterprise software” using the Wikipedia definition. Remember: Marketing is a department, not the entire business! This definition implies that the solution can be used throughout the organization to solve a variety of problems (for example, online and offline data analysis using the same suite of tools.)

Most interesting to this conversation are these criticisms of the use of the term enterprise (again from Wikipedia):

Often the term is used to mean virtually anything, by virtue of it having become the latest corporate-speak buzzword.
Some enterprise software vendors using the latter definition develop highly complex products that are often overkill for smaller organizations, and the application of these can be a very frustrating task.
Sometimes “enterprise” might be used sarcastically to mean overly complex software.

Ah ha, now the truth comes out! Perhaps the use of the term “enterprise” in web analytics is just a marketing ploy, designed to sound good but not really say anything at all about the sophistication of the solution.

Hmm …

Avinash goes on to offer a “not so humble rant” (his words):

“The reason most of corporate America is saddled with billions of dollars of sub optimal software is that companies judge tools/vendors on this vague quality called “enterprise class”, while completely ignoring what they actually need.”

While not necessarily untrue, Avinash incorrectly assumes that most companies know what they actually need. If most companies were able to “look deeply within themselves and figure out exactly what they need and then go get it” (his words) there would be no need for the vendor constellations that Forrester, Gartner, JupiterResearch and others produce every year. But these vendor assessments are reportedly among the most valuable of all analyst documents; for good or ill, they simplify the problem of differentiating hundreds of vendors who largely all say they do exactly the same thing.

So maybe, just maybe, corporate America has been fooled into buying solutions that are good but aren’t really “Enterprise class software” after all. But maybe they were only doing what they thought was best? As someone commented in Avinash’s post: “Nobody was ever fired for buying Enterprise class software …”

And maybe when the analyst firms call something “Enterprise software”, at least in the web analytics market, they’re simply looking for something that will let them write about a reasonable number of companies, not the 100+ vendors that Sebastian documents at web analytics book? Having written a constellation in the past, I can attest to the complexity involved in covering even a dozen vendors, much less 100!

Perhaps the best possible outcome from this conversation would be that all of the vendors stop calling themselves “Enterprise-class” altogether and instead work to differentiate themselves along other, more substantial lines? I can think of dozens of other good points of differentiation … cost, support strategy, approach to data integration, etc.

In the end Avinash comes closest to the truth when he reminds us that:

“Smart people with crappy tools can move mountains, without smart people even the most expensive and expansive tools can’t help a company move beyond measuring Visits.”

Which I would modify to say:

“Smart people leveraging good business process, despite crappy tools, can move mountains, without smart people and good process, even the most expensive and expansive tools can’t help a company move beyond measuring visits.”

Anyway, with all this in mind I suppose I don’t care if Google believes that GA is “high-end” or “Enterprise class” or whatever. I’m just happy that they’ve raised the bar on the online visualization interface and are continuing to drive interest in web analytics in general. Again, thanks Google (and thanks Avinash for picking up the conversation!)

As always, I welcome your comments and criticism.

Analytics Strategy, General

I'm in Europe today and announcing my partnership with OX2

Posted on May 30, 2007April 8, 2017 by Eric Peterson

Today in Holland I am announcing Analytics Demystified’s partnership with the Belgian firm OX2.
The first partnership we announced, you may recall, is with Aquent–the world’s largest marketing staffing organization–and is designed to let Analytics Demystified help companies find, train, and hire web measurement staff while establishing a strategic road map for web analytics success.

But in my experience not all companies wanting to think strategically about web analytics are able to immediately tackle the organizational hurdles associated with hiring staff or managing teams of consultants. In situations like these, the approach we will take is to bring in business partners who have strong, demonstrated experience providing long-term web analytics consulting. Our partners will then help implement the shared vision of the client, Analytics Demystified, and the partner on an ongoing basis.
In central Europe the team I believed to be best suited to that work was Rene and Aurelie’s team at OX2 (the “Web Analytics Dream Team”.)

Please read the press release and if you have any questions about this announcement please contact me directly.

Analytics Strategy, General

Who said that Google Analytics was Enterprise analytics?

Posted on May 29, 2007April 8, 2017 by Eric Peterson

I know I’m going to get shit for saying this, but I think that my good friend Judah Phillips and the fine folks at CMS Watch may have too high an expectation about what Google Analytics is supposed to be. Not that Judah or Phil Kemelor are incorrect in their assessment that G.A. is missing a lot of functionality you expect to find in “Enterprise class” web analytics applications—stuff like APIs, data warehouse query, advanced visitor segmentation, look-up tables, data loading, etc.—all of which appears to still be lacking in the new version of Google Analytics.

The thing is, I’m not sure anyone at Google ever said it was supposed to have all that functionality.

Now, I don’t read every word Googler Avinash Kaushik writes (much as I try Avinash, I’m waiting for the book!) so maybe he said something like “Google Analytics version 2 eliminates the need for any investment in web analytics technology, period!” I kind of doubt he’d say something like that, partly because that doesn’t sound like Avinash, but mostly because it’s ridiculous.

Even with Kaushik’s whole 10/90 rule, it’s hard to imagine a thoughtful guy like Avinash saying something like “You should dump your existing million-dollar, multi-year investment in Visual Sciences for Google Analytics and spend the money you save on analysts.”

And I don’t think I’ve ever heard Brett Crosby talk about Google Analytics replacing the other vendors. Again, maybe I’m wrong, but Brett (and Google’s) goal for Google Analytics has always struck me as being basically “great web analytics at the best possible price” not “the only web analytics platform you’ll ever need.” Granted, I kinda stopped paying attention for 17 months there, but given the media fascination with all things Googl-y I bet someone would have said “Hey Eric, did you read what Google said about dominating the web analytics market and destroying the other vendors?”

Judah pointed out this language at Google extolling the virtues of Google Analytics:

“Google Analytics has all the features you’d expect from a high-end analytics offering.”

Yeah, this is wrong, but this is clearly marketing and kind of hinges on the words “you’d expect” doesn’t it? In a high-end analytics offering, I personally expect a high-end analytics offering to provide visitor-based analytics with unlimited real-time visitor and session-level segmentation over the entire data-set plus the ability to define as many data dimensions and custom metrics as I like, combined with a flexible ETL tool that allows me to combine multiple disparate data into a single combined view of my site visitors and marketing campaigns.

Maybe my expectations are too high?

Personally I think that the new version of Google Analytics is one of the best things to ever happen to our industry.

Yep.

Even though Google Analytics still lacks some core functionality that most companies absolutely need when they get really serious about web analytics, the new release does one thing that the industry absolutely needed to push it to the next level: The new Google Analytics interface encourages exploration and supports drilling-down by allowing the user to maintain their inquisitive momentum.

This is HUGE and is one of the things that really excited me about Visual Sciences Visual Site a few years ago. Now, don’t get me wrong, GA2 is no Visual Workstation, but the AJAX and pre-loading they’re doing allows me to quickly change tabs, re-order columns, etc. without having to waaaaaaaaiiiiiiitttttt for the page to load. I am probably the world’s antsiest analyst and I totally hate waiting for data to appear and reports to be generated. GA2 hasn’t done away with that, but it is clearly heading in the right direction.

On this I think Judah, Phil, and I all agree.

I don’t personally think that Google Analytics is high-end or Enterprise-class, and that’s okay. But I do think the new Google Analytics does create real pressure on other vendors to re-evaluate their UI and perhaps places some increased pressure on everyone else to further differentiate their product and solution offerings. Especially if you believe we’ll see an integrated Google Analytics + Website Optimizer + Feedburner (which Ian would probably call the “Fondleburgerwizer”) sometime in the near future …

Not that this should be a problem, since each company has different goals for their application in the marketplace. But it is worth noting that Google Analytics is showing up all over the place. Have a look at this snapshot from the unreleased update to the Vendor Discovery Tool:

What this says is that of the 9,181 URLs tracked by the system, the tool found Google Analytics code on 25% of them. More importantly, both GA and WebSideStory code were found on 6% of tracked URLs, GA and Omniture on 4% of tracked URLs, GA and WebTrends Hosted on 4% of tracked URLs, etc. Personally, if I’m any of these vendors, I really don’t want to see Google Analytics on my paying customer’s web sites.

I’m not trying to be obtuse, but I think it’s only fair to take the recent update for what it is: a really thoughtful overhaul of the primary integration point for most people with their web analytics data. Faster access to data, more relevant metrics tied to dimensions (“bounce rate” against campaigns and search keywords, how freaking cool!), and a brilliant UI built by some of the best in the business.

Nice work, Google.

Adobe Analytics, Analytics Strategy, General

Heading out on the road next week

Posted on May 24, 2007April 8, 2017 by Eric Peterson

UPDATED: Geez, I totally forgot I am presenting at the Business Marketing Association annual conference in Las Vegas on June 14th! My friend Aaron Kahlow of BusinessOnline hooked me up with this event and I’m excited about presenting on KPIs.
Phew, it has been kind of a crazy two weeks. Nobody told me that starting a company would be this much work (or this much fun!) I’m pretty excited about a handful of things I’m not quite ready to talk about yet but I’m heading out into the world and would love to get together with any of you reading my blog who might like to catch up.

Here’s where you’ll find me:

Next week I’ll be just outside of Amsterdam in Zeist at the Webanalytics Congres. I was invited by Marco Derksen from the Marketingfacts.NL blog and Upstream.NL and he who referred to my visitor engagement metric as “the mother of all KPIs!“
On June 1st, Rene and Aurelie and I will be gathering folks for a dinner in Amsterdam. Details are still being worked out but if you want to join us either email me or comment to Rene and Aurelie on their blog post.
On June 19th I’m flying into Boston just in time for a very special Web Analytics Wednesday hosted by my good friend Judah Phillips, my business partner Aquent, and the kind people at Unica (hi Rand!) I am going to present some research I’ve done and the drinks are complimentary.
On June 20th I’m presenting at the Visual Sciences ActiveInsights conference (details TBD but contact Visual Sciences for more information)
On June 21st I’m in New York doing a special Jim Sterne event that I’m not supposed to talk about yet but please let me know if you’re in New York and are interested in seeing some very bright folks (and me) talk about web analytics and I’ll get you the 411.

It’s more of the same through the summer and I’ll try and keep my schedule up-to-date. You can also see where I’ll be out presenting by checking my presentation calendar on the Analytics Demystified web site.

Analytics Strategy

Jennifer LeClaire calls the Aquent partnership "brilliant"

Posted on May 18, 2007 by Eric Peterson

Jennifer LeClaire, a well-respected reporter for The NewsFactor Network who has worked in the past for the AP and the New York Times, and who is a web analytics author herself, picked up the news about the Analytics Demystified/Aquent partnership on her site AnalyticsInsider.

Regarding the partnership, LeClaire has this to say:

“Eric is on to something here. I knew it would only be a matter of days after leaving Visual Sciences that he would strike a partnership of some sort, but I didn’t see this one coming. This is brilliant, what with the online job search space growing and talent crunches making it more difficult for recruiters to find superstar employees becoming a long-term reality in many industries.”

I have a tremendous amount of respect for Jennifer and her knowledge of the sector as a reporter is top-notch so this compliment hits home. When the folks at Aquent first approached me about the idea I had the same response Jennifer did — there are far too many companies struggling to get appropriate staff in-house and up-and-running for our industry to really take off in the direction it needs to go (in my humble opinon.)

I view the opportunity to work with Aquent as a big step towards correcting the problem. Our combined clients will get the benefit of working with two industry leaders to build out their internal web analytics expertise.

Thanks for the nod, Jennifer.

Analytics Strategy

Analytics Demystified and Aquent

Posted on May 18, 2007April 8, 2017 by Eric Peterson

On Thursday I was extremely happy to announce Analytics Demystified’s first partnership. We are officially teaming up with Aquent, the world’s largest marketing staffing firm, to provide an innovative “strategy plus staffing” offering built around my experience in the field.
Aquent is an amazing company and were the sponsors for my AMA webcast back in March that was the largest such event in AMA history.

Aquent and Analytics Demystified will be working together to provide end-to-end strategic web analytics consulting services. Analytics Demystified is focused on providing strategic guidance covering the process of “doing” web analytics successfully; Aquent has the proven ability to place great consultants who can then be converted to full-time employees.

This combined offering is GREAT because it allows me to stay well-connected to client organizations as their understanding and use of web analytics processes matures, kind of like having Eric T. Peterson as a senior member of your staff helping provide direction as necessary. The partnership allows Analytics Demystified to scale more efficiently while helping a significantly greater number of clients than I could otherwise.

Oh, this partnership also allows me to work more closely with Nina Eigerman, a member of Analytics Demystified’s Board of Advisors, who has been a huge help to me personally as I’ve put the company together (thanks Nina!) I’m hoping someday Nina will introduce me to her friend shown in this picture.

If you have any questions about how the Analytics Demystified/Aquent partnership could benefit your organization, especially if you’re looking to ramp up your analytics expertise and staffing, please don’t hesitate to email me or call me directly at (503) 282-2601.

Read the Aquent/Analytics Demystified partnership announcement.

Analytics Strategy

Free white paper on Web Analytics Business Process

Posted on May 14, 2007April 8, 2017 by Eric Peterson

Phew, what a week last week was … thanks again to everyone who has been so encouraging about my starting Analytics Demystified. I realize in the midst of getting everything up and running that my blogging has been somewhat spotty (and perhaps boring if you’re not interested in the feedback I’ve been getting about the company, sorry about that!)

In an effort to get back on track, I wanted to let all of you know about a free white paper I published on the “how” of doing web analytics successfully. Many of you have seen or downloaded my PowerPoint from last October’s Emetrics Summit on the subject but I finally had the time to write something more formal.

If you’re interested, you can get this and all of the documents I am now making freely available from this page on my site: http://www.analyticsdemystified.com/wp

I hope to build a feed for my White Papers and Presentations list since I have a bunch of documents and data queued up. Until then, check back frequently as I’ll be pushing up reports, data, and presentations that I will be giving around the globe.

As usual, I welcome your feedback on my work.

Analytics Strategy, General

CondeNet is willing to train you to learn Visual Sciences

Posted on May 13, 2007April 8, 2017 by Eric Peterson

If you live in Manhattan and have been thinking about getting your hands on Visual Sciences technology, the nice folks at the CondeNet division of Conde Nast Publications would love to talk to you. The Web Analytics Systems Analyst position will report to the Senior Manager of Internet Applications and CondeNet is looking for a jack-of-all-trades.

While I usually help companies find good old-fashioned analysts and analyst managers, this position is definitely more technical. Conde needs someone with web analytics knowledge but also experience in shell scripting, Ruby on Rails, and export and integration processes. Part of what CondeNet is doing is exporting data from the Visual Sciences environment into a customized internal data portal, and this position will have an opportunity to build out that environment.

In addition to the technical piece, there will be an opportunity to do analysis directly with the marketing organization. When I talked to the folks at CondeNet they had an excellent example of how this position helped solve a critical business problem regarding a significant change in one of the companies tracked key performance indicators.

The HR person I talked to said they were definitely open to talking to people without direct Visual Sciences experience, and they were happy to send the successful new hire to relevant training courses offered by Visual Sciences. This alone is huge for anyone wishing they had access to the kind of technology I’ve talked about from time to time in this blog. Trust me: It’s not everyday someone says they’ll pay you to learn how to manage Visual Site, much less pay you handsomely to do so.

Did I say “pay handsomely”?! Yep, the salary is VERY COMPETITIVE and they report a strongly held belief in work-life balance that is sometimes missing out there.

Interested? Check out the posting on my premium job board and apply today!

Adobe Analytics, Analytics Strategy, Conferences/Community, General

More feedback from the blogging community about my new company

Posted on May 10, 2007April 8, 2017 by Eric Peterson

I am blown away by the coverage of my announcement that I’ve left Visual Sciences to form my own company. I’m so behind on thanking people who have written about the news I wanted to summarize their thoughts and thank everyone en masse.

E-consultancy in the UK wishes me luck and says that my decision to help companies understand the process of doing web analytics is “bang on” which I’m pretty sure is a compliment. Thanks to Richard Maven for reaching out to me after the press release and I look forward to hopefully meeting the e-consultancy crew when I fly to Europe later this month.
Gary Angel from SEMphonic is a good friend but I was still humbled by his declaration that my announcement was “the biggest news out of Emetrics” Gary says that “[Eric is] consistently thoughtful. His opinions are rooted in real-world work. And see seems remarkably able to deal with web analytics at every level very successfully: from introducing web analytics to marketers to working hands-on with real analysts to helping senior executives get a handle on web measurement process. When you can do that and not get on the nerves of people like me, you’re doing something pretty special.” Coming from someone I respect as much as Gary this means a huge amount to me.
The bloggers at FutureNow congratulate me on not being afraid to “ask bigger questions“
My good friend Judah Phillips (who blogs here at Analytics Demystified) says I have achieved self-actualization. Judah also bought me dinner on Tuesday night after I had been celebrating for several hours over vodka drinks so thanks to both Judah and his fiance.
Jacques Warren, who also recently left the comfort of his day job to become a consultant in Canada and who clearly read the copy on my web site and understands what I’m doing. Jacques comments “Do you realize? This guy is going to make a living telling companies how to organize Web Analytics” to which I can only say, “Yes.”
Anil Batra, who looks nothing like that lousy little picture of him on his blog, had an inkling of my intentions but was surprised by the timing. Thanks for the encouragement, Anil!
My good friend René Dechamps Otamendi from Belgium who has a good summary of my business model and who captured an interesting photo of my singing happy birthday to Aurelie Pols (his wife, and a member of my company’s advisory board.) I consider Rene and Aurelie two of the nicest people I know and am excited about the opportunity to work more closely with them in the future.
Sebastian Wenzel at Web Analytics Book wishes me luck and speculates that I’ll be plenty busy soon.
Marshall Sponder says he’s listening to me when I give him advice, which is good to hear, since he’s now on a WAA board with $250,000 in cash burning a hole in their pocket. Hey Marshall, you should buy a copy of my book for everyone in the organization. 😉

Phew. And that’s just what I culled out of my feeds this morning. I sincerely want to thank all of the bloggers I mention here and anyone I forgot. I’m hugely excited about having the freedom to explore other ways I can help the web analytics and broader business community and hope all of you will keep reading this blog and provide feedback as you think I need it.

Analytics Strategy

Some coverage on my formation of Analytics Demystified

Posted on May 8, 2007 by Eric Peterson

My announcement yesterday has received some coverage in the press:

Web Analytics Guru Eric T. Peterson leaves Visual Sciences in DM News.
Web Analytics Vet Starts Advisory Firm at ClickZ
Eric Peterson Goes Solo from Jeremiah the Web Prophet

You can keep track of all of this news and my citations in the press on my new “Articles and Interviews” page.

I’m back at Emetrics today so if you’re here in San Francisco and want to meet in person, look for me or shoot me an email.

Adobe Analytics, Analytics Strategy, General

Analytics Demystified, Incorporated

Posted on May 7, 2007April 8, 2017 by Eric Peterson

It’s late and it’s been a long day but I wanted to share the news with all of my readers that I announced today at the Emetrics Summit in San Francisco: I’ve left Visual Sciences and am building a company around Analytics Demystified.

You can read the press release here and if you come to my web site you’ll notice some new additions to the site:

Information about the company

http://www.analyticsdemystified.com/about/index.asp

My presentation from Emetrics

http://www.analyticsdemystified.com/sample/Web_Analytics_Demystified_-_March_Survey_Results.pdf

Press release on the formation of the company

http://www.analyticsdemystified.com/link_list.asp?l=PR

Information about my consulting offerings

http://www.analyticsdemystified.com/inc/index.asp

Our free white paper on web analytics business processes

http://www.analyticsdemystified.com/wp

Thanks to everyone who has helped me so far and to everyone who has kept my secret while I got ready to present today. I’m happy to answer any questions you have about my decision to form my own company and please keep watching my blog as I will be making additional announcements over the rest of the month.

Adobe Analytics, Analytics Strategy, Conferences/Community

Can't make Emetrics? You're in luck if you're in the WAA!

Posted on May 3, 2007April 10, 2017 by Eric Peterson

We’re just a few days away from the big event in San Francisco and I admit I’m getting excited. It’s so nice to be able to see old friends (congrats Ian, by the way!) and hopefully make a few new ones. I’ve even started making final preparations for the event.

I’m not sure how they’re advertising this event, but the WAA International Committee is piggybacking on the “Ask the Guru’s Breakfast” that Jim and Matthew have put together and offering the ability to join the breakfast virtually. According to the WAA web site:

30 minute exclusive analytics discussion between Eric Peterson, Bryan Eisenberg and Avinash Kaushik
Virtual attendance at the San Francisco Emetric’s Summit Guru’s Breakfast – your chance to listen in to this one hour session with Eric Peterson, Avinash Kaushik, Bryan Eisenberg and Jim Sterne,
A question and answer session for international attendees

This event is limited to WAA members but my understanding is that the recorded event will be made available to everyone sometime in the coming weeks. If that is not the case, I will have some of my deadhead friends in San Francisco make a bootleg tape and we’ll share via the Archives.

Sign up today to join us in San Francisco, even if you’re not coming to Emetrics!

Adobe Analytics, Analytics Strategy, Conferences/Community, General

Are you coming to Emetrics?

Posted on April 30, 2007April 8, 2017 by Eric Peterson

It occurred to me today that by this time next week I will have already given my “Guru” presentation at Emetrics and will be dining with some of the brightest minds in the web analytics industry. While I have been almost zealous in my recommendation of Jim’s conference for the past six years (am I “old school”, or just “old” … you decide) I have to say I am excited about this particular event more than any other.

Why, you ask?

Robbin’s “Birds of a Feather” Web Analytics Blogger’s luncheon on Monday, which I think was not Robbin’s idea but nonetheless is a wonderful idea! There are dozens of web analytics bloggers now, many of whom will be at the conference, and we’re having a private little meet-up. This is cool since my friend Judah can join us now, having recently joined the blogosphere!
Great looking presentations from some of my favorite people in the industry, including Jim Sterne, Jason and Shane, Brett Crosby, Megan Burns, Robbin, Jason Palmer, John Marshall, Bill Gassman, Seth Romanow, Jodi McDermott, Jen Veesenmeyer, Rand Schulman, Josh Manion, Ian Houston and Judah Phillips, Kevin Heisler, Eric Hansen, Bryan Eisenberg, Bob Chatham, Gary Angel, … okay, so I pretty much just listed half of the presenters. As I look at that, do any of you remember when vendors weren’t allowed to present? I used to be special …
Even greater looking presentations from a ton of people I know of but have never had the chance to meet formally, including Tim Kopp, Lou Rosenfeld and Rich Wiggins, Patrick Moran, Chris Gemignani, Paul Holstein, Lissa Gatz, and more!
The Web Analytics Wednesday on Tuesday special event, which Jim and Matthew have graciously allowed to occur (and more importantly, have fully funded so we can have drinks, etc!) Are you already signed up? If not, sign up right now!
Most importantly, I get to present to the entire audience this time in my “Guru Session” where, despite what you’ll read in the literature, I will be sharing the results of my recent web analytics survey. If you’re a data junkie like me, I promise that you will not be disappointed by my presentation.

Plus, the last five minutes of my presentation are not to be missed! Trust me on this one … there was one number that came out of the research that more than any will change our understanding of the web analytics industry forever. Sadly, you’ll have to wait until 4:45 PM on Monday, May 7th to hear what it was.

Anyway, if you’re coming to Emetrics and you’d like to get together please let me know. I will be in Sunday and in-and-out of the WAA training day so will have tons of time to sit down with folks and catch up.

Oh, if you have another minute, check out this humorous tribute Rene and Aurelie did for Jim Sterne.

Analytics Strategy, General

Ian Houston publishes very interesting cookie deletion data of his own

Posted on April 23, 2007 by Eric Peterson

My friend Ian IM’d me last week and said he had confirmed comScore’s data on first-party cookie deletion. Since Ian is easily one of the sharpest people I know, I was immediately intrigued, given that he has been working on a methodology to restore deleted cookies using the browser’s cache and a dynamically generated script. Unfortunately Ian hasn’t been able to implement his strategy on a high-volume site, yet, but he did do a very robust comparison of measured site traffic data to comScore numbers.

What Ian saw by comparing the number of measured unique visitors based on accepted cookies to comScore data for the same site was, well, roughly a 2.5X inflation from panel to measured visitor counts. His monthly numbers ranged between 1.99X and 3.15X but he reports the average as 2.47X.

Ian also reported data for daily unique visitors where he saw an average inflation of 1.96X (range of 1.35X to 2.84X). Ian commented, and I agree, that the daily numbers are somewhat disconcerting given that they appear to support the notion that “serial deleters” are among the most engaged.

Keep in mind, these numbers are based on a direct comparison to the comScore panel-based numbers, numbers whose accuracy has long been questioned and continues to be questioned today.

As usual, Ian’s writing continues to be well thought out and well written, and I highly recommend reading him if you’re not already. I also want to congratulate Ian on joining the team at WebSideStory/Visual Sciences. The blogosphere loses a great practitioner but gains a great vendor/consultant (and to be fair, Ian has been a private consultant for as long as I have known him.)

Analytics Strategy, General

Welcome to the blogosphere Judah Phillips!

Posted on April 22, 2007April 10, 2017 by Eric Peterson

There was a question posted to the Yahoo! group recently asking about web analytics practitioner blogs. I looked at the post and thought “what a funny thing to ask about …” but it stuck in my head. I’m not even sure if the author got many answers given that most of the folks that pop into my head are all either working for vendors or working in consulting groups.

Some of the best practitioner-bloggers, folks like Clint Ivy and Avinash Kaushik, have gone to work for vendors (and I happen to know of one more such defection about to occur but have been sworn to secrecy, for now …) Of the folks I track in my blogroll, I believe that only Dustin Wallace, Paul Strupp, and Scott Baldwin are real web analytics practitioners-not-consultants-not-vendors … surprisingly few when you consider that the vast majority of people doing web analytics are not working for vendors or consulting firms.

Well all that just changed: My friend Judah Phillips has joined the fray. Not only has Judah joined the fray, Judah is the first practitioner to join me by blogging under the Analytics Demystified brand.

Now, in my experience, Judah has never been at a loss for words so I won’t do him the injustice of putting words into his mouth. He’s written an introductory post and I would encourage you to give it a read.

Welcome to the blogosphere Judah, and welcome to Analytics Demystified.

Analytics Strategy, General

comScore answers a few of my questions about their recent report

Posted on April 19, 2007 by Eric Peterson

As I mentioned a few times in the Yahoo! group, I have been talking to the folks at comScore about their recent report on cookie deletion. I got an email back from Andrew Lipsman with some more information and partial answers to questions of mine and a few passed to me by other cookie-savvy folk.

According to Andrew, comScore will be publishing a more complete document describing their research methodology in the next few weeks. Until then, they’re giving me the scoop so here you have it, direct from comScore (my questions in bold type, comScore’s answers follow in normal type):

(Andrew provided this preamble to his answers …)

The reason we have done this study for two cookies is to ensure that we are very familiar with the cookie structure, the different value pairs (e.g. GUID=1234) and their purpose. We are in particular interested in ID value pair that identifies a user over time, and does not change when the cookie gets refreshed.

How did they identify the unique values of the cookies? Using the Set Cookie Response header, the Cookie Request header, or the actual storage (cookie) file?

We are reading the cookie request call and parsing out specific ID value pairs. Over time we will observe a time series for each panelist for the value of this identifier corresponding to each cookie request. Cookie reset events are based on qualified value changes for a targeted ID value-pair.

How did they take into account non-persistence and/or cookie expiration settings?

The cookie domain value-pairs were chosen to represent passively assigned unique identifiers designed to be persistent over time. Cookies of this nature should only expire in the event that the visitor never returned within a relatively long expiration window.

How did they identify First vs. Third party cookies?

We are reading specific value-pairs for specific domain cookies. The first party cookie is the cookie used by the Portal site. The third party cookie was used by the ad serving company. All information is directly observed from metered panel activity. Recall information was not a source of determining preservation rates.

How was cookie blocking treated or accounted for?

The analysis is based on a sample for which at least one cookie value was observed.

What were the domains they examined? If not the domains, what was the nature of the first-party site?

We will not disclose the names of the sites used for the analysis. First party site is a major internet portal. The third-party site is a major ad server.

What were the survey questions asked? How many people were asked and how were they selected?

All deletion/retention figures were derived from direct panel observations, not from a recall-based survey. Only qualitative information came from the survey.

Obviously some of the answers provided are lacking but I’m willing to admit that it may be more a function of my incomplete knowledge of what the comScore panel application is able to capture.

One particularly good question from a reader essentially asked if P3P-instigated cookie blocking could be artificially running up cookie reset counts (essentially counting each page request as a new cookie) to which comScore answered that the study only included panel members for which “at least one cookie value was observed.” (plus, P3P is less likely to impact the first-party cookies that I’m more interested in …)

The encouraging news is that comScore is now officially on the record as willing to produce additional documentation about the study within the next week or so. I conveyed to Andrew some of the skepticism about the results they report, skepticism I told them they would hear, and pointed him to the ongoing conversation so hopefully the community’s concerns will be directly addressed in their methodology document.

Suffice to say, if some major flaw appears in their research, the company will have major egg on their face as they approach their announced IPO. Conversely, if the research proves sound under examination, regardless of whether you’re a data purist looking for “perfection” or willing to manage based on trends however flawed the underlying data might actually be, we all have something to consider the next time someone asks us “how many visitors come to your web site?”

Perhaps the only true and precise answer is, “It depends!”

What do you think about the answers that comScore provided? As always, your comments are greatly appreciated!

Analytics Strategy, General, Industry Analysis

comScore study sheds new light on risks to cookie-based measurement

Posted on April 16, 2007April 8, 2017 by Eric Peterson

Awhile back the folks at comScore called me and asked if I would be surprised to learn that cookies were being deleted at a pretty high rate. Of course I said, “No, because I reported as much in 2005.” Through the course of the conversation, however, it became clear that comScore had the ability to shed new light on our understanding of cookie-based measurement; specifically they had the ability to measure the rate of deletion associated with first-party cookies.

comScore published the results of that study today.

I will fight the temptation to smugly say, “Ah ha! I told you so …” since the comScore data shows that I was both right and wrong when I first wrote about cookie deletion when I was with JupiterResearch. I was right in my assessment that this is happening far more frequently than those of us in the web analytics field particularly want to believe. But I was wrong in my assumption that cookie deletion was largely limited to third-party cookies.

The comScore data reports that over 30 percent of their panel of 400,000 home user computers deleted both first- and third-party cookies. Now, when I talked to Andrew Lipsman and Gian Fulgoni from comScore I repeatedly encouraged them to check and double-check these findings since especially their number for first-party cookies is much, much higher than I think any of us expected to see.

That said, I have no reason to believe that comScore would make this claim frivolously (okay, except for the fact that they provide a competing methodology to cookies) … I have asked comScore for a deeper briefing on their research but nothing has been scheduled as of this posting. Perhaps on my urging comScore took their research a step further and surveyed a subset of their panel asking about their stated behavior towards cookies. In the press release, Dr. Magrid Abraham addresses this in the context of the conventional wisdom that assigns greater risk to third- than first-party cookies:

“There is a common perception that third-party cookie deletion rates should be significantly higher than first-party cookie deletion rates,” continued Dr. Abraham. “Because many PC users reset or delete their cookies using security protection programs, conventional wisdom dictates that people are more likely to selectively expunge third-party cookies – which are generally deemed more invasive – while maintaining their first-party cookies. But these findings suggest that selective cookie management is not prevalent, a fact that comScore confirmed via a survey, with only 4 percent of Internet users indicating that they delete third-party but not first-party cookies.”

Yikes. When you look at the tables in the comScore study you can see where the problem is coming from: serial cookie deleters, the 7% of site visitors (measured via the comScore panel) that are repeatedly removing their cookies and thusly will appear as a new site visitor with every visit. I addressed the idea of serial deleters in my final JupiterResearch report on “The Crumbling Cookie” and, at the time speculated that some of the more nefarious activities available through the Internet were to blame.

Still, I never would have put the number as high as 7 percent.

It’s interesting to me that cookies are back in the news. It will be more interesting to see how all of this is digested in the coming days, weeks, and months. I wonder if Seth Godin will comment on the comScore study? I mean, I’m not sure that the “echo chamber” argument applies to comScore’s panel of 400,000 measured, identified individuals.

This seems to be a topic ripe for commentary and conversation. What do you think? Is comScore crazy? Is this report flawed? Or are we just fooling ourselves when we believe that “unique visitor” counts are an accurate representation of the number of real human beings coming to our web sites over long periods of time?

Adobe Analytics, Analytics Strategy, General

Want to see the results of my recent web analytics survey?

Posted on April 14, 2007April 8, 2017 by Eric Peterson

So after Zori and I received 1,076 total responses to our recent web analytics survey we realized we had more than enough data and, gasp, needed to close the survey and start the analysis. Zori was emphatic that the fact that over 1,000 people worldwide responded to the survey was a very enthusiastic response so I thank all of you who participated.

If you’d like to see the results of my survey you have two options:

Go back to my site and give me your email address so that when the first report is available I can send you a note.
Come to my “Guru” presentation at 4 PM on Monday, May 7th at Jim Sterne’s Emetrics summit.

Again, thanks to everyone who participated!

Analytics Strategy

University of Fairfax, VA needs some help with their reporting

Posted on April 12, 2007 by Eric Peterson

I got an email a few days back from the President of the University of Fairfax, VA looking for someone with WebTrends expertise to help them out.

I don’t get to VA nearly often enough to help out but thought that someone reading my blog might be willing to lend a hand. I don’t know exactly how they’re defining “limited budgets” but I suspect this is an excellent opportunity for someone just learning web analytics to get their feet wet and help out a good cause in the process.

If you’re interested in getting the contact information, ping me directly.

Analytics Strategy

Seven free resources to get started with Google Web Site Optimizer

Posted on April 4, 2007 by Eric Peterson

I’m sure by now you’ve heard that Google started giving Web Site Optimizer away. But you may not have heard about all the great content that the gang at FutureNow has to help you get started on the platform? Guides, plug-ins, podcasts, presentations, … it’s all there!

It makes sense that Bryan and Jeffery would provide all this information given that they’re Google Authorized Consultants, but still, it’s awfully nice of them.

I’ve been running the platform on my web site for a few months and I love how easy to use it is. It may not provide all the rich features of an Offermatica, but it’s a great place to start if you have been hesitant to try A/B testing because of the price.

Check out Google Web Site Optimizer and check out the free resources from the Eisenberg brothers today!

Analytics Strategy

Please take 15 minutes to participate in my web analytics survey!

Posted on March 31, 2007April 8, 2017 by Eric Peterson

Many of you have probably already seen the invitation given that I emailed it to well over 10,000 people around the world but just in case … a former JupiterResearch data analyst and I are currently conducting a survey covering web analytics. The survey is totally anonymous and we’re asking what we believe to be a series of valuable questions regarding both personal attitudes and organizational uses of web analytics.

The response so far has been phenomenal and we’re closing in on 600 responses but my goal is 1,000 responses (which my partner laughed at since nobody, nobody gets a 10% response rate. I told her that web analytics people were different and they understood the value of collecting good attitudinal data …)

If you haven’t yet completed the survey, you can participate via this URL:

http://www.surveymonkey.com/s.asp?u=353583551869

To thank you for the small amount of time it will take you to complete the survey I’m offering two incentives:

A free copy of the aggregated survey results, including analysis, available through my web site in mid-May
A 25 percent discount off of the purchase price of Analytics Demystified and The Big Book of Key Performance Indicators.

Again, the URL you need to participate in this survey is:

http://www.surveymonkey.com/s.asp?u=353583551869

If you have any questions about the survey I welcome them here in comments or via email if you’d like to be more discreet. There is more information about the survey on my web site.

Analytics Strategy

Eric Enge interviews Bryan Eisenberg

Posted on March 27, 2007 by Eric Peterson

Eric Enge has a great interview with one of my good friends Bryan Eisenberg. Eric is a great interviewer and Bryan is a great subject. Makes for good reading. Check it out!

Analytics Strategy

Interview with web analytics job seeker Jaimie Scott

Posted on March 26, 2007April 8, 2017 by Eric Peterson

I got such good feedback after interviewing my friend June Dershewitz back in February I’ve decided to keep talking to some of the bright people I know who are actively looking for work in the web analytics industry. Recently I had the chance to interview my friend Jaimie Scott, formerly of Intel and one of the original Web Analytics Wednesday participants here in Portland, Oregon.

Jaimie has always struck me as someone who has really figured out the whole “work/life balance” thing that so many of us talk about but so few of us seem to pull off. Since Jaimie doesn’t have kids, I find myself often living vicariously through Jaimie as he describes hikes and trips he takes around the state and in the Northwest. That’s not to say that he’s not into web analytics … he definitely is.

My interview with Jaimie is as follows:

Eric T. Peterson: Jaimie, thanks for joining me this evening. First question: Tell me a little bit about your experience with web analytics systems?

Jaimie Scott: I’ve been doing analytics for two years, mostly on the Omniture platform.

Much of that experience has centered around establishing KPIs and fixing the implementations to collect the right data.

The emphasis has been primarily on optimizing the site design and marketing efforts for each company.

Eric T. Peterson: Very cool.

Do you mind saying where you worked? Since you’re here in Portland, I know your employer is pretty well known locally …

Jaimie Scott: No, I don’t mind. I worked at Intel for 12 years, doing web analytics for the last 18 months, and then at Powell’s Books after that.

Eric T. Peterson: At Intel, how did you get introduced to web analytics?

Jaimie Scott: Well, that’s long story, but I’ll try to keep it brief.

In my 12 years there I made five career changes, which is pretty common at Intel.

Each time I changed careers I figured out a little more about the types of things I enjoyed doing.

All of the positions I held had a few things in common including data analysis, translating between technical and non-technical folks, and communication.

When my next to the last position went away, I came across the analytics position and it sounded intriguing to me.

So I applied and got the job and that was how I got my start in web analytics.

Eric T. Peterson: I’ve found, in my research, that the ability to translate technical findings for non-technical people is critical. Can you describe briefly one instance where you were really successful doing that? You don’t have to be too specific …

Jaimie Scott: At Intel I was tasked with creating a dashboard that would at a glance tell the managers that used the data what they needed to know.

Each manager that used the dashboard had their own requirements.

So I had to evaluate their individual business needs and the available data and then develop an Excel dashboard with logic and pivot tables to represent data in a way that would empower management to see enabling program performance by business group, functional team, field audience, and geography.

I accomplished this in a single dahboard that allowed each user to select their own pivot tables to view the data that was important to them and they were ecstatic with the result.

Eric T. Peterson: Very cool. So you are adept with Excel then I take it?

Jaimie Scott: I am.

Eric T. Peterson: Excellent! So tell me this, what do you like BEST about web analytics? What gets you up in the morning, as it were?

Jaimie Scott: I’d have to say it’s the challenge. I really enjoy trying to glean understanding of customer behavior, and the motivations for their behavior from pouring over the data.

Eric T. Peterson: What, would you say, is the hardest web analytics problem you’ve ever run up against?

Jaimie Scott: Well, I suppose that would be establishing KPIs for a non-retail site for which the website owners were not clear on what they hoped to accomplish with the site.

That makes things very difficult.

But I learned quite a few tricks in the last UBC course that should help me the next time I encounter that situation.

Eric T. Peterson: Hey, speaking of, you’re one of the first WAA UBC class graduates, right? What did you think of the courses?

Jaimie Scott: I am. I really enjoyed the courses and look forward to applying everything I learned.

I thought that the last course in particular, Creating and Managing The Analytics Businesss Culture did a great job of tying the material from the previous courses together and filling in the gaps.

I felt that the content in all of the courses was excellent. I’d strongly recommend the courses to anyone considering taking them.

Eric T. Peterson: Did you have a favorite instructor? (Note: I am not asking Jaimie to say “Eric Peterson” since I am not a teacher in the UBC classes …)

Jaimie Scott: Yeah, but I can’t remember his name. Better not include this.

Let’s just say they were all great.

Eric T. Peterson: Would you say you’re more technically minded or more business minded regarding web analytics? More of a coder or an analyst, or both?

Jaimie Scott: Well, I’m not sure how to answer that. I’d say I’m more technical than business oriented only because my math and engineering background is more technical.

I’m definitely more analytically inclined than a coder. I’ve written a fair amount of code in my day and let’s just say I’m glad I’m not a programmer by trade.

It’s kind of funny. I took one of those “social style” classes at Intel and I was told that I was an analytical-analytical by nature.

That means I am more detail-oriented and deliberate in my pace even than other analytical types, so I have to be careful in that regard.

Eric T. Peterson: Okay, last few questions:

You’re looking for a job doing web analytics, right? Describe your ideal job …

Jaimie Scott: Yes, I am. I really want to find a position that will give me the chance to apply what I’ve learned in my last two positions and the UBC courses. I’d say if I had my choice, I’d like to find a local company that would allow me to focus on an e-commerce site.

To really delve into the site and all of the available data sources and understand customer behavior.

Working with the marketing and design teams to optimize conversion on the site.

And since we are talking “ideal” I’d rather not travel more than a few days once or twice a month.

Eric T. Peterson: Great answer!

You mentioned the Omniture product, have you worked with any other web analytics platforms?

Jaimie Scott: Very little to date, although I just entered into a temporary collaboration today with a local company that will hopefully give me some more experience with Google Analytics on a retail site.

I’ve had some exposure to other platforms since I read a number of blogs and the Yahoo Newsgroup daily, but other than that I haven’t really worked on other platforms. Yet.

Eric T. Peterson: Good answer. Last question. If my readers or recruiters want to contact you, what is the best way to get in touch?

Jaimie Scott: jaimie.scott@gmail.com

Eric T. Peterson: Excellent!

Jaimie, thanks so much for talking to me this evening. I owe you a beer at the April Web Analytics Wednesday here in the Rose City … unless you find your next job because of this blog post, in which case you can buy the first round …

Jaimie Scott: Yeah! Thanks Eric!

If you’re like Jaimie and looking for a new job in web analytics, you should check out my premium job board where there are always great jobs listed.

Analytics Strategy

Are you an experienced business manager who knows web analytics?

Posted on March 24, 2007April 8, 2017 by Eric Peterson

If so, your dream job just happens to be posted on my premium job board! The posting is calling for a “Director Web Analytics” but after talking to the VP of Global Marketing and Director of Web Marketing at this unnamed company I can tell you that this job is a whole lot more than that. Just have a look at the RELEVANT EXPERIENCE section of the posting and you’ll see what I mean … to get this job you’ll need to have a ton of experience, but man oh man, the things you’re going to get to do!

I can’t name the company but I was really impressed with their attitude towards online marketing: They focus on web marketing FIRST and back up to offline marketing secondarily. The successful candidate will play a huge role in how the company spends money and this role will have direct access to the CEO. This is a high-profile position to be sure.

When I asked about how they’re already set up for web analytics I was surprised by what I heard. This company is absolutely doing it right in the B2B space — they have their analytics integrated with their marketing automation system and SalesForce.com such that they’re able to generate a weekly dashboard reporting against the on-site activity of a group of “watch list” companies. Their sales staff rely on these dashboards to make key decisions about prospect status and complex RFP processes. This process alone has helped the company generate a nine-figure sales funnel, which has in turn ensured that when the VP of Global Marketing goes asking for money for projects, he never hears the word “no.”

This company is doing predictive marketing based on their web analytics data. How often do you hear that?
When I commented that they sound like they’ve got everything pretty much dialed-in and asked what the new hire will be responsible for, they indicated that it’s all about “Web 2.0” at this point. They need a forward-thinking strategic player who can help them bring emerging Internet technology into the B2B marketplace. It took their competition three years to catch on to their investment in PPC marketing, and now they want to create a similar gap based on podcasts, vidcasts, mobile marketing, RSS, etc.
The candidate who gets the job will manage a big outside design and development agency, the company that manages their SEO/SEM strategy, an internal web analyst and will support marketing around the world. The job is in Baltimore, MD and they are willing to relocate the right person.

Check out this position and apply today.

Analytics Strategy

comScore adds sessions to their reporting

Posted on March 16, 2007April 8, 2017 by Eric Peterson

A bunch of you read and commented about my post from February titled “Afraid of page views dying? Don’t be!” in which I proposed that we use “visits” or “sessions” instead of “page views” or “unique visitors” (sic) to compare traffic across different sites. It appears perhaps that someone at comScore was paying attention as well. According to their press release:

“As technologies like AJAX change the Internet landscape, certain measures of engagement, such as page views, are diminishing in significance for many Web properties,” said Jack Flanagan, executive vice president of comScore Media Metrix. “The introduction of these new metrics based on ‘visits’ provides an alternative for measuring user engagement that tells us how frequently visitors are actually returning to the site to view more content.”

I don’t necessarily agree that visits tells us that much about visitor engagement but I do agree with comScores reasoning behind using visits. I actually suggested something like this to comScore’s Andrew Lippsman on February 15th, 2007 when I wrote him:

“What prevents you guys from reporting on SESSIONS rather than page views or unique visitors? Anything?”

Far be it from me to assume that my email had anything to do with comScores decision to roll sessions into their reporting … let’s just assume that great minds think alike!

So here is a test of the value of comScore’s decision: For all the site operators in the world who have ever complained about the differences between comScore’s unique visitor counts relative to their own web analytics tracking, compare your session (= visits) counts to what comScore reports. Because, as I said back in February, sessions is a much more clearly defined and less controversial metric, my suspicion is that you’ll see much greater alignment between the two systems than you were seeing based on UVs.

I would LOVE to get comments or email from anyone using both systems (comScore and traditional web analytics) to see what correlation (or lack thereof) you see. If I’m right (not saying I am), even without bugging Andrew and Gian Fulgoni about the specifics behind their “visit” counting, I bet we’re going to see far less variation when comparing visits between these technologies, thusly reinforcing the idea that sessions (= visits) IS the right metric.

Either way, thanks to comScore for being open minded enough to revisit the numbers they report to tell a more complete story to their customers and the rest of us too cheap to pony up for their reports but are still paying attention.

(Thanks to Marshall for catching the comScore press release, by the way …)

Analytics Strategy

How you know the tech economy is booming back …

Posted on March 15, 2007May 11, 2017 by Eric Peterson

… no parking spaces in Silicon Valley. On my way to the very successful WebGuild event last night I dropped by the Googleplex to chat with their newest consultant, my friend Avinash Kaushik. I had no idea that it would take me 20 minutes to find a parking space at Building 41 but I guess that means that “Google is hiring!”
Then I dropped in on the nice folks at SimplyHired who power my premium job listing service and again drove around in circles looking for parking.

When I went to Google’s Building 44 I was able to find a parking space … the last one … probably only because I was 45 minutes early for the event. All of this was in stark contrast with visits to Mountain View just a year or two back when parking was plentiful and cubicles were empty.

The WebGuild event, by the way, was a HUGE success by all measures. There were well over 300 people in attendance including Bay Area analytics luminaries like Avinash, June Dershewitz, and Adobe’s Fred Kuu. The audience had great questions and I was fortunate enough to sit on a panel with two incredibly bright guys representing WebTrends (Barry Parshall) and Google Analytics (Paul Hutton).

If you’re in the Silicon Valley and not a member of the WebGuild, you’re missing out. I strongly recommend checking the group out and joining. They have a conference in June on “Search 2.0” that I’m trying to present at, so maybe I’ll see you there!
If you were at the event last night I’d love to hear your comments on the presentations and panel session.

Analytics Strategy

WebGuild event at Googleplex, Wednesday March 14th

Posted on March 13, 2007May 11, 2017 by Eric Peterson

If you live anywhere near the Googleplex in Mountain View, and if you have $25 burning a hole in your pocket, you should join me, Barry Parshall from WebTrends, and Brett Crosby* from Google Analytics at the WebGuild Web Analytics Panel tomorrow night (Wednesday, March 14th.) It’s largely a Q&A format so likely you’ll get to hear Barry and I swap Rand Schulman stories and argue about which of our CEOs is the nicest (It’s mine, by a hair …)
Oh, and I have something special planned for my five minute presentation. I can’t tell you what just yet, but you web analytics wonks will want to hear it, trust me. No, it’s not my new haircut … nor is it stories from Baby Loves Disco … it’s about web analytics. Serious web analytics.
Anyway, if you make it, come say “hi” and provide blog comments the old fashioned way, over free drinks.
(* Mr. Crosby is rumored not to attend due to a previous engagement.)

Analytics Strategy

Webcast link for my AMA/Aquent presentation last week!

Posted on March 11, 2007April 8, 2017 by Eric Peterson

Late last week the folks at American Marketing Association sent me the following link to the full webcast for the presentation I gave on their behalf titled “Analytics Demystified: Ten Simple Strategies for Using Web Analytics to Improve Your Online Marketing Efforts” If you missed the original presentation, or if you suffered from webcast-related issues, you can now access the event from the following link:

https://amaevents.webex.com/amaevents/onstage/g.php?AT=VR&RecordingID=402661864

You do have to register with AMA/WebEx to see the presentation but a lot of folks had already registered. If you have any questions about the presentation, I welcome your comments directly.

Thanks again to everyone who helped make this event such a huge success!

Analytics Strategy

Do you have little kids?

Posted on March 11, 2007May 11, 2017 by Eric Peterson

(Sorry, this post is hugely off-topic for a web analytics weblog but some days a dad just has to be a dad …)

If you have little kids like I do and if you were born in the late 1960’s or early 1970’s you have to know about Baby Loves Disco. My wife heard about this event from friends and we went to the Portland, Oregon event today. The web site says it all but I have to tell you, when despite the “Disco” moniker, the DJ played a Ramones mix that started with “I Wanna Be Sedated” and the heads of roughly 500 moms and dads all started bopping to Joey, Johnny, Dee Dee, and Marky, a huge smile spread on my face that didn’t leave until we were well on our way home.

No matter how young your kids, no matter how old you are, check out Baby Loves Disco when it comes to your town. You won’t regret it.

Analytics Strategy

Some days you just feel more loved than others …

Posted on March 6, 2007April 8, 2017 by Eric Peterson

… and today is one of those days. Not only did my AMA/Aquent webcast apparently draw the largest online audience in AMA history, but Anil Batra keeps uncovering web analytics professionals that are going on record as having benefited from my writing. Have a look at quotes from the following practitioners:

“Mr. WebTrends” Michael Notte
Dennis Digital’s Joel Collymore
Adam Berlinger from Deloitte and Touche
Julien Coquet, part of the Web Analytics Dream Team in Belgium

I have to say, Anil has come up with one of the most interesting series I’ve read on the subject in quite some time. I’m humbled that so many brilliant and talented people have read and benefited from my writing.

Adobe Analytics, Analytics Strategy, General

Guest blogger: Robbin Steif from Lunametrics!

Posted on March 5, 2007April 10, 2017 by Eric Peterson

[ I’m really happy to have my first “guest post” from blogger Robbin Steif from Lunametrics. Robbin really liked my “gradual building of context” post from awhile back and she and I have been discussing a related metric that she thinks builds nicely on my visitor engagement metric. Without further ado, Robbin Steif … ]

On the one hand, I thought that Eric’s recent post, The Gradual Building of Context, was just awesome. Although every site has to define visitor engagement for itself, every site is still capable of pulling together similar numbers (which is why I loved it.)

On the other hand, I disagreed with Eric’s final conclusion, “I need to reach out folks like Matt, Marshall, and Clint and see if there is some way I can get them to more passionately advocate for my books in their weblogs. Given that their visitors are more highly engaged than the “average visitor’, I have to believe their is an opportunity to sell more books.”

I took one look at the numbers in the last chart and thought, well, that doesn’t make sense. Sure, Matt’s visitors (or Clint’s or Marshall’s) are somewhat more engaged than the average visitor, but their “start the purchase cycle” numbers are pitiful. If Eric were to put effort into this, the place to put effort in is where both those metrics are strong.

Eric was good enough to send me the spreadsheet, and I pushed the numbers. (Well ok, technically he and I pushed the numbers at the same time over the phone …) On the phone, he called it “Robbin’s Metric” and I left it that way. It is the product of his Visitor Engagement and Percent Buy Path Sessions:

[ Ugh! Yes, I know that image is hard to read! I will correct ASAP!!! ]

By multiplying the two metrics and then ranking all the referring blogs by that metric, you see where Eric should put in extra effort. I agree witrh the first conclusion that Eric already came to in his blog, i.e. that he needs to work out some kind of deal with Anil. However, the two blogs where he should put time/effort would be Justin Cutroni’s and ROI Revolution’s. Interestingly, they are both Google Analytic blogs, so there is a decent chance that the reader is newer at analytics and probably could really benefit from Eric’s books. I didn’t highlight Steve Jackson’s blog, Xavier’s or Aurelie’s because they are already converting well (if there is such thing as converting well.)

Finaly thoughts: An engaged visitor to a site that is also content rich, like Eric’s, doesn’t necessarily make a good customer. In fact, Clint did a survey on his blog and saw that many of his visitors already own many, if not all, of Eric’s books. When visitors go to the beginning of the checkout, we can actually see interest in the purchase, as well as interest in the content — and as direct marketers know, you should always pursue the customer who already has a propensity to buy.

[ Thanks to Robbin for taking the time to take visitor engagement to the next level! What do you think? Is Robbin on the right track? Did I miss the mark? As always, your comments are greatly appreciated! ]

Analytics Strategy

Avinash Kaushik plans to donate his book advance to charity

Posted on March 1, 2007April 10, 2017 by Eric Peterson

I consider myself incredibly lucky. I have a wonderful family, two beautiful kids, a nice house, and a great job. I’m also lucky because I know some of the most wonderful, intelligent, witty people in the web analytics universe. People like Bryan and Jeffery Eisenberg, Jim Sterne, Gary Angel, and Robbin Steif. People whom every time I talk to them I get all energized about what we’re able to do with web analytics and how our craft is positively impacting businesses all around the world.

But my hero of the day is Avinash Kaushik.

You all already know this since I know you read his blog as actively as I do, but for the record, Avinash is one brilliant cat. I don’t agree with everything he says, but even when we disagree, we disagree professionally not personally and quickly agree to move on.

One thing I do agree with is Avinash’s decision to donate his book advance to two charities, The Smile Train and Doctors Without Borders. Like many of you, I was nearly brought to tears when I read about Avinash’s daughter Damini having to have another major surgery. Having a daughter myself, it pained me to think about how scared I would have been in the same situation. Reading about how brave Damini is made me respect Avinash and his wife even more (if that’s possible), knowing that our children reflect us in so many ways.

And so when Avinash puts his money where his mouth is and makes this kind of gesture to the universe, I am humbled. I am absolutely humbled to know Avinash and to consider him a friend.

I strongly recommend you check out Web Analytics: An Hour a Day and pre-order your copy from Amazon.com today.

Analytics Strategy

Speaking of my good friend Bryan Eisenberg …

Posted on February 28, 2007April 8, 2017 by Eric Peterson

I just got done reading Bryan’s new book with Anthony Garcia of FutureNow, Inc. titled “Which Sells Best? A Quick Start Guide to Testing for Retailers” which you can get from the FutureNow web site for under fifteen bucks. Controlled experimentation is one of the most critical of the Web Analytics Business Processes, perhaps the most critical. But frequently folks I talk to complain that they’re interested in doing testing but there is some barrier to their getting started …

… well, knowing how to start is no longer a barrier.

In their book, Bryan and Anthony break down the process from beginning to end and cover all the bases. The chapters on “Know Thy Customer” and “Factors That Determine Online Success” are uniquely Eisenberg and uniquely FutureNow, which is great, because these guys know their stuff!

One of my favorite quotes in the book is this: “Proper testing is always hard work.” Bryan and Anthony don’t sugar coat it, but they’re clear that what you’re working to do with experimentation is not figure out what is best but rather what is best for your customers. In my experience, the difference between “best practices” and “best for your customers” is often millions of dollars.

Get your copy of “Which Sells Best? A Quick Start Guide to Testing for Retailers” right now!

Analytics Strategy, General

Playing catch up again after a family outing to SeaWorld!

Posted on February 26, 2007April 10, 2017 by Eric Peterson

Right after I got a new hair-do I took a quick trip to San Diego for BusinessOnline’s Online Marketing Summit which was great. Aaron Kahlow really did a fantastic job getting over 300 marketers to attend his first-ever event and the conference was very well done. If you’re in online marketing, definitely keep your eyes open for this event in the future.

I was fortunate enough to take my kids to SeaWorld and enjoy two crowd-free days in the sun, watching Shamu and petting the bat rays. While I was away, I missed a few things that I thought were worth passing along:

Gary Angel from SEMphonic who I am fortunate to speak with frequently had an interview he and I conducted published in ADOTAS. Gary prefaced the interview by commenting that he wasn’t much of an interviewer but somehow he managed to get the single most interesting interview I have ever given on the subject of web analytics ever. We talked about a variety of subjects including my AMA/Aquent webinar on March 6th, the Web Analytics Business Process, and the notion that web analytics professionals are very much craftspeople.
Jim Novo, a true web analytics guru, picked up on my recent rant about the death of page views … NOT! and did a little analysis of his McAfee anti-virus application and how the application treats tracking cookies. The thing I love the most about Jim is how practical his guidance is, especially regarding sessions as a replacement for unique visitors and page views. Jim closes his post with this:

“My advice to you is to start shadow tracking now using Visits or Sessions as the base in your most important metrics, the ones you are on the hook for. You don’t have to show them to anybody, just keep track of them in Excel or something and note the trends. Then when you start seeing your Unique Visitor based metrics collapsing on you, you can whip out the Visit / Session based metrics and say, “See! See! It’s really not happening! We’re doing much better than you think!”

The often-cited-and-always-delightful Mr. Avinash Kaushik was interviewed by SearchEngineWatch’s Eric Enge. Avinash is always delightful to read, especially more so for me when Mr. Enge asks him about me an my recent work on measuring visitor engagement and Avinash replies:

“You know that Eric is obviously a leader in the industry. We are all following the trail that Eric has blazed. He is just an awesome guy and a really great thinker.”

I am truly humbled. If you’re not reading Gary, Avinash, and Jim … you should be!

Analytics Strategy, Reporting

Worried about page views dying? Don't be.

Posted on February 15, 2007April 10, 2017 by Eric Peterson

I found myself thinking, “Are we really having this conversation?” today after reading Steve Rubel’s post today on “What will replace the allmighty page view?” where Mr. Rubel commented:

“The page view is on life support. It fails to capture all of the myriad of ways consumers engage in online activities without ever leaving a web page.“

Okaaaaaaaaaaaaay.

I suppose Steve is coming at this from a different perspective than anyone who works in the web analytics field, more-or-less looking at page views as a basis for comparing the relative value of one advertising opportunity to another. If that’s the case then yeah, page views are becoming increasingly limited in their utility.

But damn, as a web analytics professional, doesn’t all this talk about page views going the way of the Dodo bird just make your stomach feel all funny? Like, you know there are problems with the metric, but A) when compared to the other problems web site operators have vis-a-vis counting (cookie deletion, cookie blocking, poor implementations, caching, robots, lack of understanding, lack of interest) and B) when put in the context of the number of sites that still rely on good old fashioned HTML, don’t these proclamations seem a bit premature?

Is it just me? Maybe it’s just me …

Anyway, we can stop worrying about dying pages and dying page views now since the answer has been with us the whole time. It’s not unique visitors … too many problems with how unique visitors are counted, what with cookie deletion and some of the inaccuracies ascribed to panel-based services. It’s not time spent on site … the problems with this metric as the basis of comparison are many (connection speed, amount of content, quality of content, bathroom breaks, etc.)

It’s sessions.

Yep, sessions. Good old “start ’em with the first page view and stop ’em after 30 minutes of inactivity” sessions. And while they don’t necessarily solve the problem of how many impressions a site can serve (you need old fashioned web analytics for that), they provide a stable basis for comparison across sites:

Sessions are defined by a widely-used and widely-understood standard, the 30 minute timeout between subsequent page views. Heck, in the web analytics industry, it’s pretty much the only standard we have …
Sessions are counted once and only once when a visitor goes to a web site in a single web browser and are thusly not subject to inflation due to crappy web design or RIAs. No more complaints about MySpace!
Sessions are time independent, except for the session timeout. You can click away all day and you’ll still only count one session, unless you walk away for 30 minutes and one second …
Sessions mitigate out issues associated with error pages and the such, because again, the number of pages viewed is irrelevant after the visitor views the first page. Again, no more complaints about MySpace …
Sessions are not affected by cookie deletion and are not always affected by cookie blocking. Whoopie! We can stop bugging out about cookie deletion …
Sessions are not affected by users visiting sites from multiple web browsers, since regardless of location (home, work, etc.) the session is counted. Hurrah! No more massive over-counting of unique visitors during Fantasy Football season …
Sessions can be counted even when the visitor is not on your web site, depending on what tracking technology you’re using and how it’s deployed. For example, a session can be counted when someone reads a post in their RSS reader …
Sessions are easily tied back to relevant referring sources, such as advertising units, RSS feeds, search terms, etc. Yippie! Not only do we get more accurate counts, we know from where the sessions are originating …

Yep, good old fashioned sessions … who’da thunk it? You can call them “visits” if you’d like!

What’s better is that the reporting networks should just as easily be able to report on sessions as they do unique visitors. If they can report on “unique searches” and “time per person” and “page views” and all that, nothing should theoretically stop them from using “sessions” as the basis for reporting.

Clint Ivy pointed out to me that Hitwise uses sessions as the basis for their reporting platform, only they report however on percent market share and not the actual number of sessions which is almost certainly what advertisers would prefer to see. Neither of us were sure why they don’t give raw session counts, do any of you?

Just think of all the problems we can solve by using sessions to compare the popularity of web sites! No more complaints about newspaper sites reporting more unique visitors than live in the entire state. No more complaints about huge differences in reported numbers ascribed to cookie deletion. No more freaking out about inanimate objects dying …

What do you think? Am I crazy? Is it just me? As always, I welcome your comments.

Adobe Analytics, Analytics Strategy, Reporting

Measuring social activities online using my visitor engagement metric (Part V in a series)

Posted on February 15, 2007April 8, 2017 by Eric Peterson

(If you need to catch up on where we are to date, have a look at my last post in this series on measuring visitor engagement.)

I had a nice conversation a few days ago with Jeremiah Owyang, Web Strategist at PodTech.net, on how I have been measuring engagement. Jeremiah has been thinking about how engagement is defined for some time and had a very fresh perspective on the subject which has somewhat expanded my thinking on the subject. Jeremiah, by virtue of being an “A-list” blogger (IMHO) gets great critical feedback from folks like Forrester’s Charlene Li (who says that my measurement is too explicit, oh well …) After we talked, I realized that I really needed to get the promised post on measuring “social engagement in a Web 2.0 world” out the door. So here it is.

One of the links that Jeremiah references is this one from Wiredset, published in November of last year. In their post, Wiredset gives a definition of engagement as “a consumer based measurement that regards interaction with an aspect of a brand or media property” and goes on to say that “Web 2.0 Engagement” could include activities (Jeremiah refers to these as “gestures”) like:

Publishing
Creating and Publishing to a Group
Posting
Subscribing
Favoriting
Adding Friends
Bookmarking
Emailing
Distributing
Streaming
Networking
Creating Mash-up Content

I absolutely agree with Wiredset, and they go on to say:

“When measuring engagement, the level of user interaction (i.e. 200 vs. 2,000,000 streams) is an obvious and important component. Yet engagement is complex in that it is not comprised solely by clicks, but also a range of involved user actions.“

If you’ve been reading along the entire time, you’ll note that my current definition of visitor engagement is derived exclusively from click-stream data and it tries to be as independent of content as possible. While this makes sense for a lot of reasons, the larger conversation (as Clint and Jeremiah wisely point out) is about how a visitor engagement metric can help us better understand the value of emerging Internet technologies.

While Analytics Demystified is not your typical Web 2.0 or social community site, I have enough of the activities listed above on my site to apply a social media filter to my measurement calculation and look at the effects. Again, if you’ve been reading along, I covered many of these in Part III of this series.

Here is the list of things that I am tracking vis-a-vis social media/Web 2.0 on my site:

Now, up until this point I have basically fought applying any weighting to the visitor engagement metric, mostly because I think it’s pretty difficult to rationalize any particular weighting over another and it will complicate what has already been described as “the mother of all KPIs”. That said, I am scoring these social activities into what I call an “interaction index” (ratio of sessions with one of the activities above vs. sessions without) and using the interaction index to weight the visitor engagement metric.

So instead of the existing definition of visitor engagement:

We have the new definition of “Social Engagement”:

Both metrics are the sum of component indices divided by seven, so you can hopefully see that the latter metric is weighted by any contribution made by the “Interaction Index”. For definitions of the component indices, please see Part IV in this series.

So what does this give us? Well, if you were interested in tracking individual users based on their level of visitor or social engagement, you would be able to drill-down along each Web 2.0 activity and perhaps learn something interesting:

There is Frank Faubert from Unica again, not much more socially engaged with my site than he is otherwise engaged. Remember that Frank initially complained about his only having a 21 percent engagement score, to which I responded that I had lost him in my data. Well, I found him, and based on the evolving calculation, Frank is over 31 percent engaged but little of his measured engagement is “social” in nature.

But what if I drill-down along each of my defined social activities, what can I learn?

First we can see my good friend Jeff Katz, formerly of WebTrends, who is a regular reader of my blog and whose social engagement score is much higher than his visitor engagement score. Jeff has repeatedly joined the community (Web 2.0 Measurement Working Group, Web Analytics Wednesday attendee) and has also hosted a WAW event here in Portland, OR.

Looking at direct engagement via email, we can see the great Aurelie Pols from OX2 Belgium who has also submitted comments to my blog.

I can also apply the visitor and social engagement scores to other relevant dimensions like referrers:

Here you can see that I’ve calculated the variance between visitor and social engagement and am color-coding that against my site referrers. O’Reilly’s XML.com, E-consultancy, and Jim Sterne’s Emetrics web site all are sending visitors who are well-engaged socially.

Finally, you can see the difference between visitor and social engagement applied to the various blog posts I am tracking for Clint Ivy, Ian Houston, Robbin Steif, and Avinash Kaushik. Clint’s open letter to Jeff Jarvis (a controversial piece if ever there was one) is driving a great deal of Web 2.0 engagement amongst Clint’s readers. Nice work, Clint!

Hopefully you get the picture here. By weighting the visitor engagement metric with these social media activities, I am able to easily identify individuals, referring sources, marketing campaigns, rich Internet applications, etc. that are actively interacting, both on my site (join community, engage directly, submit a comment, contribute content) and off (host an event, share a social bookmark).

Wiredset’s proposes a distilled definition of “Engagement = Interaction/Attention” which makes sense to me … you have attention by virtue of their coming to the site, but can you drive interaction? I would propose that the visitor and social engagement metrics I have described in this series of blog posts describes this equation practically applied.

As always, I welcome your comments and criticism.

Analytics Strategy, General

I was recently interviewed by Eric Enge of Search Engine Watch and Stone Temple Consulting

Posted on February 12, 2007April 8, 2017 by Eric Peterson

I got an email last night from Eric Enge who writes for Search Engine Watch. A conversation about web analytics that we had last month was recently posted. Eric asked me a variety of questions about Visual Sciences, making decisions based on data, uniquely identified users, content groups, some of the challenges associated with page tagging, and Avinash Kaushik’s 90/10 rule (which I disagree with due to the rule’s impracticality …)

If you have the time and inclination, give the interview a read, and thanks to Eric Enge for interviewing me.

Adobe Analytics, Analytics Strategy

The gradual building of context …

Posted on February 9, 2007April 10, 2017 by Eric Peterson

Man, it seems like I just cannot get away from Gary Angel lately. He and I are engaged in some kind of crazy mutual-admiration society thing, which would normally worry me, but I know few people as into all this as Gary. He recently posted about “that darn engagement metric” where he expanded on some of my ideas and his response to my ideas and my response to his response to my ideas, etc. One thing Gary said really stuck out in my mind. Regarding my use of the visitor engagement metric to tell a story about the traffic that Marshall Sponder sends to my web site, Gary commented:

“This gradual building of a context around a measure is exactly how I think reporting actually works – and how analysis drives to actionable understanding.”

Exactly! The gradual building of context is what this is all about. The reason I’ve defined a visitor engagement metric is to provide another firm basis for the establishment of said context, another indicator on which we can draw to better understand a dimension or set of dimensions of data we collect.

Much has been written about the value of bloggers to business; it seems like you can’t open Business Week, Fortune, or even Newsweek anymore without having to read about the next big thing that bloggers and blogging are doing to change both business and society. But what can we know about the traffic bloggers send us? And what actions can we take based on that information? Let’s have a look …

Here I’ve rank-ordered the folks I list in my blogroll by percent of sessions they drive back to my web site. No great insights here based on my KPI “percent of sessions” but I suppose if I wanted to I could add Clint, Steve, Tim, and Eric Butler to my holiday card list.

Now I’ve added the session conversion KPI to my list of bloggers. Immediately I see two sets of actions I could possibly take: The first would be to send “much love” to Steve, Mike, Gary, Xavier, Manoj, Aurelie, and Tim for helping me pay for my children’s college education (all book proceeds go to my kids’s Fidelity 529 plans). The second would be to see what I could do to get the rest of the bloggers to say something like “Hey, go buy Peterson’s books, they’re great!”

Still, I should probably check to see first that these folks aren’t referring me traffic that later returns to the site and makes a purchase, right? I need to roll in a visitor-based conversion metric:

Ah ha, now I can see that I owe some serious thanks to Steven Jackson and the folks at the Blackbeak Blog! Better than one in ten people Steve has sent my way have made a book purchase, which is awesome. But it looks like I have a problem with bloggers like Anil Batra, Matt Jacobs, Robbin Steif and even Marshall Sponder. Zero percent visitor-based conversion to purchases on my site … must be some problem with how those folks are talking about me, right?

Oh, or maybe not, at least not in every case. I added a KPI for percent buy path sessions, basically the percentage of sessions in which a visitor at least starts down the book purchase path. Now I can see that Anil, Justin, and Robbin are all doing a pretty good job of getting people into the purchase consideration process, but for some reason those folks aren’t completing the purchase on my site. It’s not their fault, it’s my fault!

Damn.

But hey, maybe it’s still not my fault. Maybe even though the bloggers are sending me traffic that hits the buy path, maybe those folks aren’t really all that engaged with my site and content. Maybe the visitor’s they refer me are just looking at one page in the buy path and leaving, never to return.

Okay, or maybe not. Anil, Matt Jacobs, Marshall, and Clint are all sending me visitors that I consider to be “well engaged” with my site (my site-wide visitor engagement average is 30 percent.) Now I can see two clear action items:

I need to reach out to Anil and see if he and I can work out a deal to help further encourage his readers to completing the book purchase on my site. Anil, if you’re reading this, call me, we need to talk.
I need to reach out folks like Matt, Marshall, and Clint and see if there is some way I can get them to more passionately advocate for my books in their weblogs. Given that their visitors are more highly engaged than the “average visitor”, I have to believe their is an opportunity to sell more books.

But wait, I’m not done. In fact, I’ve only just begun to mine for the true opportunity here. But hopefully you can see, this gradual building of context is well-supported by each of the key performance indicators I keep in my arsenal, both simple metrics like “percent of sessions” and the more complex “visitor engagement”.

This post was a really long way of saying I agree with Gary about no one KPI driving a specific and easily understood action. All of our efforts are ultimately designed to help the online business better mine for opportunity and understand how that opportunity might potentially be leveraged. There are no easy answers, there are no silver bullets, there is no magic, nor mystery, nor puzzles …

There is only the gradual building of context …

Analytics Strategy, Conferences/Community

A few items of note

Posted on February 8, 2007April 8, 2017 by Eric Peterson

[UPDATED: I mis-read the WAW announcement from Rene and Aurelie (see Rene’s comment on this post. I wish them all the best in Brussels!]

Here are a few small updates that I noticed that I thought worthy of passing along:

Bryan and Jeffery Eisenberg at FutureNow have completely redone their web site and have added a very cool RSS feed feature at GrokDotCom.
Jim Novo, one of the “OGs” of web analytics (man, I hope he takes that the right way!) is blogging! I didn’t see any email go out, or any announcement in the Yahoo! group, but sure as sunshine Jim is blogging at blog.jimnovo.com. Welcome to the navel-gazing contest, Jim!
Aurelie and Rene, the fine folks running the show at WebAnalytics.BE in Brussels, have announced that they’re going to bring Web Analytics Wednesday to the capital of Europe (Brussels!)
Speaking of Web Analytics Wednesday, remember that this month’s event is on FEBRUARY 21st to accommodate all you lovebirds out there who will rightly be spending February 14th with your sweetie. So far there are events planned in Quebec, Irving, Texas, and Sommerville, Massachusetts. Sign up and PARTICIPATE!
The fine folks at Digitas have two jobs posted on my premium job board, including a Vice President-level position.

I’m sure there is more but that’s all I have time for right now.

Analytics Strategy, Conferences/Community, General

An interview with one of my personal heros in the web analytics world

Posted on February 7, 2007April 8, 2017 by Eric Peterson

Often times people congratulate me for being smart enough to have started Web Analytics Wednesday, an event that was recently referred to as “the world’s only global social networking event.” While I think it’s great that the idea has caught on, I cannot claim sole ownership of the idea. In fact, the idea was nothing more than a fantasy until one person I had never met put the wheels in motion in her own community and simply made it happen.

That person was June Dershewitz.

When I finally met June for the first time, she turned out to be much quieter than I expected. June had done something nobody really expected people to actually do, she had paid her own way to Emetrics in Santa Barbara, not an insignificant investment. Over coffee one morning she told me she looked at is as an investment in her future. Boy howdy was she right!

June is as close to a web analytics superstar as they come in my book. She has experience, she is eloquent, and she understands how all the pieces come together to paint the bigger picture. She is one of the few people I know available (occasionally) for contract work in San Francisco and I consider her very much to be part of the heart and soul of our web analytics community. (Shhhh, don’t tell her I said that, okay?)

Recently I launched a new premium job board on my site with the folks from Simply Hired (the Job*a*Matic) in part because of the number of inquiries I would get from companies looking to hire bright folks just like June. But it occurred to me, what better way to explain what someone like June is looking for than to ask her directly. Our conversation went as follows:

Eric T. Peterson: June, tell me a little bit about your experience to date doing web analytics?

June Dershewitz: I’ve got 8 years of experience in the field of web analytics. Back in 1999 I took a job as a web analyst for a startup. After that I took a more technical role in data warehousing (still web measurement data). Since 2004 I’ve held a series of contract assignments as a web analytics specialist.

Eric T. Peterson: What kind of contract work do you usually do? More technical stuff or more analysis or more reporting?

June Dershewitz: It depends on what the client needs. Sometimes I get to do system implementation work. On other occasions I focus more on analysis, scorecards, reporting.

Eric T. Peterson: A personal question: what do you really like to do in the web analytics field? Put another way, given all your talents, which excites you the most?

June Dershewitz: I like to help businesses make the best use of web measurement data. I enjoy making improvements to what gets collected and how it gets interpreted and used within the company.

Eric T. Peterson: Can you share, without naming the company, some of the successes you’ve been or been a part of?

June Dershewitz: At one company, I was able to build out their existing web analytics system into something that’s now a useful and trustworthy source of information for a large audience. I felt like it was a big achievement to spread the word and help people see the business value in web activity data.

Eric T. Peterson: Excellent! No simple feat to drive widespread adoption of web analytics through a company.

New line of questions: What do you look for most in a job posting on the Internet (other than contract work and your local geography)?

June Dershewitz: As I scan a list of job postings, I look first at the job title and the company name. Job title usually gives me some clues about responsibilities and compensation. The company (and its industry) interest me as an analyst because I enjoy the prospect of a new and challenging set of data.

If I get that far I’ll read the whole job description and try to get a better sense of what the day-to-day work would actually involve, what tools they’re using (or hope to use), and if my skills match with what they’re looking for.

Eric T. Peterson: When you apply for a job, roughly what percentage of the time are you invited in for an interview? Given eight years of experience I’m thinking “often” but what would you say?

June Dershewitz: These days I post my resume on a couple of job boards and let interested parties contact me; I get a fair number of responses this way. I also monitor job postings and I’ll submit an application if I feel that I’ve found an especially good match. I’ve actually found four gigs in a row through postings where I’ve taken the initiative to apply.

Eric T. Peterson: Would you say it’s a “sellers” market for web analytics talent today? Perhaps more so than past years?

June Dershewitz: Definitely! As a job-seeker, it’s great to see so many open positions out there.

Eric T. Peterson: When you’re in the interview process, what impresses you most about a company vis-a-vis web analytics?

I really appreciate enthusiasm for the subject matter. It really grabs my attention when the prospective employer says, “We’ve got bucket-loads of web data and we could be making such better use of it! We could really use someone with your skills to help us figure out what’s interesting and what’s important and what we should be using to drive business decisions.”

To me that means there’s a challenge to be had.

Eric T. Peterson: What, in your opinion, are the best things someone relatively new to the field can do to prepare for a career in web analytics?

June Dershewitz: If the analyst track is your objective, it’s important to convince your prospective employer that you’d make a great analyst. Also, I’d recommend that you keep up with current issues in the field, read books, read blogs, talk to other analysts.

Show that you’ve got the curiosity it takes to enjoy this line of work

Eric T. Peterson: Okay, last few questions: In major markets, what do you think experienced web analytics professionals are worth from a salary perspective? I know you do more contract work but I wanted to see if you could/would share ballpark numbers that you’re hearing from folks?

June Dershewitz: I sometimes reference recruiting agency job postings where they’ve got salaries listed.

Eric T. Peterson: Cool. So are you looking for contract work now? If so, how can my readers get in touch with you and what should they know in advance?

June Dershewitz: I’m booked solid through the end of April, but I may be up for new opportunities after that. I also welcome communication from other veteran web analysts who’ve chosen to take the contract route. Find me on LinkedIn (http://www.linkedin.com/in/jdersh), send me an email (june.dershewitz@gmail.com), or say hi at Emetrics in San Francisco.

Eric T. Peterson: June, you’re awesome! Thanks for spending some time with me today. I’d wish you luck out there but knowing you, you don’t need luck.

Thanks again!

June Dershewitz: And thank you, Eric! I appreciate that job board you’ve put together.

It’s great to know that someone with June’s skills is reading my job board. In fact, she bugged me to set up an RSS feed for job postings which I quickly put together, just for June. If you’re looking for great jobs in web analytics, you might be interested in the feed as well:

http://www.analyticsdemystified.com/job_feed.asp

Analytics Strategy, General

Etc.

Posted on February 5, 2007April 8, 2017 by Eric Peterson

Some interesting things I saw in the blogosphere the last few days:

Robbin Steif interviewed me on behalf of the American Marketing Association for whom I’m doing a free webinar on web analytics in early March. Robbin always asks really good questions but sometimes my answers get lost in translation.
Eric Enge has a really good interview with Jim Sterne. Jim is such a great leader of our industry and Eric Enge is really knocking ’em down interviewing Brett Crosby, Dennis Mortenson (IndexTools), and Jim.
Ian Thomas is busy telling us what Microsoft Gatineau will be without actually saying the words. Expect to see something outcome focused that is easy to use that provides people in a May/June timeline. Or, if you’re anxious to figure Gatineau out now, just grab the JavaScript code out of Ian’s blog and have a look-see.
My very bright friend Ian Houston is starting a series breaking down the web analytics data model by focusing on the canonical data unit, the “event”. WARNING: Eat your Wheaties before you read Ian Houston. Ian makes perfectly logical statements like “refining the definition of Events to the concept of an Event Super Class where the sub-dimensions by type are not children of the Events dimensions but rather sub-classes of the Events class that inherit their properties and relations within the data model from the Super Class.”
There are new jobs posted in my premium job board from PayPal, Stratigent, Cox Newspapers, Staples, and the World Wrestling Entertainment group.

Congrats to the Colts on yesterday’s Super Bowl win. I’m from outside of Chicago so my loyalties lie elsewhere but it’s nice to not see the freaking Patriots in the big game for a change.

Analytics Strategy, General, Reporting

The myth of actionability

Posted on February 4, 2007July 10, 2017 by Eric Peterson

A few weeks back, Gary Angel from SEMphonic published an oddly-titled post called “Why 100% Conversion is a Very Bad Thing” in which he calls into question the whole notion that a key performance indicator (KPI) is only good if a change in the indicator suggests a specific action that can be taken. Gary calls this kind of thinking “the myth of actionability” and says:

“The myth of actionability is conventional wisdom in web analytics – and it suggests that you shouldn’t report on anything unless changes in the measured value can be directly addressed by specific actions. In other words, if you can’t answer the question “What would I do if the value changed up/down?” then you shouldn’t report on the measure.This criteria is designed to eliminate lots of useless data from report sets and insure that what is in report sets has substantive value.

Unfortunately, I believe the criteria of actionability is unsound in almost every way: being both wrong-headed about the purpose of reporting and impossible to actually satisfy in the real-world.”

Obviously Gary is not one to pull punches. Unsound, wrong-headed, impossible … yowch!

Gary calls the myth of actionability “conventional wisdom” and I absolutely agree with him. Everywhere you go, when people are working out key performance indicators and building dashboards, the basis for inclusion or exclusion is usually “is there some action that a change in this metric will encourage us to take?”

Where does this kind of thinking arise? Well, let’s look at page 10 of The Big Book of Key Performance Indicators by Eric T. Peterson. In the section titled “What is a a Key Performance Indicator?” under the subsection on “Action”, in 2006 I explicitly stated:

“Key performance indicators should either drive action or provide a warm, comforting feeling to the reader; they should never be met with a blank stare. Ask yourself “If this number improves by 10 percent who should I congratulate?” and “If this number declines by 10 percent who should I scream at?” If you don’t have a good answer for both questions, likely the metric is interesting but not a key performance indicator.There is enough data in the world already. What most people need is data that helps them make decisions. If you’re only providing raw data, you’re part of the problem. If you’re providing clearly actionable data, you’re part of the solution. If you discover you’re already doing the latter (being part of the solution), give yourself a hug.”

Hmmm, it seems like I am one of the sources of “the myth of actionability” Gary is railing against. But it gets worse; while I was at JupiterResearch I published and presented a number of times on the subject of key performance indicators, and every time I talked about the subject, I stated unequivocally that the “core” of a good key performance indicator was it’s ability to drive action.

Remember, Gary said “Unsound, wrong-headed, impossible …”

Now, I don’t feel the need to defend myself, not because I disagree with Gary, but rather because I think Gary (and perhaps other folks) have taken the interpretation of “needs to drive action” to an unreasonable extreme. Let’s quickly have a look at the history of Eric T. Peterson’s guidance on key performance indicators:

In 2004 in my first book Analytics Demystified, I wrote in Chapter 15: Bringing it All Together Using Key Performance Indicators that “the most common complaint about Web analytics data and the applications that provide said data is that there is simply “too much information”; too many graphs, too many charts, too many options, too many variables—too much for the average user to understand and make use of.“
In 2005, in my second book, Web Site Measurement Hacks, I wrote in Hack #94: Use Key Performance Indicators that “the best KPIs are those that, when people look at them and realize that they’ve gone down from week to week, make people freak out and call meetings.” I also said, relevant to Gary’s complaint regarding the establishment of which indicators to use, “if you’re thinking about a number but cannot think of any action you would take if that number absolutely tanks, set that number aside.“
In 2006, in my third book, The Big Book of Key Performance Indicators, I wrote “key performance indicators should either drive action or provide a warm, comforting feeling to the reader; they should never be met with a blank stare. Ask yourself “If this number improves by 10 percent who should I congratulate?” and “If this number declines by 10 percent who should I scream at?” If you don’t have a good answer for both questions, likely the metric is interesting but not a key performance indicator.“

If I’m wrong, at least I’m consistent huh?

When I first started pushing the idea that indicators needed to be tied to some type of reasonable action, my statements were a direct response to the dominant paradigm at the time: that all the information you needed to run your online business was contained in the hundreds of reports all web analytics applications generate, all you need to do is find the right data and take the appropriate action.

The problem I saw with this was, well, almost nobody was being successful with this strategy. Not only were most companies hamstrung and suffering from data overload leading to analysis paralysis, senior managers were asking for relevant data from the web analytics systems but not getting particularly satisfying responses from the people running the systems. Relatively boring metrics like “page views” and “visits” were being pushed up the food-chain, but except in rare cases, an increasing number of page views and visits were only loosely tied to increasing business success.

And so I proposed an Occam’s Razor for web analytics reporting, one that mandated that companies actually carefully consider the metrics bound for widespread distribution, and choose those metrics based on their ability to generate some action.

I never said, and I’m not sure anyone really says, the “actions” that would be taken were as granular and spuriously precise as “if this metric declines, reduce your PPC spending by 10% per 3% point decline observed.” Web analytics just doesn’t work that way folks, and here I agree with Gary when he writes:

No single measurement can ever suggest an action – cannot, in fact, even be interpreted directionally as either good or bad. Only in the context of a complete view of the business system (and the knowledge that all other things are equal or heading in some specific direction) can a judgement be made about the meaning of single measure. I think this make it clear that no one measure can ever really be “actionable” when taken in isolation. And if no one measure is actionable, then surely the criteria of actionability is fruitless.”

So let me clarify my position, as I am perhaps the high priest of the “cult of actionability”:

At design time, key performance indicators should be included or excluded from a hierarchical reporting strategy as outlined in The Big Book of Key Performance Indicators based on the likelihood that the indicator will spur some type of action in the organization when the indicator unexpectedly changes.
The action the organization would take, when unexpected change occurs, is never precise. The action is nearly always “conduct additional analysis” at which time the indicator’s definition provides at least the nominal basis for the starting point of the analysis.

At the end of the day, my view on key performance indicators is that they are intended to promote the visibility of web analytics throughout the organization, especially to the upper echelons where it is increasingly unlikely that traditional web analytics reports will be given the attention they deserve.

By creating a reasonable set of metrics and indicators, derived directly from the site’s business objectives and supporting click-stream activities, and then delivering said metrics throughout the organization with serious thought to definition, presentation, and potential for action, companies have been shown to significantly improve the level of attention given to web analytics data.

All of this helps to directly combat Gary’s observation that he too is “often disappointed in the report sets [SEMphonic] generate[s].” At the end of the day, regardless of which side of the fence you’re on, I believe we all agree that the central goal of web analytics is to help the business make better decisions. We do this by continually refining the web analytics business process and striving to better educate decision makers about the actions they can take to improve the web site. We repeat as necessary and hopefully go to bed happy.

Anyway, I’m a huge fan of Gary Angel so I hope we can continue this debate. What do you think? Am I crazy? Is Gary crazy? Or, like so much in our industry, is the reality something between the lines?

Analytics Strategy, General

Have you read Lee Sherman's piece on Actionable Analytics?

Posted on February 3, 2007April 8, 2017 by Eric Peterson

I’ve never met Lee Sherman, Senior Vice President and General Manager for Global Solutions at AvenueA/Razorfish but I had a chance to chat with him recently about the AvenueA document “Actionable Analytics” that the company recently began distributing. Gary Angel’s eloquent rant on the “myth of actionability” aside, I found that I agreed with much of what Lee had to say in his document.

I say “much” but not all, especially not his comments about web analytics applications not being able to track advertising from end-to-end, nor his comments about web analytics applications not being good at tracking Rich Internet Applications. To his credit, Lee admitted that he had not had a very good look at Visual Site and Platform 4, but Seattle is only a short drive from Portland so hopefully I can fix that before long.

Some of the things that really interested me in the document were Lee’s discussion of RIAs. We talked about his assertion that “rich internet applications are the best vehicle to engage users online and build brand loyalty,” an assertion that was made in a PPT that AvenueA uses to support the document. I was really interested in this statement, especially the use of the word “best” since I’ve never seen data to confirm a broad statement like that, and also the use of the word “engage” since I’m really into figuring out how companies will measure engagement online.

Lee admitted that “best” was perhaps too assertive of a term but that anecdotally, in AvenueA tests, in almost every case the RIA version of a process outperforms the HTML version of the process. When I asked about the performance criteria, Lee said they were looking for differences in subsequent perception of brand, intent to purchase, and actual transactions. Clearly, Lee is from Avinash’s school of the trinity and believes in asking visitors directly to augment observed behavioral data.

We then talked about how RIA’s should be measured, a conversation I hoped to promote in the Web 2.0 Measurement Working Group. Lee said he felt that that are no standards for RIA measurement yet and that most companies he works with are more worried about what they can measure than what they should measure. He believes we’re more likely to see “custom” measures, perhaps supplemented by online surveys designed to gather attitudinal data after visitors interact with RIAs.

Not a bad idea, really, but I sure would feel better if we could just agree on a handful of easily understood measurements for success to apply to RIAs. Wouldn’t you?

Regarding Lee’s definition of “engagement”, he cited a few measurements that I haven’t really heard from anyone other than AvenueA: Page IQ and scrolling analysis. The former is basically the percentage of visitors interacting with a page at all (i.e., not closing the browser, not clicking the back button) and the latter is an assessment of visitor’s scrolling down a page. Both measures are clearly page-centric, not that that is bad, but neither seem to easily lend themselves to the concept of visitor engagement.

All-in-all the report is worth a read if you haven’t already seen it. You have to send Avenue A some personal information to get the download but hey, few good things in life are truly free. You can request the download here.

Thanks to Lee for taking the time to chat with me and to David Deal for setting up the call.

Analytics Strategy

Like hitting golf balls in the fog

Posted on January 31, 2007 by Eric Peterson

Via Juice Analytics, I was up early this morning to catch up on my reading and I found myself flipping through Matthew May’s Change This presentation on elegant solutions. The slide deck is a great read and I’m definitely going to check out The Elegant Solution: Toyota’s Formula for Mastering Innovation. One thing really stood out for me as particularly relevant to the plight of companies working to be successful with web analytics.

“Make Kaizen Mandatory” is his key practice driving innovation #9. May says:

Kaizen has three steps: First, create a standard. Second, follow it. Third, find a better way. Repeat endlessly. Trying to improve and innovate without a standard as reference is like a journey with no starting point. It’s like hitting golf balls in the fog.

Over the years many companies have asked me “How can we be more successful with web analytics?” to which I invariably respond “Determine where you are today, establish goals for improvement, and iterate until you achieve your goals.” Generating and distributing the “right” reports is the key–creating a common understanding of where the business is today, and where the business needs to go tomorrow.

Sounds reasonable, doesn’t it? Innovation through iteration. Don’t “swing for the fences” or get too clever if you can avoid it; rather, implement a functional process that allows you to move from point “A” to point “B” systematically. Often times folks are surprised that the simple solutions are also the right solutions.

I don’t know golf, but hitting golf balls in the fog sounds like a waste of money. Check out Mr. May’s ChangeThis presentation.

Analytics Strategy, General

My interview with Megan Burns of Forrester Research

Posted on January 30, 2007April 8, 2017 by Eric Peterson

I recently had the pleasure of chatting with Megan Burns at Forrester Research. Megan covers, among other things, web analytics. Despite her being with Forrester for just about a year now, she is one of the leading analysts thinking about how companies actually deploy and use web analytics technology.

Megan hit the ground running, filling some pretty big shoes, and has published nearly a half-dozen reports directly relevant to the web analytics market. I don’t normally interview people in my weblog but when Forrester analysts talk, people listen. The transcript of our conversation follows:

Eric T. Peterson: First question: You’ve been in the job at Forrester Research for just over a year now. What would you say the biggest thing you’ve learned about the web analytics industry is so far?

Megan Burns: There are probably two things that stand out. First, the fact that measuring a Web sites is tougher than many people think.

There’s so much data to chose from, and all of it’s imperfect. Plus there are multiple ways to solve most measurement problems. Deciding which approach to take isn’t always straightforward and for many people measurement is just one part of their job, so even though they’d like to dedicate time to thinking about the best way to leverage all the data they could be collecting, they have to make some tough priority calls.

But the second thing I’ve learned is that many people believe very passionately in the power of data, and they’re committed to figuring out the tough problems.

Eric T. Peterson: So on one side of the coin you have the complexity of measurement as a function of expertise, approach, and time, and on the other side of the coin a strong desire to make it work.

Megan Burns: Absolutely.

Eric T. Peterson: So the folks you talk to who are being successful with web analytics, is there something that sets them apart? Something quantifiable?

Megan Burns: I’m not sure if it’s quantifiable, but there’s an understanding that metrics are a means to an end not an end unto themselves. They constantly think about what they’re trying to do, and how data can act as a tool to help them do it.

Eric T. Peterson: In your experience, does success with analytics improve with company size or does it appear to be tied to motivation?

Megan Burns: I haven’t seen a correlation to company size, but I haven’t looked at that relationship specifically.

I don’t think company size is a factor, though. I think it has more to do with attitude and approach to the problem. Often it’s about teaching people in the company about data and what it can do for them so that they change the way they make decisions. Any time you’re dealing with people and change it takes a good dose of both patience and time.

Eric T. Peterson: You and I have talked in the past about the importance of “process” to web analytics. You commented once that “people think process is a four letter word” which made me laugh and wince at the same time.

First, can you describe your position on the need for process in web analytics? And second, do you have any advice to help companies get past their fear of the “P” word?

Megan Burns: Sure. I think process is important in any discipline as a way to help people make sure the right things get done by the right people at the right time. We’re all trying to do so much these days, it helps to have a process that reminds us what needs to get done. It also sets clear lines within the organization as to what each person or group is responsible for and who they are dependent on. Web analytics is no different.

People who design sites need to understand that others in the business have to be able to measure the impact and success of those sites. They need to factor measurement requirements in to the process. But they have many other people asking them to build in other requirements, so it helps to have a checklist to make sure you’ve thought about all the different types of requirements you need to capture before you build something. That checklist is part of the process.

But it’s important to remember that the “Web analytics process” is really a sub-process of the larger eBusiness process. That data is needed by certain people in the firm at certain points in their decision making cycle. If the two aren’t integrated properly, things break down. People don’t get the data they need when they need it.

Changing people’s perceptions of process can be tough, depending on their experience with it. But I think the most important thing to remember is that process != bureaucracy.

When the only thing people are trying to do is check off boxes on the process so they’re “in compliance”, you’ve totally missed the point. It needs to be detailed enough that it’s useful and insures key steps don’t get missed, but it shouldn’t impose unnecessary restrictions or red tape. That’s a very fine line — one that’s not easy to get right.

Eric T. Peterson: So tell me the truth and don’t hold back … in my presentation at Emetrics where I advised our community to go so far as to draw business process diagrams for how web analytics integrates into the bigger picture … good idea or a superfulous waste of time?

Megan Burns: Somewhere in between. I think people responsible for analytics should start by looking at the larger site design/interactive marketing process (which probably isn’t written down anywhere, by the way ) and see where and when the data needs arise. Then look at where they need to be involved (i.e. requirments, development) in order to get what they need to meet those requirements. To me, measurement is a section in the business and technical requirements documents that must always be filled out. Even if it just says “No Impact” or “No new requirements”. But at least that way you know someone thought about what new data might be needed, or what code might have to change to maintain existing measurement.

Metrics need to be considered in the project planning process, like any other feature of the site. How long will it take to define requirements for this? To implement and test them? The process is no different, but it’s not something customers use so often it gets missed.

Or skipped intentionally to save time. But then it takes twice as long to add in after the fact, so you didn’t really save any time.

Eric T. Peterson: Excellent points, all.

You have a background in software development process, don’t you?

Megan Burns: Yes, I do.

Eric T. Peterson: Okay, new direction here: You’ve written a ton on web analytics since joining Forrester Research. I especially enjoyed your work on the ROI of dedicated headcount for analysis. What was the overall response to that report?

Megan Burns: The response was extremely positive. So many people I talk to tell me that resources are their biggest obstacle to using and interpreting the Web analytics data they’re collecting. The report helped them explain to senior management what analysts do and how they add value to the organization.

In a quantitative way, that is.

Eric T. Peterson: When companies ask you where to find experienced web analytics talent, what kind of advice do you give?

Megan Burns: That’s a tough question, because experienced Web analytics talent is so hard to find these days. My advice is usually to engage professional services consultants from either their vendor or an independent consulting firm to act as mentors for existing staff.

If they really want to hire, I suggest networking, networking, networking.

This is such an active community with the Yahoo group, blogs, Web Analytics Wednesdays … there are plenty of ways to meet others who might be able to lead them to a qualified candidate.

Eric T. Peterson: We’re just about out of time and I want to thank you for being so generous in allowing me to interview you.

Megan Burns: Quite welcome. Glad we could finally arrange it.

Eric T. Peterson: Last few questions … what book or books are currently on your nightstand?

Megan Burns: There are so many books … Information Dashboard Design, by Stephen Few is one.

Eric T. Peterson: What music would we find on your iPod?

Megan Burns: My iPod’s full of all sorts of music. Everything from rock to oldies to show tunes.

Eric T. Peterson: Who are some of your favorite bloggers?

Megan Burns: I wish I had time to read as many blogs as I’d like to.

I try to keep up regularly with Charlene Li and the other Forrester blogs, you, Avinash, and Om Malik. A few others, too. It also depends on what I’m working on.

Eric T. Peterson: Megan thanks very much for taking the time to chat with me today. I hope to see you in San Francisco in May and look forward to your upcoming research on Rich Internet Applications.

Megan Burns: You’re quite welcome. Take care, and I’ll see you in CA (if not sooner).

Adobe Analytics, Analytics Strategy, Reporting

A sample of how my visitor engagement index drives insights

Posted on January 28, 2007April 8, 2017 by Eric Peterson

While I have not had time to write Part V of my series on measuring visitor engagement, I wanted to take a few minutes to address some comments folks have made about the metric recently. It’s very encouraging to see folks like Gary Angel and Daniel Markus pushing the conversation about measuring engagement along as I can think of few more qualified to critique this work.

Gary Angel, who had very nice things to say about the metric, commented on how in some areas the metric is biased, specifically towards search engines and specific types of content. Gary is concerned that the Brand Index will unfairly bias towards search engines (given that one component is searches for brand-specific terms like “eric t. peterson” and “web analytics demystified”.) I examined this effect and it turns out that “branded searches” make up only a small part of the index for my site but Gary makes an excellent point, unnecessary bias should be removed from the index whenever possible. As such, in my current calculation I have removed this weighting from the Brand Index, redefining said index to only be direct sessions (non-search, non-referred.)

Score one for Gary.

Gary also commented that:

“… if I’m using my metric to measure the “engagement” produced by visitors who used a specific part of a site (like the blog or the press releases), it’s vitally important that my metric not include a strong built in bias toward one of the areas (like blogging). Some analysts might argue that this represents a flaw in the metric Eric proposes. I don’t think so. Every metric carries with it some biases – and no metric is appropriate to every situation.”

This is a good point, one that had been made by a handful of other folks who critiqued the metric early on. The problem I have with removing the Blog Index (ratio of blog reading sessions to all sessions) is the evidence that my weblog is a prime driver of engagement with my site and overall web analytics brand: Over the last 12 months, weblog subscribers are nearly 400 percent more likely to have returned to the site recently than non-readers; those visitors not subscribed to my blog (e.g., in Bloglines or Google Reader) but who are still reading blog content are 300 percent more likely to have returned recently.

Score one for Eric.

One thing worth noting, the way I am using Visual Site to measure weblog readership and subscription, this activity does not show up as traditional “page views” unless the reader A) reads the post on my web site or B) clicks through to the web site (at which time the post appears as a session “referrer”) — Visual Site is able to track external RSS and XML-based content using a non-page view event (something I call “reads”.) Not all web analytics systems afford their operators this flexibility so I thought it would be worth bringing up. This is part of the reason that the Blog Index needs to be a separate index, not part of the Click Depth Index as some have questioned.

But enough about Gary … Daniel Markus posted what I surmise to be a nice post about my visitor engagement metric at Marketing Facts late last week in which he called my calculation “the mother of all Web Analytics KPIs.” The post is entirely in Dutch and my Dutch is horrible so I wrote to Daniel and asked for a rough translation . While there were many good comments about the metric, they raised two concerns:

The calculation is complicated and difficult to understand.
There was some question of the utility of this metric, essentially calling into question the overall “actionability” (not a word) of visitor engagement.

Regarding the complexity of the calculation, as Gary has so eloquently stated any number of times, no indicator or metric is any use without understanding its components, its definition, and its inherent biases. Clearly the onus is on the web analyst to explain the metric and it’s definition to any audience they present engagement data, especially given the complete lack of formality around measuring “engagement” (at least until you started reading my posts on the subject.)

Given the complexity of the calculation, the latter concern is valid but one that misses the point of the metric. There are any number of loose definitions of “engagement” floating around in our community — duration, page views, average page views per session, sessions per visitor, etc. But none of these more easily understood (note: not easily interpreted) metrics, in my mind, captures the essence of an engaged visitor.

Visitor engagement has to be examined over diverse criteria, simple assessments simply do not work. To wit:

To say that session duration is a good measure of engagement is fine, unless the visitor never returns to the site.
To say that a high number of page views is a good measure of engagement is fine, unless the visitor runs up those page views in a very short period of time and was unlikely able to actually read content.
To say that recency of visit is a good measure of engagement is fine, unless the visitor has only looked at your home page and left.
To say that direct visits are a good measure of engagement is fine, unless those direct visits lead to short sessions of few pages viewed and the visitors never return.

I believe that the complexity of the calculation is where visitor engagement derives its value. For practitioners who are lucky enough to have access to a platform that can actually make this calculation and who are willing to take the time to explain to their audience what the metric measures and what its limitations and biases are, the metric can yield insights that would be unlikely to fall out of “traditional” web analytics.

I will leave you with an example of how I am deriving small insights from my measurement of visitor engagement.

Marshall Sponder is the WebMetricsGuru blogger and all-in-all a pretty nice guy. He and I had a little tiff awhile back over Avinash’s web analytics blogger index (something Avinash has stopped doing for some reason …) when I was less than complimentary about the volume of web analytics posts that he produced relative to his blogging in general. Examining traffic metrics from Marshall’s blog I would interpret the value of having a good relationship with him based on a set of commonly understood data:

Almost no volume and no books sold. Come on Marshall, let’s see a nice recommendation for Analytics Demystified already! 😉

But wait, what if I have a closer look at the measured engagement of the visitors he’s been sending to my site:

While my “average” visitor to the site is only 24.2 percent engaged, visitors from Marshall’s posts are nearly 40 percent engaged with my site and, more importantly, of these visitors almost 10 percent are “highly engaged” (50 percent engagement or better.)

Marshall may not be selling books yet, but I have the nagging feeling if he tried even just a little, he could probably drive pretty good numbers given the engagement of the audience he referrers.

Now just imagine that you were running a million or billion dollar business, looking for new opportunities on the Internet. You have hundreds-if-not-thousands of sites sending you visitor traffic all day, every day. Maybe some of these people make purchases, but maybe you have nothing for them to purchase … how do you decide who to spend more time with and who to ignore?

Me, I’m going to write nice things about Marshall Sponder and if the folks from e-consultancy call me and want to do another interview, I’m taking that call right away! How’s that for a KPI defining an action?

Adobe Analytics, Analytics Strategy, General

Gary Angel breaks down my engagement metric

Posted on January 21, 2007April 8, 2017 by Eric Peterson

I wish I had more time right now to address all the valuable points that Gary Angel of SEMphonic makes in his critique of my visitor engagement metric but I’m heading out early in the AM tomorrow. Suffice to say, Gary took the time to really drill-down into my work on measuring engagement and his analysis is great. Personally, when someone of Gary’s caliber says, “… while I’d quibble with one or two of his choices, I think it’s one of the best metrics for this that I’ve ever seen in web analytics” I get a big smile on my face.

Check out Gary’s great post on “Eric Peterson’s Engagement Metric“, and be sure to read it in the context of his last post on “the myth of actionability” … something sure to raise some hackles (mine are up, but I have not had a chance to respond!)

Adobe Analytics, Analytics Strategy, General

What a busy week in the web analytics world!

Posted on January 18, 2007April 8, 2017 by Eric Peterson

Wow, once again I manage to fall woefully behind and am forced to play catch-up. Likely you’ve seen all of this but just in case:

Mike Keyes at On The Trail references my engagement metric but wonders if the all-mighty dollar is not a better measure of web site quality. He wonders aloud about the engagement profile for people who submit leads to the site, something I can easily measure but will have to follow-up on later. I don’t think Mike’s idea is all that goofy!
Daniel Riveong at Emergence Media describes my engagement metric as “pretty raw” but wisely points out measuring “buzz” is only a component of measuring engagement and that external data needs to be incorporated into site-based measurement. Sounds like Daniel is connected in the Bay Area so perhaps I’ll get invited to the next Social Media roundtable to talk about my metric.
I am on the Online Marketing Blog’s list of blogs found useful to TopRank Online Marketing, along with many of the other bloggers I regularly read. Thanks!
Robbin Steif at LunaMetrics blogs a conversation she had with Bob Chatham of Visual Sciences. It’s a little dated at this point but still a good read IMHO.
There is an interesting “tags vs. logs” debate happening in the Yahoo! group. No great surprise that people have pretty strong opinions on either side of this debate. Me, I say “why choose? Why not have both if that’s what you need?!”
Finally, two of my favorite people have finally bust forth into the blogosphere, Ian Houston of Visioactive and Dylan Lewis of Intuit. Both Ian and Dylan contributed to my O’Reilly book, Web Site Measurement Hacks and are among the brightest people I know in the web analytics arena.

Since I lined all these links up two days ago I’m sure I’m behind again but you do what you can. Think I missed something big? Send it to me in your comments!

Adobe Analytics, Analytics Strategy, Conferences/Community, General

A question I get with some frequency these days …

Posted on January 15, 2007April 8, 2017 by Eric Peterson

As I have been doing more and more lately to help folks find and fill web analytics positions it seems like more emails like the following come in:

“I have just been promoted into a Marketing position and one of my primary responsibilities is web analytics. I do not have any formal education or experience with web analytics and I am expected to educate myself by any means necessary. After doing some research online, the first thing that I did was order your book, Analytics Demystified. I also picked up a copy of Waiting for Your Cat to Bark by Bryan and Jeffrey Eisenberg.What kind of advice would you offer to help me outline a practical education process? I know reading, as much as I can, will be expected and I have already begun the process. What about web marketing conferences? What about online courses like UBC Award of Achievement in Web Analytics? Would my company receive a greater ROI by having me take online classes or sending me to conferences? My gut tells me, to buy more books and take online classes, before I start worrying about going to conferences.”

The advice I would offer to anyone in this position is to do the following:

Read the rest of my books, especially Web Site Measurement Hacks and The Big Book of Key Performance Indicators. The former is an excellent overview covering the breadth of things that web analytics professionals are tasked with doing and the latter is a deep-dive into the use of KPIs to help drive awareness of web data throughout the organization.
Take the UBC Web Analytics Award of Achievement classes, offered via the Web Analytics Association. All of the feedback I hear from people who have taken the classes is excellent. The folks they have teaching the courses are world-class and the content that I’ve seen is both fresh and well-written.
Join the Web Analytics Forum at Yahoo! Groups. While the conversation has a tendency to drift away towards the banal at times, the group is still the single largest and most active web analytics conversation on the planet.
Read some of the web analytics weblogs to keep up on current happenings. I personally like mine (go figure), but I also highly recommend Avinash Kaushik and Gary Angel.
By all means, go to the Emetrics Summit. Jim Sterne’s event is the single best place to meet the brightest minds, hear the best presentations, and meet the nicest people in the entire industry. There are upcoming events in London (March), Germany (April) and San Francisco (May).
Ask your vendor what classes, documentation, presentations, etc. they have that might help you better learn how the technology is best used to create value. You’d be surprised at how much helpful information the vendors have when you ask.

If you think I’m forgetting anything I’d love to hear your comments.

Adobe Analytics, Analytics Strategy, Reporting

The engagement metric, defined (part IV in a series)

Posted on January 10, 2007April 8, 2017 by Eric Peterson

For those of you keeping track at home, this is the fourth in what will likely be a five-part series on calculating an “engagement metric”. The first three posts are here:

Part I
Part II
Part III

Originally I had postulated that an engaged visitor, at least on my web site, can be characterized as follows:

The visitor views “critical” content on the web site
The visitor has returned to the web site recently
The visitor returns directly to the web site some of the time
Some high percentage of the visitor’s sessions are “long” sessions
If available, the visitor is subscribed to at least one available site feed

Basically, the final calculation, one revised thanks to the valuable feedback of dozens of folks, is essentially the same with a few slight modifications. The final goals for my site, goals easily tweaked for any site, are as follows.

Well-engaged visitors will:

View a relatively large number of page views in a given session
Have visited the site in the last four weeks
Have relatively long sessions
Come directly to my site or come from a “Eric Peterson” branded search
Be reading my weblog in addition to non-blog content
Buy one or more of my books through my web site

As you can hopefully see, the first item in my original list (view “critical” content) has been softened somewhat. While the act of purchasing is necessitated by viewing critical content (my “thank you” page) ultimately I agreed with several reader comments that the a priori definition of visitor goals would skew the metric and reduce the metric’s ability to tell me about all of the content on my web site. Thanks to Victor and others for hammering this home.

Given all this, the visitor engagement metric is composed of six sub-metrics, each of which can be examined individually to provide context to the larger calculation. The six sub-metrics are:

Click-Depth Index: Percent of visitor sessions of “n” or more pages
Recency Index: Percent of visitor sessions occurring in the last “small n” weeks
Duration Index: Percent of visitor sessions of “n” or more minutes
Brand Index: Percent of visitor sessions originating directly or originating from search engine searches for terms like “eric t. peterson” and “web analytics demystified”, etc.
Blog Index: Ratio of blog reading sessions to all sessions
Conversion Index: In this case, session- or order-based conversion

Keep in mind, engagement is a visitor-based calculation, one designed to look at the lifetime of visitor sessions to the web site. So that the engagement of any visitor is a function of their lifetime of visits. Yeah, this assumes some stability in cookies so always use first-party cookies.

The final calculation is simply a summation of the component indices divided by the total number of components which yields a simple percentage:

If you’re looking across multiple visitors, you would read this as “the average visitor is just under 27 percent engaged, as defined by X, Y, and Z.” If you’re looking at a single visitor you can break engagement down on a session-by-session basis, watching for increases and decreases in the visitor’s engagement over time. In aggregate, visitor engagement becomes a very powerful but elegant key performance indicator that tells you a great deal about the make-up of your audience.

Once you decide that you need more information about the basis for an increase or decrease in visitor engagement, and assuming you have the right technology powering your analysis, you would simply visualize each of the core components over time:

As Clint commented in my last post, there is a surprising stability in each of the components, which is in my mind what you’re looking for. I want to see the variation show up when I examine engagement against my business-critical dimensions (referrer, campaign, page, search term, etc.)

When you analyze the visitor engagement calculation against all of your site visitors, you’re looking for a more-or-less normal distribution. This distribution is spiky because of the calculation, but if you’re able to drill-down, you should see something like this:

(The bars that exceed the visualization’s scale represent peaks that occur as visitors achieve 100% of sessions for 1, 2, 3, 4, and 5 of the engagement calculation’s core components. If you want to see this image at 100% scale let me know …)

Another way I can think about this is to use a scatter-plot, basically showing the same thing but easier to visualize differences as you drill-down into specific dimensions:

All of these calculations actually become relevant when you actually apply them to a dimension of data. Here, for example, is visitor engagement mapped to blog posts from my and Avinash Kaushik’s weblog:

Pretty cool, huh? I mean, it’s no great surprise that Avinash’s 2007 Web Analytics Predictions post has the highest visitor engagement score in this image when you think about all of the follow-up predictions his original post spawned. But boy-howdy, isn’t it nice to see that in a metric that you can understand and actually use?!

Here is the visitor engagement metric applied to some of the referrers to my web site:

No great surprise again that Feedburner, Technorati, and WordPress are driving visitor engagement given that they are likely to be driving visitors maxing out their blog index score. But what about the folks at ROI Revolution, sending me visitors who are on average over 30% engaged, or Blackbeak and the folks at Conversion Chronicles, sending me visitors who are as engaged as my 27 percent site-wide average?

Arrrrrrr, indeed!

Finally, and I know that I showed this already but I just think it’s damn sexy, I can map visitor engagement against any geographic dimension in my system (continent, country, city, state, zip code, DMA, etc.) to see where I might want to focus my local marketing efforts in the future:

You two people in Midland, Michigan, get ready for an onslaught of Analytics Demystified promotions!

Oh, some random notes:

You can add non-page view events (RIAs, AJAX, Flash, etc.) into the calculation easily. I don’t have much of that on my site but I have an “Event Index” calculation that can be added for sites heavily leveraging these types of applications.
You can add the “social media index” that I discussed in my last post just as easily as you can add content-based indices for retail, customer support, business-to-business, or content.
You can take or leave my idea of scoring the brand index against specific search terms. Visual Site gives me a really easy way to do that and I believe that engagement is very much a function of brand awareness, something notoriously difficult to measure in any practical way.
Visual Sciences customers interested in deploying the visitor engagement metric should contact me directly via normal company channels. I have pretty much everything you need to get up and running with this in a ZIP file and I’d be happy to talk you through the process.

As always, I welcome your comments and feedback on the engagement calculation and anything else that comes to mind. In the next (and perhaps final) installment I will cover Clint’s inevitable complaint of “What the heck happened to all the great ‘social media’ stuff?!?” as well as talk about some specific applications of the engagement metric.

Analytics Strategy, General

It's not even the end of January and I'm 1 for 5 on my predictions!

Posted on January 8, 2007April 8, 2017 by Eric Peterson

I got word a few days back that W. Greg Dowling, my replacement at JupiterResearch covering among other things web analytics, has left JupiterResearch for Modem Media. Greg started last week as the Vice President/Associate Director of Digital Marketing Analytics. According to Greg, his new job will have him:

“[Focusing] on developing Modem’s capabilities in Web analytics, growing Web analytics consulting revenue, and building cross-capability integration of Web analytics into all Modem deliverables.”

While I am quite bummed out to hear that Greg has left JupiterResearch, after talking to him for awhile I’m excited for him in his new opportunity. Best of luck, Greg!

Analytics Strategy, General

A few things that caught my eye this week …

Posted on January 5, 2007April 8, 2017 by Eric Peterson

UPDATED: I am a bad blogger. I referenced my friend Dylan Lewis in my original post and didn’t link to his blog. Dylan can be read at http://www.passionateanalyst.com/ and I encourage you to ask about the Platypus thing. My sincere apologies, Mr. Lewis!

UPDATED: I totally forgot about all the 2007 web analytics predictions! You can see a nice summary list posted to the Yahoo! group by Lars but this list does not include one of my favorite set of predictions from Craig Danuloff. Craig I think makes the boldest predictions of anyone playing the game.

It occurred to me last night that I’ve been so engaged with measuring engagement that some interesting stuff has passed me by. Let’s catch up over coffee, shall we?

After being “tagged” by Gary Angel I tagged a few folks. So far Clint Ivy and Eric Butler have responded to my tagging, but perhaps the most interesting tag response comes from Dylan Lewis who postulates that the game of tag is either a thin disguise to increase our page ranks for searches for things like “web analytics and the grateful dead” or some intense navel gazing.
The “death of the page view” conversation, while interesting, is starting to go too far when otherwise smart people begin to predict things like “we’ll no longer bother to collect pageviews by end of 2007.” While you can make the case for using unique visitors in comparative situations, I sincerely question statements like “[the] page is no longer considered a metric worth looking at.” Is it me?
Justin Cutroni had a really good post a few weeks back titled “Web Analytics: It’s About Process” that I loved. I’m still well-engaged thinking about the processes behind the successful “doing” of web analytics at the Enterprise-level and have had several enlightening conversations lately. One of my favorite comments was “people often consider “process” to be a dirty word …” Ouch!
There is a new job posted on my job board, which is slow to take off likely because I have been too busy to bug all the recruiters posting to the Yahoo! group to give it a try! If you’re in Utah and have experience with the local analytics technology, have a look at this posting.
Finally, and those of us thinking about how “Web 2.0” is going to be measured knew this was bound to happen, measurement tools are coming to Second Life. While I’m not a Second-Lifer (I barely get everything done in my first life) I am dying to see what kinds of metrics Electric Sheep are able to come up with. Can you imagine the KPIs? “Percent Avatars propositioning sex” and “Percent Avatars pretending to be adults who were probably eleven-year-olds” and the such. Seriously though, if you have access to these reports, I would LOVE to see them.

As usual, I welcome your comments.