The Recent Forrester Wave on Web Analytics … is Wrong

Having worked as an industry analyst back in the day I still find myself interested in what the analyst community has to say about web analytics, especially when it comes to vendor evaluation. The evaluations are interesting because of the sheer amount of work that goes into them in an attempt to distill entire companies down into simple infographics, tables, and single paragraph summaries. Huge spreadsheets of data, long written answers, and multiple calls and product demos … all munged down into a single visualization designed to tell the large Enterprise which vendors to call and which to avoid.

In the early days of web analytics having access to these evaluations could be a huge time-saver. At the time there were dozens of vendors all embroiled in a battle for market-share, and so the vendor summary provided an “at a glance” view of the landscape that had the potential to save the Enterprise time and money. Plus, during the early growth period in web analytics, no one vendor had hegemony over the market and so any errors or inconsistencies in the results could easily be swept under the rug based on this being “an emerging market …”

Today, however, the web analytics market is functionally mature, and two vendors have emerged as “market leaders” based on their particular strengths and business models. I don’t even have to tell you who these vendors are; if you work in this industry or you are paying any level of attention to the technology landscape, you already know who they are … and who they are not … which brings me to the main topic I wanted to discuss:

The most recently published Forrester Wave on Web Analytics (Q2 2014) authored by James McCormick is wrong.

You can get a free copy of this report from Adobe, and I would encourage you to have a look yourself, but based on hundreds of implementations, vendor evaluations, RFP processes, and thousands of hours of work on our part, the Partners at Analytics Demystified and I can assure you that only one of the vendors dubbed a “leader” in this document is truly leading in the market today. Additionally, another vendor labeled a “strong performer” has consistently demonstrated more leadership and commitment to digital analytics than any of the vendors evaluated.

[At this point you may be asking yourself “why isn’t he naming names?” … which is a fair question. The old me was kind of a dick; the new me is trying to be less of a dick. I suspect that I am doing a poor job at that, but I am trying …]

I would encourage you, if you are interested, to review the scoring for the Wave reported in Figure 3 on page 9 … and ask yourself “do these results and, more importantly, these weightings, make sense?” For example:

• A zero weighting for “Market Presence” … despite the fact that two vendors have an increasing lock on the market in 2014, especially when you look at wins and losses in the last twelve months.
• The “Product” and “Corporate” strategy … which to me seem arbitrary at best, reporting that Google’s product and corporate strategy is “average” while that of a company that is on their third CEO and umpteenth head of Marketing is second only to A) a true market leader who is tied with B) a behemoth who is buying great companies but struggling to retain key employes who truly understand the market.
• “Application usability and administration” … reporting that again Google is behind a vendor who has not updated their core analytics application for an estimated ten years.
• The inclusion in the report of not one but two vendors whose names have not come up in Enterprise web analytics circles for years …

Take a look when you have a chance and see what you think. Maybe I’m the one who is wrong, and perhaps after 100+ collective years in this industry it is my Partners and I who have completely lost our connection to the web analytics vendor landscape …

…

At Analytics Demystified we rather enjoy the mature technology market we are working in today. With our clients increasingly standardizing on one, the other, or both of the true market leaders, our ability to move beyond the technology to how the technology is used effectively and efficiently in the business context is made that much easier. When analytics is put to use properly … good things happen.

I welcome your comments and feedback.