We want Web 2.0 measurement standards and we want them now!

I had the great fortune this week to have dinner with Bob Page (founder of Accrue, currently at Yahoo!), Xavier Casanova (founder of Fireclick, currently running Perenety) and Bryan and Jeffery Eisenberg (who need no introduction.) At one point Bob turned to me and commented that he had a new book he wanted me to write; Bob wanted to see my take on how to measure Web 2.0.

This is a great idea and something I’ve been thinking about a lot lately. Think about the inherent complexity involved with measuring activity on your web site, the one where you ultimately own everything. Now think about how you’d get this same type of robust measurement from a mash-up. When you have a Google map, a widget like Bitty Browser, a ZoomCloud tag cloud and more in your web page—all applications unto themselves that don’t provide ANY TYPE of reporting about visitor interaction to the best of my knowledge.

So AJAX and Web 2.0 present a whole new suite of challenges to anyone considering their use who is concerned about measurement. And I think this is a freaking fascinating subject personally–something that has the potential to change our industry–so I’ve decided to propose a measurement strategy for all Web 2.0/AJAX/RIA applications designed to be mashed into other web sites.

I mean, why not just suggest that Web 2.0 is incomplete without providing the ability to measure it’s adoption in a meaningful way? You know you we’re thinking it already, right?

So here goes …

I want to be able to, with any request for an external application, pass in a visitor identifier and have the application record event level data for me. Then, later, I want to make a different request to the application and get back event-level information for all of my visitors who interacted with the application, keyed to the visitor identifier I originially passed in.

The event-level data would differ by application type. A tag cloud would report back something like:

VISITORID001 DATE/TIME Clicked on “analytics”
VISITORID002 DATE/TIME Clicked on “peterson”
VISITORID001 DATE/TIME Clicked on “kpi book”
VISITORID003 DATE/TIME Clicked on “event api”

Whereas a more complex application like Google Maps might report back something like:

VISITORID001 DATE/TIME Zoomed map to level 4
VISITORID002 DATE/TIME Added directions to [ADDRESS]
VISITORID001 DATE/TIME Dragged map to center of [LAT/LONG]
VISITORID003 DATE/TIME Printed map

There would have to be a way to sync the dates and times such that these events could then be integrated into other clickstream data, essentially allowing these externally tracked events to be treated as “page views” in a clickstream path (“page view” being the canonical event type.) Still, those of you from the old guard–back when all we had was log files–will immediately recognize a simple log file and think to yourself either “Ahhhhh …” or “Dude, logs are so 1994!”

Regardless, those of us who use analytics packages that support hybrid data collection would simply take these logs and integrate them with our normally collected site data. This log-based approach is nice IMHO because the site owner could decide for his or herself which events they wanted included in the final analysis at integration time. Or, you could write a process that translated the text strings the application provider gave you into something more meaningful, you could categorize the events, etc.

Alternatively, assuming that few if any of the Web 2.0 application providers will actually build the necessary infrastructure to support this type of data capture and reporting, one could pass a reporting URL into the API and the application could use the reporting URL to log events using your existing meausurement application. For example, if I were to instantiate a Google Map like this:

var map = new GMap2(document.getElementById(“map”));
map.setCenter(new GLatLng(37.4419, -122.1419), 13);

Then the alternative method could be something like “setReportingURL” e.g.:

map.setReportingURL(“http://www.site.com/image.gif?ID=VISITORID001&EVENT=[E]”);

where the method would know to replace “[E]” with the encoded text string describing the actual event. This way, every time their was a meaningful event occurring, Google Maps would know to fire off a request to the tracking URL such that I could incorporate the event into my data set.

You’d almost surely have to also include the “ID=VISITORID001” string to properly sessionize the events and associate them with a visitor. The actual value of “ID=” would be set from the client based on the value of the tracking cookie.

I’m honestly not sure which approach I like better. The former is almost surely more robust from the viewpoint of creating a standard reporting API for “Web 2.0” applications, something Mark Baartse doubts will happen (Mark’s post being the only relevant Google result when I searched for “measuring Web 2.0”.) The latter is probably easier for the application providers to implement and more likely to work broadly given that only a handful of web analyics applications support integrating multiple data sources tied together by a common unique user identifier.

And yes, I realize that given a sufficiently robust API that you can already build this type of reporting. But man, wouldn’t it be easier if there was a standard tracking and reporting method that came with every such application that behaved the same way every time? Given that our industry has grown up almost entirely without standards, doesn’t this sound like a step in the right direction?

Keep in mind that this is only a technology proposal and only serves to highlight that every application provider would need to define and defend which events they reported and which they ignored. This is pretty much the same question we all asked ourselves seven years ago when Flash hit the scene and people started embedding script-based tracking inside Flash ActionScript.

It’s also the same question you have to ask and answer when you’re building any RIA: Which behaviors and actions are worth tracking and which should be ignored?

Maybe the practical solution is to set a level or threshold for event reporting–sort of a “few”, “many”, “firehose” hierarchy that would allow application designers to support a “log everything” mindset but allow their end users to be more conservative in what they actually collect and use. Few of the good Web 2.0 applications I’ve seen out there have enough events to really warrant this type of hierarchy; most are simple enough and likely to report click events which still provide great insight into visitor behavior.

Either way, there you have it, a line drawn in the sand. If Web 2.0 is going to change the Internet then I think the folks building these applications should play ball and report back on how our visitors are using their technologies. From the basic reporting I propose, sufficiently robust analytics packages will be able to calculate whether these applications have any effect on visitor retention, conversion, revenue, etc. Which, if you think about it, is exactly why the application providers should push reporting out ASAP.

Think I’m crazy? Have a great Web 2.0 application and want to implement my suggestion? Want to form an ad hoc Web 2.0 Measurement Standards Committee that meets on the second Wednesday of every month in a bar near you? Your comments are greatly appreciated and email is always answered.