Beefing Up the Integration of Optimizely and Google Analytics
Coming from a web analytics background, I tend to want almost any technology that relates to my site to be hooked in to the site’s web analytics. It doesn’t matter what your testing platform is or what your web analytics platform is, you’ll almost certainly want to link them up (Exhibit 1: Bryan Hawkins just wrote a great post detailing how to push Adobe Test&Target data into Google Analytics.)
What’s the Point of Integrating?
First, let’s define “integration” in the context of this post:
Integration means passing information into your web analytics tool that tells the tool if a visitor was exposed to a certain test, and, if they were, to which variation of the test they were exposed.
By enabling this integration, you enable segmentation of your traffic based on the different test variations, and you can do side-by-side comparisons of visitor behavior and any metric based on which variation of the test the visitor experienced. This is similar, in some ways, why you put campaign tracking parameters on links from banner ads to your sites: you get powerful data to augment the impression, clickthrough, and conversion (if tracking pixels are implemented) metrics provided by the media server.
The depth of out-of-the-box integration is one of the selling points for using testing and web analytics solutions from the same company (Adobe, Google, or Webtrends), but it is always possible — and not all that difficult — to enable a link between any testing technology and any web analytics platform.
This Post Just Covers Optimizely –> Google Analytics
Recently, we had a client that was using Optimizely and Google Analytics…and we quickly ran into some limitations of the out-of-the-box integration, which is well-documented by Optimizely. What the integration does is populate a visit-scope Google Analytics custom variable: the variable key is the name of the experiment, and the variable value is the variation that was served to the visitor. (My favorite explanation of Google Analytics custom variables, including what “visit-scope” means and the implications therein, is the first one Justin Cutroni wrote on the subject).
One shortcoming to Optimizely’s documentation is that, when it comes to using the integration, it only details how to build an advanced segment based on the custom variable. Custom variables also have their own reporting, including many standard metrics, available under Audience » Custom » Custom Variables, and custom variables can be used in custom reports. So, the post stops short of providing sufficiently deep documentation on how to get to custom variable data within Google Analytics.
But…the bigger issue are two limitations of the integration itself.
First, there is the fact that the free version of Google Analytics only has five custom variables available. If you’re not already using custom variables, then that means you can run five Optimizely experiments concurrently…so you’re probably fine. But, if you are using custom variables (to track registered users, logged in users, returning customers, page type, or any of the slew of other valuable uses), well, you should be! (Justin actually has written multiple posts with specific suggestions on that front.) With Optimizely’s standard Google Analytics integration, you can only integrate as many concurrent experiments as you have available custom variables.
A More Flexible Integration Approach
The approach we took allows an unlimited number of experiments to be run concurrently (of course, the more experiments there are running at the same time, the more risk there is that experiments will overlap, which has implications for test duration and, in some cases, results interpretation for individual tests). We did this using:
- Google Analytics Non-Interaction Events — a way to pass test information into Google Analytics without affecting any web analytics metrics (more on non-interaction events can be found in the Google Analytics documentation for event tracking)
/* _optimizely_evaluate=force */
if (typeof(optimizely) != "undefined" &&
window._gaq = window._gaq || ;
_gaq.push(['_trackEvent', 'Optimizely', optimizely.data.experiments[experimentId].name, optimizely.variationNamesMap[experimentId], 1, true]);
/* _optimizely_evaluate=safe */
With this code, a non-interaction event gets sent to Google Analytics each time the experiment is displayed. That event has a Category of “Optimizely,” an Action that is the name of the experiment, and a Label that is the variation of the experiment that was displayed.
With the event values sent to Google Analytics, you can explore visitor behavior for each version of the test to which the visitor was exposed:
- By building custom segments for each variation (use the Action and Label values to isolate visitors who were exposed to each variation of the experiment)
- By drilling down on Content » Events » Top Events » Optimizely and analyzing site usage and/or Ecommerce metrics
- By creating a custom report that breaks out the experiment variations as warranted with specific metrics of interest
Of course, before you go too nuts with your in-Google Analytics analysis, be sure to do a quick cross-reference with the Optimizely Results page for the experiment to make sure Google Analytics and Optimizely are within the same ballpark when it comes to how many times each variation of the test has been served (use the “Unique Events” or “Visits” metric in Google Analytics). They will never match, as they are capturing the test counts in very different ways, but they should be within spitting distance of each other. Be sure you have both Optimizely and Google Analytics set up to use the same timezone!
That’s all there is to it! Happy test analyzing!
*If you’re not familiar with the “up a creek” reference, see this link.