A/B Test Bounce Rates

In the past, I have written about Bounce Rates, Traffic Source Bounce Rates , Segment Bounce Rates and Site Wide Bounce Rates. In the latter, I even promised I was finished writing about Bounce Rates, but, alas, I have yet another Bounce Rate installment. I was recently in a conversation with a peer and she asked me how they could see the bounce rates of the various landing page A/B tests they were running via Test&Target. I told her that this was easy to do if you follow my instructions in the Segment Bounce Rate post, but she asked if I could write a brief post with more specifics so here it is…

Why A/B Bounce Rates?
Before getting into the solution, let’s re-visit why this is of interest. Test&Target (and other tools like GWO) are wonderful when it comes to optimizing landing pages. They allow you to alter content/creative elements and see what works and what doesn’t. I have seen many cases where clients have used tools like Test&Target to change content based upon when brought the user to the website (i.e. Search Keyword) or demographic information (i.e. Location). Regardless of the reason you want to test, if it is a landing page, one of the questions you often get asked is related to Bounce Rate. Understanding how many people saw “Version A” of a test and bounced vs. those who saw “Version B” and bounced usually comes up for discussion. To answer this question using Omniture/Adobe tools you have the following options:

• Create a unique page name for each test variation and use the regular Pages report and Bounce rate metric. However, this can get very messy, so unless your website is small, I don’t recommend this approach.
• Use ASI or Discover to build a segment for people coming from “Version A” or “Version B” and then compare the bounce rates. This is a viable option if you have access to these tools and are well versed in Segmentation.
• Attempt to track Bounce Rates from within Test&Target. This does not come out-of-the-box, but if you have mboxes on all of the pages the landing page links to, I have heard of some people setting conversion events on the landing page and the subsequent pages, but I don’t think this is for novices (if you are interested, I’m sure @brianthawkins could figure out a way to hack this together!)
• Do what I suggest below!

Implementing A/B Bounce Rates
Luckily, implementing this in SiteCatalyst is relatively simple. All you need to do is the following:

1. Enable a new Traffic Variable (sProp)
2. In this new sProp, concatenate the Test&Target ID and the Page Name on each page of your website
3. Enable Pathing on the new sProp

That’s it! By concatenating the Test&Target ID and the Page Name, you create a unique join between the two and can find the combination of the Test ID you care about and the page name that you expect them to have landed on. Once you find this combination in the report, you can add your Bounce Rate Calculated Metric (Single Access/Entries – which hopefully you already have as a Global Calculated Metric) and you are done. Here is an example of a report:

In this report, you have all of the ID’s associated with the US Home Page, how many Entries each received and the associated Bounce Rate. If you wanted, you could perform a search for the specific Test&Target Test ID you care about and then your report would be limited to just those ID’s. In the example above, we have multiple tests taking place on the US Home Page. However, in the following example we can see a case where there is just one test taking place on the UK Home Page and the associated Bounce Rate of each:

Other Cool Stuff
But wait…there’s more! Since you have enabled Pathing on this new A/B Test sProp, there are some other cool things you can do. First, you can look at a trended view of the report above to see how the Bounce Rate fluctuates during the course of the test. To do this, simply switch to the trended view and choose your time frame:

Another benefit of having Pathing enabled on this sProp is that you can see how visitors from various tests navigated your site using all of the out-of-the-box Pathing reports. Here is an example of a next page flow for one of the tests:

You can run the preceding report for each test variation and compare the path flows to see if one version pushes people more often to the places you want them to go. Another report you could run is a Fall-Out report which can show you how often people from a specific test made it through your desired checkpoints:

In this example, instead of seeing how the general population falls-out from the Home Page to a Product Page and then to a Form Page, we can limit the funnel to only those people who were part of Test ID “18964:1:0.” I like to run this report and the corresponding one for the other test version(s) and add them all to a SiteCatalyst Dashboard where I can see the fall-out rates side by side.

Final Thoughts
As you can see, by doing a little up-front work, you can add an enormous amount of insight into how your A/B tests are performing on your site including Bounce Rates, Next Page Flows, Fall-Out, etc…Enjoy!