Optimization Test Techniques – Part I of II
As I talk to more and more companies that are using testing solutions, I find many of them are unaware of the test techniques that are available in their testing platform. Testing solutions available today offer more then just A/B and multivariate testing capabilities. There are different techniques around multivariate tests but there are also other test techniques available that offer additional strategy for your tests. Familiarizing yourself with the different techniques available will allow you to get much more value out of your testing solution and your optimization program.
| Here I will share what test techniques or types that are currently available in Adobe’s Test&Target (T&T) platform.
In T&T, tests types are referred to as campaigns and campaign types. All the campaign types use the same core components such as the mbox and the offer. The mbox, which is short for marketing box, does many things but for this topic, it is best to think of it as the area of real estate on the website that you wish to assign content as part of a test. That content that you assign as part of the test is your offer. Campaigns are where you assign business rules to your mboxes and offers. |
1:1 Campaign
The 1:1 campaign is a campaign type that is only available to those customers that have a Test&Target1:1 license. Test&Target1:1 is the former Touch Clarity product acquired by Omniture and has since been incorporated into the Test&Target platform as a test type allowing users to leverage a shared profile and a single platform for their optimization efforts.
The 1:1 campaign is designed to leverage models to determine the right content to present to the individual vs. a segment of visitors. These models are focussing on a single success event that you specify in the campaign setup. These events can be anything that can happen in a session such as: click through, form complete, purchase, revenue per visitor, etc…
There will be two branches of this type of test, similar to an A/B test. The first branch serves as a control and is presented to 10% of traffic. These visitors will randomly see any one of the offers you are using in the test. The engine learns from this 10% of traffic by understanding how visitors react to the content and then correlating that reaction to the profile attributes of those visitors.
The other 90% of traffic benefits from this by receiving targeted content based off of the real time scoring the 1:1 engine provides.
I have seen this campaign type offer a ton of value to customers in highly trafficked pages such as the home page or main landing pages. The big benefit here is the automation. You set it up and let it do its thing with minor tweaking here and there.
This is what the summary report looks like in the 1:1 campaign type where you can see the two branches of the test:
The other key value that this test type provides is what is called an “Insights” report. Yep, Adobe has an Insight product for analytics and also a report in 1:1 called Insights. This Insights report in 1:1 provides data on what profile attributes of visitors are offer a positive and negative propensity against a given offer. In other words, this report discovers segments or profile attributes that are impactful. Here you can learn things like people on their 3rd session and are from California respond positively to a particular offer – hence discovering this segment for you and providing a marketing insight that can be used in other tests or in offline marketing!
1:1 Campaign Display
This campaign type is the exact same at the 1:1 except that it is used in display ads versus a website.
A/B..N Campaign
This is by far the most popular of the campaign types and, as you can imagine, it is the test type that allows you to compare two different experiences. You can have just two experiences competing against each other but you can also incorporate as many different experiences as your traffic and creative permits. Here is what the architecture of a standard A/B test looks like with two different offers being assigned to two different mboxes:
An important thing to note regarding the A/B test is that whatever experience or branch of the test the visitor falls into, they are stuck with the experience for the life of the campaign. That is, if they continue to visit the area that is being tested, they will continue to see that test content until they convert which is defined as the primary success event in T&T.
Flash Campaign
This campaign type is used when you wish to test content within flash files. This is a great technique to use if you wish to apply optimization to your display ads. Onsite profiles collected by T&T can be used for quick and easy targeting with this type of campaign.
Adobe’s CS5 of Flash has productized the integration with T&T in that within CS5 of Flash, you can leverage a Flash Extension to quickly “mbox” components of the flash asset to be used as part of a test in T&T. In T&T then you select the Flash campaign and during the campaign setup, you either upload the “mboxed” flash file or point to where it lives in the network. T&T then identified the “mboxed” components where you can assign alternative content to it as part of the test.
The flash campaign follows the same technique as an A/B test in that visitors are stuck with whatever experience they were originally provided.
In the follow up post, I will highlight the Monitoring Campaign, Multivariate Campaigns and the Optimizing Campaign.
