Product Segmentation Gotchas
If you have used Adobe Analytics segmentation you are likely very familiar with the hierarchy of container. These containers illustrate the scope of the criteria wrapped in the container and are available at the visitor, visit, and hit levels. These containers help you control exactly what happens on each of those levels and your analysis can be heavily impacted by which of these you use. These are extremely useful and handle most use cases.
When doing really detailed analysis related to products, however, the available containers can get confused. This is because there can be multiple products per visitor, visit, or hit. Scenarios like a product list page and checkout pages, when analyzed at a product level, can be especially problematic. Obviously this has a disproportionate impact on retailers but other industries may also be impacted if they use the products variable to facilitate fancy implementations. Any implementation that has a need to collect attributes that have a many-to-many relationship may need to leverage the products variable.
Following are a few cases illustrating where this might happen so be on the lookout.
Product Attributes at Time of Order
Let’s say you want to segment for visits that purchased a product with a discount. Or, rather than a discount, it could be a flag indicating the product should be gift wrapped. It could even be some other attribute that you want passed “per product” on the thank you page. Using the scenario of a discount, if a product-level discount (e.g. 2 for 1 deal) is involved and that same discount can apply to other products, you won’t quite be able to get the right association between the two dimension. You may be tempted to create a segment like this:
However, this segment can disappoint you. Imagine that your order includes two products (product A and product B) and product B is the one that has the “2_for_1” discount applied to it (through a product syntax merchandising eVar). In that case the visit will qualify for our segment because our criteria will be applied at the hit level (note the red arrow). This setting will result in the segment looking for a hit with product A and a code of “2_for_1” but it doesn’t care beyond that. This segment will include the correct results (the right discount associated with the right product), but it will also include undesired results such as right discount associated with the wrong product. This is caused when the correct product just so happened to be purchased at the same time. In the end you are left with a segment you shouldn’t use.
This example is centered around differing per-product attributes at the time of an order but really the event doesn’t matter. This could apply at any time you have a bunch of products collected at once that may each have different values. If multiple products are involved and your implementation is using merchandising evars with product syntax (correctly) then this will be a consideration for you.
Differentiating Test Products
I once had a super-large retailer run a test on a narrow set of a few thousand products. They wanted to know what kind of impact different combinations of alternate images available on the product detail page would have on conversion. This included still images, lifestyle images, 360 views, videos, etc. However, not all products had comparable alternate images available. Because of this they ran the test only across products that did have comparable imagery assets. This resulted in the need to segment very carefully at a product level. Inevitably they came to me with the question “how much revenue was generated by the products that were in the test?” This is a bit tricky because in A/B tests we normally look at visitor-level data for a certain timeframe. If someone in the test made a purchase and the test products were only a fraction of the overall order then the impact of the test could be washed out. So we had to get specific. Unfortunately, through a segment alone we couldn’t get good summary information.
This is rooted in the same reasons as the first example. If you were to only segment for a visitor in the test then your resulting revenue would include all orders for that visitor while in that test. From there you could try to get more specific and segment for the products you are interested in; however, the closest you’ll get is order-level revenue containing the right products. You’ll still be missing the product-specific revenue for the right products. At least you would be excluding orders placed by test participants that didn’t have the test products at all…but a less-bad segment is still a bad segment 🙂
Changes to Product Attributes
This example involves the fulfillment method of the product. Another client wanted to see how people changed their fulfillment method (ship to home, ship to store, buy online/pickup in store) and was trying to work around a limited implementation. The implementation was set up to answer “what was the fulfillment method changed to?” but what they didn’t have built in was this new question — “of those that start with ship-to-home products in the cart, how often is that then changed to ship to store?” Also important is that each product in the cart could have different fulfillment methods at any given time.
In this case we can segment for visits that start with some product with a ship-to-home method. We can even segment for those that change the fulfillment method. We get stuck, though, when trying to associate the two events together by a specific product. You’re left without historical data and resorting to implementation enhancements.
Other Options
The main point of this post is to emphasize where segmenting on products could go wrong. There are ways to work around the limitations above, though. Here are a few options to consider:
- In the case of the product test, we could apply a classification to identify which products are in the test. Then you would just have to use a table visualization, add a dimension for your test groups, and break that down by this new classification. This will show you the split of revenue within the test group.
- Turn to the Adobe Data Feed and do some custom crunching of the numbers in your data warehouse.
- Enhance your implementation. In the case of the first scenario where persistence isn’t needed you could get away with appending the product to the attribute to provide the uniqueness you need. That may, though, give you some issue with the number of permutations that could create. Depending on how into this you want to get, you could even try some really crazy/fun stuff like rewriting the visitor ID to include the product. This results in some really advanced product-level segmentation. No historical data available, though.
- Limit your dataset to users that just interacted with or ordered one product to avoid confusion with other products. Blech! Not recommended.
Common Theme
You’ll notice in all of these examples the common thread is where we are leveraging product-specific attributes (merchandising eVars) and trying to tease out specific products from other products based on those attributes. Given that none of the containers perfectly match the same scope of a product you may run into something like the problems described above. Have you come across other segmenting-at-a-product-level problems? If so please comment below!
