SiteCatalyst Implementation Pet Peeves – Follow-up [SiteCatalyst]

I recently blogged a list of my top Omniture SiteCatalyst implementation “Pet Peeves.” While the response to my post was very positive, one reader agreed with most of what I said, but disagreed with a few of my assertions or felt I had made some omissions. First, let me state that I always encourage feedback and comments to my blog posts since that helps everyone in the community learn. In general, the reader was making the point that my post only took into account an implementer’s perspective vs. the perspective of the web analyst. Personally, I don’t like to divide the world into implementers and analysts, since some of the best implementers I know also have a deep understanding of web analysis and vice-versa. Having been a web analytics practitioner using SiteCatalyst at two different organizations, I feel that I am in a good position to know if items I suggest (or discourage) will lead to fruitful analysis. I always try to write my blog from the perspective of the in-house web analyst who has to deal with things that I dealt with in the past, such as adoption, enterprise scalability, training, variable documentation, etc… In fact, I attribute much of my consulting success to the fact that I have been in the shoes of my clients and that they appreciate that my recommendations are based upon actual pains that I have experienced.

Since my original post was a very quick “Top-10” list and didn’t provide an enormous amount detail, and given the interest that it generated, I thought it would be worthwhile to write this follow-up post to address the concerns raised related to my post and to elaborate on the rationale behind some of my original assertions. In the process, it will become clear that I don’t necessarily agree with the concerns raised to my original post, but I am always cognizant of the fact that every client situation is different and every SiteCatalyst implementer has experiences that color their own implementation preferences. I don’t see it as my place to say which techniques are right and which are wrong, but rather to do my best to state what I think is/is not “best practice” and why based upon what I have seen and experienced over the past ten years and let my readers decide how to proceed from there…

Tracking Every eVar as an sProp

The first pet peeve I mentioned is when I find clients that have duplicated every eVar with a similar sProp. I stated that there are only specific cases in which an sProp should be used including a need for unique visitor counts, Pathing, Correlations and to store data that exceeds unique limits for accessing in DataWarehouse. The reader seemed to think I was being hard on the poor sProp and listed a few other cases where they felt duplicating an eVar with an identical sProp or adding additional sProps was justified including:

1. Using List sProps – The reader suggested that I had made an omission, by not mentioning List sProps as another reason to consider using an sProp in an implementation. I maintain that the use of List sProps was justifiably covered in my statement of other sProp uses that are “few and far between.” I don’t use List sProps very often because I feel that there are better ways to achieve the same goals. As the reader stated, List sProps have severe limitations and there is a reason that they are rarely used (maybe 2% of the implementations I have seen use them). I have found that you can achieve almost any goal you want to use List sProps for by re-using the Products variable and its multi-value capabilities instead. By using the Products variable, you can associate list items to KPI’s (Success Events) rather than just Traffic metrics. Using the reader’s own example of tracking impressions, illustrates the differences perfectly. You can store impressions and clicks of internal ads and calculate a CTR using the Products variable and two success events. This also gives you charts for impressions, clicks and the ratio of the two which can be easily added to SiteCatalyst dashboards. I have found that doing this with a List sProp is difficult, if not impossible and reporting on it is tedious. For more information on my approach, please check out my blog post on the subject.
2. Page-Based Containers & Segmentation – Here the reader suggested that the need to isolate specific pages using a Page View-based container is important to the life of the web analyst. Ben Gaines from Omniture also commented about this on my original post and I do agree that this can be useful for some advanced segmentation cases. I did not include this in my original list because I find it to be a much more advanced topic than I intended to cover for this quick “Top 10” post. While there may be cases in which you want to set an sProp to filter out specific items using a Page View-based segment container, I find that I often do this using the Page Name sProp which is already present. I do not see too many cases where a client is storing an eVar (let’s say Zip Code) and will say, “I am going to duplicate it as an sProp for the sole purpose of building a Page-Based container segment to include or exclude page views where a page is seen where a Zip Code equaled 123456.” Maybe that happens sometimes, but I still think it falls out of the scope of the primary things you should be considering when deciding whether to duplicate an eVar and I think it is a stretch to say that this functionality establishes the line between those who care about implementation and those who care about web analysis.
3. Correlations – With respect to Correlations, the reader suggested that users correlate as often as they can since cross-tabulation is so essential to the web analyst. This is exactly why I included Correlations in my list! I also mentioned that this justification for using an sProp may go away in SiteCatalyst v15 where all eVars have Full Subrelations. Also, one of the reasons I prefer Subrelations to Correlations is that Correlations only show intersections (Page Views) and do not show any cross-tabulation of KPI’s (Success Events). Personally, I would disagree with the reader about over-doing Correlations, since in my experience, implementing too many Correlations (especially 5-item or 20-item Correlations), with too many unique values, can cost a lot of $$$, lead to corruption and latency.
4. Pathing – In the area of Pathing, I think the reader and I are on the same page about its importance which is why I have published so many posts related to Pathing such as KPI (Success Event) Pathing, Product Pathing, Page Type Pathing, etc… Again, I might differ with the reader in that I don’t think enabling Pathing on too many sProps is a good idea since it can cost $$$ and produce report suite latency, which is why I prefer to use Pathing only when it adds value.

At the end of the sProp duplication section, the reader stated that there was no downside to duplicating every eVar as an sProp since it has no additional cost. To this, I would reiterate that my post was not advocating abandoning the use of sProps, but instead, attempting to help readers determine when they might want to use sProps so as to avoid over-using them when they will not add additional value. Even after years of education, I still find that many clients get confused as to whether they should use an eVar or an sProp in various situation, and most people I speak to welcome advice on how to decide if each is necessary.

However, I disagree with the reader’s assertion that duplicating every eVar as an sProp has no costs. Maybe it is due to the fact that I have “been in the trenches,” but in my experience I have seen the following potential negative ramifications:

• Over-implementing variables and enabling features unnecessarily can cause report suite latency
• Over-implementing variables can increase page load time, which can negatively impact conversion
• Over-implementing variables and features can cost additional $$$ as described above (e.g. Pathing, Correlations)
• When you implement SiteCatalyst on a global scale, you often need to conserve variables for different departments or countries to track their own unique data points. This means that variables (even 75 of them!) are at a premium. Therefore, duplicating variables has, at times, caused issues in which clients run out of usable variables.
• Most importantly, however, is the impact on adoption. Again, I may be biased due to my in-house experience, but here is a real-life example: Let’s say that you have duplicated all eVars as sProps. Now you get a phone call from a new SiteCatalyst user (who you have begged/pleaded for six months to get to login!). The end-user says they are trying to see Form Completions broken down by City. They opened the City report, but were only able to see Page Views or Visits as metrics. Why can’t they find the Form Completions metric? Is SiteCatalyst broken? Of course not! The issue is that they have chosen to view the sProp version of the report instead of the eVar version. That makes sense to a SiteCatalyst expert, but I have seen the puzzled look on the faces of people who don’t have any desire to understand the difference between an sProp and an eVar! In fact, if you try to explain it to them, you will win the battle, but lose the war. In their minds, you just implemented something that is way too complicated. You’ve just lost one advocate for your web analytics program – all so that you can track City in an sProp when you may not have needed to in the first place. In my experience, adoption is a huge problem for web analytics and is a valid reason to think twice about whether duplicating an sProp is worthwhile. While I’ll admit that duplicating all variables certainly helps “cover your butt,” I worry about the people who are at the client, left to navigate a bloated, confusing implementation…

Therefore, for the reasons listed above, I remain steadfast in my assertion that there are cases where sProps add value and cases where they just create noise. While there will always be edge cases, I think that the justifications I laid out in my original post are the big ones that the majority of SiteCatalyst clients should think about when deciding if they want to duplicate an eVar as an sProp or use an sProp in general.

As an aside, while we are revisiting my original post, I thought of a few more items I wish I would have included so I will list them here:

1. One other justification for setting an sProp I should have mentioned is Participation. There are some fun uses of Participation that can improve analysis and I find that sProp Participation is easier to understand for most people than eVar Participation so I would add that to my original list.
2. If you do find a need to duplicate an eVar as an sProp, but it is only for “power users,” keep in mind that you can hide the sProp variable from your novice end-users through the security settings under Groups.
3. Finally, I see Omniture ultimately moving to a world where there will only be one variable so if you want to be part of that world, please vote for my suggestion of doing this in the Ideas Exchange here.

VISTA Rules

Another pet peeve I mentioned is that I often find clients who are using VISTA rules too often or as band-aids. The reader stated that VISTA rules are a good alternative to JavaScript tagging since they can speed up page load times. I think this is another situation where my time working at Omniture and in-house managing SiteCatalyst implementations may bias my recommendations. While I agree that page load time is important, most Omniture clients I saw never mentioned using VISTA rules as a way to decrease page load time, but rather as a way to avoid working with IT! Usually, when I find a client that has many VISTA rules, it is because they have a bad relationship with IT, who doesn’t want to do additional tagging, rather than to save page load time. However, if I were to address the reader’s point of page load speed, I would agree that there are cases where using VISTA rules over JavaScript can decrease page load time, but I certainly do not think this should be the primary deciding factor. Great strides have been made in tagging including things like dynamic variable tagging and tag management tools which have greatly reduced page load speeds. I suggest readers check out Ben Robison’s excellent post on VISTA vs. JavaScript which discusses not only page load speed, but also the many other important factors to consider before jumping into VISTA rules.

Another point I’d like to make about VISTA rules is that, in my experience, they have a high likelihood of breaking and leading to periods of bad data. VISTA rules are like Excel macros. They do what you tell them to do, but if something changes, it can easily throw off a VISTA rule and cause incomplete or inaccurate data to be reported in SiteCatalyst. In this point, perhaps I am a bit jaded because I have seen so many different VISTA implementations go awry while I was at Omniture. In fact, it is rare that I find clients that have a VISTA rule that has worked for several years without ever having an issue. And if you do encounter an issue, you will have to pay Omniture around $2,000 to update it – every time. Want to make an update to the VISTA rule? $2,000. Want to turn off the VISTA rule or move it to a different report suite? $2,000! Consultants don’t have to write these checks, but guess who does – the in-house people do! This is why people are so excited about the new V15 processing rules and emerging tag management vendors. It is this tendency to break and the risk of bad data that makes me a bit gun-shy about using VISTA rules simply as a replacement for JavaScript tagging. Moreover, since the reader’s overall premise was that one must keep the web analyst in mind during implementation, I would be cautious about being overly-reliant on a solution like VISTA that is so prone to causing data issues which could thwart the analyst’s ability to do web analysis. I have seen companies that have 20+ VISTA rules and I promise you that they are not huge fans of VISTA right now (though they should really blame themselves not the tool!)! If you do pursue VISTA rules, my advice is that you consider using DB VISTA over VISTA. DB VISTA rules cost a bit more, but do offer more flexibility since you can at least make updates to the data portion of your rules without having to pay Omniture additional $$$.

One additional point to think about when it comes to VISTA rules is the impact they can have on report suite latency. Having too many VISTA rules can slow down your ability to get timely data in SiteCatalyst and I have seen many large organizations have severe (several days) report suite latency due to multiple VISTA rules acting on each server call. This impacts the web analyst’s ability to get the data they need and should be factored into decisions about VISTA rules.

As I stated in my original post, I have nothing against VISTA rules, but do find the overuse of them to be a potential red flag when I look at a new implementation. I often find that excessive use of VISTA Rules can be a symptom of bigger problems which merit investigation. Just like I don’t advocate duplicating sProps or enabling Pathing when not necessary, I don’t advocate the use of too many VISTA rules since it can be great in the short term, but bad in the long term. Now that I am a consultant again, it would be easy for me to recommend VISTA rules left and right, but since I like to have long-term relationships with my clients, I don’t do this since I know what it is like to be around later if/when they have issues!

Final Thoughts
I hope this post provides some good food for thought and more in-depth information about some of the items I listed in my original post. If you would like to discuss any of the above topics in more detail, feel free to leave comments here or e-mail me. Thanks!