Big vs. Little Implementations [SiteCatalyst]

Over the years, I have worked on Adobe SiteCatalyst implementations for the largest of companies and the smallest of companies. In that time, I have learned that you have to have a different mindset when it comes to each type of implementation. Implementing both the same way can lead to issues. Big implementations (which can be either large due to complexity or traffic volume) are not inherently better or worse, just different. For example, an implementation at a company like Expedia is going to be very different than an implementation at a small retail website. Personally, I find things that excite me about both types. When working with a large website, the volume of traffic can be amazing and your opportunities to improve conversion are enormous. One cool insight that improves conversion by a small percentage, can mean millions of dollars! Conversely, when working with a smaller website, you usually have a smaller development team, which means that you can be very agile and implement things almost immediately.

Hence, there are pros and cons with each type of website and these are important things to consider when approaching an implementation or possibly when considering what type of company you want to work for as a web analyst. The following will outline some of the distinctions I have found over the years in case you find them to be helpful.

Implementation Differences

The following are some of the SiteCatalyst areas that I have found to be most impacted by the size of the implementation:

Multi-suite Tagging
Most large websites have multiple locations, sites or brands and use multi-suite tagging. When you bring together data from multiple websites into one “global” suite, you have to be sure that all of the variables line up amongst the different child report suites. Failure to do this will result in data collisions that will taint Success Event metrics or combine disparate eVar/sProp values. If you have 10+ report suites, it almost becomes a full-time job to manage these, making sure that renegade developers don’t start populating variables without your knowledge. If you use multi-suite tagging and have a global report suite, my suggestion is to keep every report suite as standardized as possible. This may sound draconian, but it works.

For example, let’s say you have five report suites that are using eVars 1-45 and a few other report suites that require some new eVars. Even if the latter report suites don’t intend to use eVars 1-45 (which I doubt), I would still recommend that you use eVars 46 on for the new eVars for the additional report suites. This will ensure that you don’t encounter data conflicts. Taking this a step further, I would label eVars 1-45 as they are in the initial report suites using the Administration Console. I would also label eVars 46 on with the new variable names in the original set of report suites. At the end of the day, when you highlight all report suites in the Admin Console and choose to see your eVars, you should strive to see no “Multiple” values. That means you have a clean implementation and no variable conflicts. Otherwise, you will encounter what I call “Multiple Madness” (shown here).

If you really have a need for each website to track its own site-specific data points, one best practice is to save the last few Success Events, eVars and sProps for site-specific variables. For example, you may reserve Success Events 95-100 and eVars 70-75 to be different in each report suite. That will provide some flexibility to site owners. You just have to recognize that those Success Events and eVars should be hidden (or disabled) in the global report suite so there is no confusion. Another exception to the rule might be sites that are dramatically different than the core websites. For example, you may have a mobile app or intranet site that you are tracking with SiteCatalyst. This mobile app or intranet site may be so drastically different from your other sites that you want to have it in its own separate report suite that will never merge with your other report suites. In this case, you can either create a separate Company Login or just keep that one report suite separate from the others and use any variables you want for it. Keep in mind that the Administration Console allows you to create “groups” of report suites so you can group common ones together and use that group to make sure you don’t have any “multiple” issues. You can also use the Menu Customization feature to hide variables in report suites where they are not applicable. Even if you don’t currently have a global report suite, I still recommend following the preceding approach. You never know when you might later decide to bring multiple report suites together, and using my approach makes doing so a breeze (simply changing the s_account variable) versus having to re-implement variables and move them to open slots at a later date. The latter will cause you to lose historical trends, modify reports and dashboards and confuse your end-users.

When you have a smaller implementation, it is common to have just one production report suite. This avoids the preceding multi-suite tagging issues and makes your life a lot easier!

Variable Conservation
As if coordinating variables across multiple report suites isn’t hard enough, this issue is compounded by the fact that multi-suite tagging means that you only have ~110 success events, ~78 eVars and ~78 sProps to use for all sites together vs. being able to use ~250 variables differently for each website. This means that most large implementations inevitably run out of variables (eVars are usually the first type of variable to run out). Therefore, large implementations have to be very aggressive on conserving variables, which can handcuff them at times. As a web analyst, you can often make a case for tracking almost anything, since the more data you have the more analyses you can produce and the more items you can add to your segments. Unfortunately, when dealing with a large implementation, for the reasons cited above, you may need to prioritize which data elements are the most important to track lest you run out of variables. This isn’t necessarily a bad thing as it helps your organization focus on what is really important across the entire business and tracking more isn’t always better.

If you contrast this with a smaller implementation that has no multi-suite tagging and no global report suite, the smaller implementation is free to use all variables for the one site being tracked. This provides ~250 variables to use as you desire. That should be plenty for any smaller site, so variable conservation isn’t as high of a priority. A few times, in my SiteCatalyst training classes, I have had both large and small companies sitting next to each other, and have witnessed the big company drooling over the fact that the smaller company was only using 20 of their eVars (wishing they could borrow some)! While it may sound strange, there are many cases in which I would tell a smaller organization to set success events and eVars that I would conversely tell a large organization not to set. For example, if I were working with a small organization that had only one workflow process (i.e. credit card application) and they wanted to track all six steps with success events, I might say “go for it!” But if that same scenario arose for a large website (i.e. American Express), I would encourage them to only set success events for the key milestone workflow steps to conserve success events. This is just one example of why I tend to approach large and small implementations differently.

One final note related to variable conservation. Keep in mind that you can use concatenation combined with SAINT Classifications to conserve variables. For example, instead of storing Time of Day, Day of Week and Weekday/Weekend in three separate eVars, you can concatenate those together into one and apply SAINT Classifications. This will save a few eVars and a similar process can be replicated for things like e-mail attributes, product attributes, etc.

Uniques Issues
If you have a large website, there is an increased chance you will have issues with “uniques.” Most eVar and sProp reports have a limit of 500,000 unique values per month. I have many large clients that try to track onsite search phrases or external search keywords and exceed the unique threshold by the 10th day of the month. This makes some key reports less useful and often results in data being exported via a data feed or DataWarehouse report to back-end tools for more robust analysis. For some large implementations, since the data points can’t be used regularly in the SiteCatalyst user interface due to unique limits, I sometimes have clients pass data to an sProp to conserve eVars, since in DataWarehouse, Discover and Segmentation, having values in an sProp is similar to having it in an eVar.

Smaller implementations normally only hit uniques issues if they are storing session ID’s (i.e. ClickTale, Tealeaf) or customer ID’s.

Large # of Page & Product Names
Many large websites have so many pages on their site (i.e. one page per product and over 100,000 products) that having an individual page name for each page is virtually impossible. In these cases, you often have to take page names up a level and start at a page category level. The same concept can apply to individual product names or ID’s as well.

Smaller implementations rarely have these issues since they tend to have fewer pages and numbers of products.

Page Naming Conventions
Another area where I see those running large implementations make mistakes is related to page naming across multiple websites. If you are managing a smaller implementation, you can name your pages anything you’d like. For example, while I don’t recommend it, if you want to call your website home page, “Home Page,” you will be ok. However, this approach won’t always work with a large implementation. If you have five report suites and one global report suite and you named the home page of each “Home Page,” in the global report suite, you would see data from all five report suites merged into one page name called “Home Page.” While there may be reasons to do this, you will probably also want to have a way to see things like Pathing and Participation for each of the home pages from each site individually in the global report suite. In this post, I show how you can have both (“have your cake and eat it too!”), but this example highlights the complexity that can arise when dealing with larger implementations.

SAINT Classifications
Large websites can often have a variable with more than a million SAINT classification values. Updating SAINT tables can take days or weeks unless you are methodical about your approach. Smaller sites with lower numbers of SAINT values can often re-upload their entire SAINT file daily or weekly to make sure all values are classified. Large implementations don’t have this luxury. They have to monitor which values are new or missing SAINT values so they can only upload the new or changed items so it doesn’t take weeks for SAINT tables to be updated. If you work with a large implementation, keep in mind that you can update SAINT Classifications for multiple report suites with one upload if you use the FTP method vs. browser uploads.

Time to Implement
In general, large implementations tend to move slower than smaller ones. While tag management systems are helping to remedy this, I still find that adding new variables or fixing broken variables takes much longer with large implementations (often due to corporate politics!). This means that you have to be sure that your tagging specifications are right the first time, since getting changes in after a release may be difficult.

Conversely, with smaller websites, you can be much more nimble and update SiteCatalyst tagging on the fly. For example, you may doing a specific analysis and realize that it would be helpful for you to have the Zip Code associated with a form. If you work with a smaller site, you may be able to use a SiteCatalyst Processing Rule or call your developer and have them add Zip Code to eVar30 and have data the same day!

Globally Shared Metrics, Dashboards, Reports, etc.
When you work with a small implementation, you may have a few calculated metrics, dashboards or reports that you share out to your users. This is a great way to collaborate and enforce some standards or consistency related to your implementation. However, when you have a large implementation, sometimes with 300+ SiteCatalyst users having logins, this type of sharing can easily get out of control. Imagine each SiteCatalyst user sharing five reports or dashboards. The shared area of the interface becomes a mess and you are not sure which reports/dashboards you should be using. Therefore, when you are working with a large implementation, it is common to have to implement some processes in which reports and dashboards are sent to the core web analytics team who can then share them out to others. This allows the SiteCatalyst user community to know which reports/dashboards are “approved” by the organization. You can learn more about centralizing reports and dashboards by reading this blog post.

Final Thoughts

As I mentioned in the beginning of this post, bigger isn’t always better. As shown from the items above, I often find that bigger implementations lead to more headaches and more limitations. However, keep in mind that with great volume, comes conversion improvement opportunities that often dwarf smaller sites.

One over-arching piece of advice I would give you, regardless of whether you work with a large or small implementation, is to review your implementation every six months (or at least yearly) and determine if you are still using all of your variables. It is better to get rid of what you no longer need periodically than to have to do a massive overhaul one day in the future.

While this post covers just a few of the differences between large and small implementations, they are the ones that I tend to see people mess up the most. If you have other tips for readers, feel free to leave a comment here. Thanks!