New or Old Report Suite When Re-implementing?
In the recent white paper I wrote in partnership with Adobe, I discuss ways to re-energize your web analytics implementation. Often times, this involves re-assessing your business requirements and rolling out a more updated web analytics implementation. However, if you decide to make changes to your implementation in a tool like Adobe Analytics (SiteCatalyst), at some point you will have to make a decision as to whether you should pass new data into the existing report suite or begin fresh with a new report suite. This can be a tough decision, and I thought I would use this blog post to share some things to consider to help you make the best choice for your organization.
Advantages of Using The Existing Report Suite
To begin, let’s look at the benefits of using the same report suite when you re-implement. The main one that comes to mind is the ability to see historical trends of your data. In web analytics, this is important, since seeing a trend of Visits or Orders gives you a better context from which to analyze your data. In SiteCatalyst, you get the added benefit of seeing monthly and yearly trend lines in reports to show you month over month and year over year activity. Obviously, if you decide to start fresh with a new report suite, your users will only see data from the date you re-implement in the SiteCatalyst interface.
Another benefit of continuing with your existing report suite is that you will retain unique visitors for those that have visited your site in the past and have not deleted their cookies. When you begin with a new report suite, all visitors will be new unique visitors so you will be starting your unique visitor counts over from the day you re-implement. Starting with a new report suite will also result in some recency reports(i.e. Visit Number, Returning Visitors and Customer Loyalty) being negatively impacted. Additionally, using an existing report suite allows you to retain any values currently persisting in Conversion Variables (eVars). Often times you have eVar values that are meant to persist until a KPI takes place or until a specific timeframe occurs. If you create a new report suite, all eVars will start over since they are tied to the SiteCatalyst cookie ID.
Another area to consider is Segmentation. It is common to use a Visitor container within a SiteCatalyst segment to look for visitors who have performed an action at some point in the past. This segment will rely on the cookie ID so if you begin with a new report suite, you will lose visitors in your desired segment. For example, let’s say you have a segment that looks for visitors who have come from an e-mail at some point in the past and ordered in today’s visit. If you create a new report suite, you will lose all data from people who may have come from an e-mail prior to the new report suite being created.
If your end-users have dashboards, bookmarks and alerts setup, using the existing report suite will avoid the need to re-create them in the new report suite for variables that remain unchanged. Depending upon how active your users are, this can have a significant impact, as re-creating these can result in a lot of re-work.
There are many other items to consider, but these are the ones that I have seen come up most often as advantages of keeping the existing report suite when re-implementing.
Advantages of Using A New Report Suite
So now that I have scared you off of using a new report suite when re-implementing, let me take the counter-arguement. Despite all of the advantages listed above, there are many cases in which I recommend starting with a brand new report suite. The most obvious is when the current implementation is proven to be grossly incorrect or misaligned. I often encounter situations in which the current implementation hasn’t been updated for years and not at all related to what is currently on the website (or mobile app). If what you have doesn’t answer the relevant business questions, all of the advantages listed above become obsolete. In this situation, seeing historical trends of irrelevant data points, losing eVar values or report bookmarks isn’t a big deal. You may still lose out your historical unique visitor counts since that is out-of-the-box functionality, but I don’t think this justifies not starting with a clean slate. If you are not sure if your current implementation is aligned with your latest business goals, I highly recommend that you perform an implementation audit. This will help you understand how good or bad your implementation is, which is a key component of making the new vs. existing report suite decision.
The next situation is one in which the current implementation is using many of the allotted SiteCatalyst variables, but the new implementation has so much data to collect that it has to re-use the same variables going forward. This gets messy since it is easy to re-name existing variables, but you cannot remove historical data from them. Therefore, if you convert event 1 from “Internal Searches” to “Leads,” because you no longer have a search function and are out of success events, you can get into trouble when your end-users view a trend of leads for this month and see that they are a fraction of what they were last year! Your users may not understand that the data they are seeing from last year is “Internal Searches” and not “Leads,” and may sound off alarms indicating that the website is broken and conversion has fallen off the cliff! While you can do your best to annotate SiteCatalyst reports and educate people, the re-use of existing variables is always a risk, whereas using a new report suite does not require the re-use of existing variables and can avoid this confusion. Where possible, I suggest that you use previously unused variables for your new implementation so this historical data issue doesn’t affect you. Obviously, this requires that your existing implementation isn’t using most or all of your available SiteCatalyst variables. Hence, one key factor when deciding whether to use an existing report suite or create a new one is counting the number of incremental variables you will need variable slots for and determining whether you have enough to avoid having to re-use old variables for new data. If you have enough, that may tip the scale to re-use, but if you don’t, it may make you lean towards a new report suite.
When it comes to historical trends, one thing to keep in mind is that even if you choose to create a new report suite, it is still possible to see historical trends for data that the new and old report suites have in common. This can be done by importing data into the new suite using Data Sources. This is most effective when the data you are uploading are success events (numbers) and a bit more difficult for eVar and sProp data. The main benefit of this approach is that it allows your SiteCatalyst users to see the data from within the SiteCatalyst interface. Another option is to use Adobe ReportBuilder. Within Excel, you can build a data block for the data in the old report suite and then another data block for the same data in the new report suite and then merge the two together in a graph using two data ranges. Doing this allows you to create charts and graphs that span the old and the new, but these are only available in Excel and not in the SiteCatalyst interface.
Another justification for starting with a new report suite is that your current suite has data that is untrustworthy. I often talk to companies who say that they simply do not trust that the data in SiteCatalyst is correct. As I mention in the white paper, trust is an easy thing to lose and a hard thing to earn back. Your SiteCatalyst reports can be correct nine times out of ten, but people will focus on the one time it was wrong. When this happens too often, it may be time to start with a new report suite and make sure that anything added to this new suite is validated and trusted. This can help you create a new perception and help you re-build the trust that is so essential to web analytics.
Final Thoughts
As you can see, there are many things to consider when it comes to re-implementation and report suites. The current state of your implementation and its data will be the biggest decision points, but every situation is different. Hopefully this helps provide a framework for making the decision and allows you to weigh the pros and cons of each approach.
