As someone who has made a living auditing, fixing and improving Adobe Analytics implementations, there are a few questions related to this that I receive all of the time. One of these questions, is whether a company doing a cleanup/re-implementation of their Adobe Analytics implementation should make changes to their existing report suite(s) or start over with brand new report suites. As you would expect, there is no one right answer to this, but I thought I would share some of the things I consider when making this decision.

Auditing What You Have

The first step of the cleanup process for an Adobe Analytics implementation is doing a thorough audit or review of what you have today. I can’t tell you how many companies I have worked with who hired me to do a “quick” review of their implementation and had no idea how inaccurate or bad it really was in its current state. Too often, I see companies that don’t question the data they are collecting and assume it is useful and that the data is correct. Trust me when I tell you that, more often than not, it isn’t!  I’d say that most of the companies I start with have scores around 60% – 70% out of 100% when it comes to their Success Events, eVars and sProps functioning properly. If you are unsure, I suggest you read this white paper that I wrote in partnership with Adobe (and if you think you may have some issues or want an objective validation that your implementation is in good shape, check out this page which describes my service in this area).

As you audit your existing implementation, you will want to make sure to look at the following:

1. How many report suites do you have and which are actively being used (and which can be hidden in the Admin Console)?  How consistent are these report suites?  If you select all of them in the Admin Console, how often do you see “Multiple” values?  The more you do, the more you may be in trouble.
2. How good is your data? If you have incomplete or inaccurate data, what is the value of sticking with your old report suites that are filled wth garbage?
3. How important to your business is year over year data? Some organizations live and die by YoY data, while others focus more on recent periods, especially if they have recently undergone a website re-design.

The answers so these types of questions will have impacts on the ultimate decision as I will dive into below.

Old Report Suites or New Ones?

So getting back to the original question – if you are going to re-implement, should you use existing report suites or new ones? The only way I can explain this is to show you the scenarios that I see most often.

Report Suite Inconsistency

If in the audit process above, you find that you have lots of report suites, and that the variable assignments in each of them are pretty different, the odds are that you should start from scratch with new report suites and, this time, make sure that they are set-up consistently. As mentioned above, the easiest way to see this is to select all of your active suites, choose a variable type (i.e. Events, eVars or sProps) and view the settings like this:

If you see this when selecting multiple report suites, the odds are you are in trouble, unless you manage lots of different websites that have absolutely nothing in common across them. I tend to see this issue most in the following situations:

1. Different sites are implemented by different parts of the business and no communication exists between them or there is no centralized analytics “Center of Excellence”
2. One business is acquired and both used the same analytics tool, but the implementations were done while the companies were separate
3. Two businesses or business units with different websites/products had implemented separately and, only later, decide they want to combine their data
4. A mobile team goes rogue and creates new report suites and tags all mobile sites/apps with a brand new set of variables without talking to the desktop group

Regardless of how you got there, the reason report suite inconsistency is so bad, is that salvaging it requires a massive variable reconciliation if you want to use your existing report suites and, even then, all but one suite is going to have different data in it than it did in the past. For example, let’s say that event 1 above is “Internal Searches” for two report suites and for the eight other suites has a different definition.  That means you have nine different definitions for event 1 across ten report suites. Even if you lay down the law and say that after your re-implementation, event 1 will always be “Internal Searches,” you will still have numbers that are not Internal Searches in eight out of 10 of your report suites historically. Thus, if someone looks at one of the suites that historically didn’t have event 1 as Internal Searches, for a long period of time, it will not actually be Internal Search data. Personally, I’d rather have no historical data than potentially misleading data. In addition, I think it is easier to start anew and tell the developers from all of your disparate sites that they must move their data from wherever they have it now to the new standard variable assignment list, rather than trying to map existing data to new variables using Processing Rules, DTM, VISTA Rules or other work-arounds.  Doing the latter just creates Band-Aids that will eventually break and corrupt your data once again in the future.

Here is an example of the eVars for one of my clients:

In this case, I only compared five different report suites out of more than fifty that they have in total. As you can see, reconciling this to ultimately have a global report suite or to send all data into one suite would be quite a challenge!

Conversely, if it turns out that all of your suites have pretty good variable definition consistency, then you can move the data from the incorrect variable slots to the correct ones in your lower priority report suites and continue using your existing report suites. For example, if you have one main report suite and then a bunch of other less significant report suites (like micro-sites), you may decide that you want to keep the main suite the way it is and force all of the other suites to change and adhere to the variable definitions of the main suite. This is a totally acceptable solution and will allow you to have year over year data for your main suite at least.

However, if you go down this path, I would suggest that if there are variables that the non-main report suite uses, that they be added to the main report suite in new variable slots so that eventually all of your report suites are consistent. For example, let’s say that one of the lesser important suites has a success event #1 that is registrations, but there is no registrations event in the main suite that you want to persist. In this case, you should move event 1 in the non-main suite to the next available event number (say event 51) and then add this event 51 to the main report suite as well. If you will never have registrations in your main suite, it is still ok to label event 51 as registrations, simply disable it in the admin console. This way, you avoid any variable definition conflicts in the future. For example, if the main report suite needs a new success event they would use event 52 instead of event 51 since they now know that event 51 is taken elsewhere. The only time this gets tricky is when it comes to eVars, since they are limited to 100 for most customers, but conserving eVars is a topic for another day!

Regardless of what you find, looking at the consistency of your variable definitions is an important first step in the process.

Data Quality

As mentioned above, if the data quality for your existing implementation isn’t very good, then there are fewer reasons to not start fresh. Therefore, another step I take in the process is to determine what percent of my Success Events, eVars and sProps in my current implementation I trust. As described in this old post, data quality is paramount when it comes to digital analytics and if you aren’t willing to put your name on the line for your data, then it might as well be wrong. Even if your report suites are pretty consistent (per the previous section), there may be benefits to starting over with new report suites if you feel that the data you have is wrong or could be misleading.

When I joined Salesforce.com many years ago to manage their Omniture implementation, I had very little faith in the data that existed at the time. When doing my QA checks, it seemed like most metrics came with a list of asterisks associated with them, such that presentation slides looked like bibliographies! While I hated the idea of starting over, I decided to do it because it was the lesser of two evils. It caused some temporary pain, but in the end, helped us shed a lot of baggage and move forward in a positive direction (you can read a lot more about how we did this in this white paper). For this reason, I suggest you make data quality one of your deciding factors in the new vs. old report suite decision.

Year over Year Data

Finally, there is the issue of year over year data. As mentioned above, if your variable definitions are completely inconsistent and/or your data quality is terrible, you may not have many options other than starting with new suites for some or all of your report suites. Moreover, if your data quality is poor, having year over year flawed data isn’t much of an improvement over having no year over year data in my opinion. The only real data that you would lose if your data quality is bad is Page Views, Visits and Unique Visitors (which are pretty hard to mess up!). In most cases, I try to avoid having year over year data be the driving force in this decision. It is a factor, but I feel that the previous two items are much more important.

Sometimes, I advise my clients to use Adobe ReportBuilder as a workaround to year over year data issues. If you decide to move to new report suites, you can build an Excel report using ReportBuilder that combines two separate data blocks into one large Excel data table that can be graphed continuously. In this case, one data block contains data for a variable in the old report suite and the other data block contains data for the same data point in a different variable slot in the new report suite. But to an end user of the Excel sheet, all they see is one large table that updates when they refresh the spreadsheet.

For example, let’s imagine that you have two report suites and one has internal searches in event 1 and the other has internal searches in event 5. Then you decide to create a brand new suite that puts all internal searches into event 1 as of January 1st. In ReportBuilder (Excel), you can create one data block that has event 1 data for suite #1 for dates prior to January 1st, another data block that has event 5 data for suite #2 for dates prior to January 1st and a third data block that has event 1 data for January 1st and beyond in the new report suite. Then you simply use a formula to add the data from event1 and event 5 in the data blocks that precede January 1st and then put that data block directly next to the final data block that contains event 1 data starting January first (in the new suite). The result will be a multi-year view of internal search data that spans all three report suites. A year later, your new report suite will have its own year over year data in the new combined event 1, so eventually, you can abandon the Excel workaround and just use normal Adobe Analytics reporting to see year over year data.

While this approach may take some work, it is a reasonable workaround for the few major data points that you need to report on year over year while you are making this transition to a newer, cleaner Adobe Analytics implementation.

Justification

Sometimes, telling your boss or co-workers that that you need to re-implement or start over with new report suites can be a difficult thing to do.  In many respects, it is like admitting failure. However, if you do your homework as described above, you should have few issues justifying what you are doing as a good long-term strategy. My advice is to document your findings above and share them with those who may complain about the initiative. There will always be people who complain, but at the end of the day, you need to instill faith in your analytics data and if this is a necessary step, I suggest you do it. I heave learned over the years that the perception of your organization’s analytics team is one of the most critical things and something you should safeguard as much as you can. In our line of work, you are asking people to make changes to websites and mobile apps, partly based upon the data you are providing. That demands a high level of trust and once that trust is broken it is difficult to repair.

I also find that many folks I work with were not around when the existing implementation was done. This is mainly due to the high turnover rate in our industry, which, in turn, is due to the high demand for our skills. If you are new to the analytics implementation that you now manage, I recommend that you perform an audit to make sure what you inherited is in good shape. As described in books like this, you have a narrow window of time that is ideal for cleaning things up and asking for money if needed. But if you wait too long, the current implementation soon becomes “your problem” and then it is harder to ask for money to do a thorough review or make wholesale changes. Plus, when you first start, you can say things like “You guys were the ones who screwed this up, so don’t complain to me if we have to start it over and lose YoY data…You should have thought of that before you implemented it incorrectly or with shoddy data quality!” (You can choose to make that sound less antagonistic if you’d like!)

Avoiding a Repeat in the Future

One last point related to this topic. If you are lucky enough to be able to clean things up, reconfigure your report suites and improve your data quality, please make sure that you don’t ever have to do that again! As you will see, it is a lot of work. Therefore, afterwards, you want to put processes in place to ensure you don’t have to do it again in a few years! To do this, I suggest that you reduce the number of Adobe Analytics Administrators to only 1-2 people, even if you are part of a large organization. Adding new variables to your implementation should be a somewhat rare occurrence and by limiting administration access to a select few, you can be sure that all new variables are added to the correct variable slots. I recommend doing this through a digital analytics “center of excellence,” the setup of which is another one of the services that I provide for my clients. As they say, “an ounce of prevention is worth a pound of cure!”

Back in 2010, I wrote about a way to see how much money website visitors were adding to the shopping cart so that amount could be compared to the amount that was actually purchased. The post also showed how you could see this “money left on the table” by product and product category. Recently, however, I had a client ask a similar question, but one focused on whether the product price was possibly a barrier to cart conversion. Specifically, the question was asked whether visitors who add products to the cart that are between $50 and $100 end up purchasing more or less than those adding products valued at a different price range. While there are some ways to get to this information using the implementation approach I blogged about in the preceding post, in this post, I will share a more straightforward way to answer this question.

Capturing Add to Cart Value

In the preceding post, to see money left on the table, I suggested that the amount of the product being added to the cart be passed to a new currency Success Event. But to answer our new question, you will want to augment that by passing the dollar amount to a Product Syntax Merchandising eVar when visitors add a product to the cart. For example, if a visitor adds product # 111 to the cart and its price is $100, the syntax would look like this:

s.events="scAdd,event10";

s.products=";111;;;event10=100;evar30=100";

In this case, the $100 cart value is being passed to both the currency Success Event and the Merchandising eVar (I suggest rounding the dollar amount to the nearest dollar to minimize SAINT Classifications later). Both of these amounts are “bound” to the product number (111 in this example).

Once this is done and repeated for all of your visitors, you can use the new Merchandising eVar to see Cart Additions by price of item added to cart using a report like this:

Since the new Merchandising eVar has been bound to the product added to the shopping cart, if the visitor purchases the product prior to the eVar’s expiration (normally purchase event), the eVar value will be applied to the purchase event as well for those products ultimately purchased. Therefore, when orders and revenue are set on the order summary page, each order will be “bound” to the product price value such that you can see a report that looks like this:

Using this report, you can see how each price point performs with respect to cart to order conversion. Since you will have many price points, you will likely want to use SAINT Classifications to group your price points into larger buckets or ranges to make the data more readable:

Once you have this, you can switch to the trended view of the report and see how each price range converts over time. Of course, you can break this down by product or external campaign code to see what factors result in the conversion rate being higher or lower than your standard cart conversion rate (Orders/Cart Additions). This analysis can be used in conjunction with my competitor pricing concept to see which products you should emphasize and de-emphasize on your online store. You can also use this new eVar in segments if you ever want to isolate cases in which a specific product price range was added to the cart or purchased.

As you can see, there are lots of handy uses for this implementation concept, so if you have a shopping cart, you may want to try it out and see what creative ways you can exploit it to further your analysis capabilities.

A few years ago, I wrote an extensive post describing how to track internal search position clicks to see which internal search positions visitors tend to click on. That post showed how to track impressions and clicks for internal search positions and how to view this by search phrase. Recently, however, I had a client ask for something tangentially related to this. This client was interested in seeing the overall average search position clicked when visitors search on their website and for each search term. While the preceding post provides a way to see the distribution of clicks on internal search spots, it didn’t provide a straightforward way to calculate the overall average. Therefore in this post, I will share a way to do this for those who want to see a trended view of how far down the search results list your visitors are going.

Calculating the Average Internal Search Position Clicked

The key difference in calculating the average internal search position clicked from what I described in my previous post, is that you need to switch from using a dimension (eVar) to using a metric (Success Event). To compute the average search position, the formula we eventually need is one that divides the summation of the position numbers clicked by the number of total internal searches. For example, if I conduct a search and click on the 10th result and then another search and click on the 5th result, I have clicked on an aggregate of 15 internal search positions (10+5) and had 2 search clicks. When I divide these two elements, I can see that my average search position clicked is 7.5 (15/2). Hence, if you apply the same approach for all of your visitors, you will be able to calculate the overall average internal search position.

From an implementation standpoint, this is relatively easy. If you have internal search on your site, you are probably already setting a metric (Success Event) on the search results page to determine how often searches are taking place. If you followed my advice in this post, you would also be setting a second metric when visitors click on a result in your search result list. Therefore, the only piece you are missing is a metric that quantifies the position number clicked. To do this in Adobe Analytics you would set a new Numeric (or Counter if on latest code) Success Event with the number value of the position clicked (let’s call this Search Position). For example, if a visitor conducts a search and clicks on the 5th position, you would pass a value of “5” to the new success event. This will create the numerator needed to calculate the average search position.

Once you have done this, you have the two metrics you need to calculate the average – search position numbers and the number of search clicks. Simply create a new Calculated Metric that divides the Search Position by the # of Search Clicks to compute the average as shown here:

This will produce a metric repost like this:

Average Search Position by Search Phrase

Since you are most likely already capturing the search phrase when visitors search, you can also view this new Calculated Metric by search phases, by simply adding it to the dimension (eVar) report:

This report and the preceding ones can be used to watch your search result clicks overall and by search phrase. This may help you determine if your search results are meeting the needs of your users and whether you even need to have pages and pages of search results.

One fun way I have used this type of analysis is to take my top search phrases and hand-pick specific links that I want them to go to for the top search phrases (recommended links). Then you can see if your users prefer the organic results or the ones you have picked for them (using a new eVar!). Another way to use this analysis is to see if changes made to your internal search makes the average search position clicked go up or down. Regardless of how you use it, if you are going to use internal search on your site, you may as well track it appropriately. Enjoy!

With the latest release of Adobe Analytics, the Analysis Workspace interface now provides a way to conduct cohort analyses. The new Cohort Analysis borrows from an existing one that Adobe had previously made available for mobile implementations, but now it is available for use everywhere and with everything that you have in your Adobe Analytics implementation. In this post, I will provide a quick “how to” since I have been surprised by how few of my Adobe customers are aware of this new functionality.

Cohort Analysis Revisited

A cohort analysis is used when you want to isolate a specific event and then see how often the same folks completing that event go on to complete a future event. In the recent decade, cohort analyses became popular due to social networking tools when they were used to judge the “stickiness” of these new tools.  For example, in the early days of Twitter, people would look to see how often users who tweeted in January were still tweeting in February. In this case, the number of people who tweeted in February was a separate number from those who tweeted in January and then in February, with the latter being “cohorts.”  For more information on this topic, check out the wikipedia page here.

New Cohort Analysis Visualization

Once you are comfortable with cohort analysis as a concept, let’s look at how you can create cohort analyses in the new Adobe Analytics interface. To start, use the left navigation to access the Analysis Workspace feature of the product (note that if you are on an older version of Adobe Analytics, you may not have Analysis Workspace enabled):

In the Analysis Workspace area, you will click the visualizations tab to see all of the potential visualizations:

From here, you will drag the “Cohort Table” visualization over to your reporting canvas and should see this:

At this point, you need to select your timeframe/granularity (i.e. Month, Week, Day) using the drop-down box and then drag over the metric you want visitors to have performed to be included in the cohort. This is done by clicking on the components tab at the top-left:

Keep in mind that you cannot use calculated metrics and some other out-of-box metrics as inclusion metrics, but you can use any of your raw success events. Also, if you click on the “Metrics” link, you can see all metrics and do a search filter, which is very handy. When contemplating your inclusion metric, think about what actions you want visitors to take to be included in the cohort. For example, if you are looking for people who have ordered, you would use the Orders metric, but if you are interested in people who have viewed content on your site, you may use a Content Views success event. As an example, let’s use the latter and build a cohort of visitors who have viewed content on the site and see how many of those visitors come back to the website within x number of days. To do this, we would change the granularity to days, add a Content Views metric as the inclusion metric and then add the Visits metric as the return metric so the cohort analysis looks like this:

You may also notice that you have the option to increase the number of times each metric has to occur before people would be be added to the inclusion or return portion of the cohort. By default the number is one, meaning that the above cohort is looking for cases in which one or more Content Views took place and then one or more return Visits took place. To narrow down the cohort, we could easily increase these numbers to force visitors to have viewed more content to be included in the cohort or returned to the site more than once to be included. But in this example, we’ll keep these set to one and run the report to see this:

Here we can see that on November 3rd, we had 6,964 unique visitors who had Content Views and that of those who viewed content on that day, 13% (892 Visitors) returned to the site within one day (had a return Visit). Keep in mind that all numbers shown in cohort analyses are unique visitor counts. The color shading shows the intensity of the cohort relative to the other cohort cells. By looking horizontally, you can see the drop-off by day for each cohort starting date and as you look vertically, the days will follow a cascading pattern with the newest starting dates having the fewest return dates like this:

Changing the granularity from Day to Week, would work the same way, but have far fewer cohorts unless you extend your timeframe:

Here is an example in which I have made both the inclusion and return metric the same thing (Content Views), but made viewing two pieces of content required to be eligible for the return cohort:

Here you will notice that requiring two return content views reduced the first (Nov 3rd) cohort from 13% down to 9%. You can use these settings to identify interesting patterns. Since you can also make as many cohorts as you want using all of your success events, the amount of information you can glean is enormous.

Putting Cohorts To Use

Once you learn how to generate cohort analyses, you may ask yourself “Ok, now what do I do with these?” That is a valid question. While a blog post isn’t the best venue for sharing all you can do with cohort analyses, let me share a couple ways I would suggest you use them. The first way is to apply segments to your cohorts. For example, you may want to determine if visitors from a specific region perform better than another, or if those using your responsive design pages are more likely to return. Here is an example in which the previous cohort is segmented for Microsoft browsers to see if that makes the cohort better or worse:

In this case, our Nov 3rd cohort went from 13% to 8% just based upon browser. Since you probably have many segments, this provides more ways you can slice and dice these cohorts and adding a segment is as easy as dropping it into the top of your Analysis Workspace page like this:

Keep in mind that any segment you apply will be applied to both the inclusion and return criteria. So in the preceding scenario, by adding a Microsoft Browser segment, the inclusion visitor count only includes those visitors who had a Content View event and used a Microsoft browser and the return visits also had to be from a Microsoft browser.

But my favorite use for cohorts is using a semi-hidden feature in the report. If you have a particular cohort cell (or multiple cells) that you are interested in, you can right-click on it and create a brand new segment just for that cohort! For example, let’s say we look at our original content to return visit cohort:

Now, let’s say something looks suspicious about the Nov 3rd – Day 4 cohort, which is at 3% (top-right cell). We can right-click on it to see this:

Then clicking will show us the following pre-defined segment in the segment builder:

Now you can name and save this segment and use it in any analysis that you may need in the future!  You can also make changes to it if you desire before saving.

While there is much more you can do with cohorts, this should be enough for you to get started and begin playing around with them. Enjoy!

Over the past year, Adobe Analytics users have noticed that the product has moved to a different model for accessing/editing/creating analytics components such as Calculated Metrics, Segments, etc… In this post, I want to touch upon one aspect that has changed a bit – the sharing of calculated metrics.

Sharing Calculated Metrics – The Old Way

In the older interface of Adobe Analytics (pre version 15.x), it was common to create a calculated metric, then select multiple report suites and apply that calculated metric to multiple suites. For example, if you wanted to create a Null Search ratio, you would create the formula and then select your report suites and save it. Here is an example in which a few calculated metrics have been applied to thirteen report suites:

This approach would save you the work of creating the metric thirteen separate times, which could be a real pain, especially if you had hundreds of report suites.

However, employing this [old] preferred approach of sharing calculated metrics can actually make things a bit confusing when you switch over to the new version of Adobe Analytics. When using the new Calculated Metrics manager, the old approach will  cause you to see the same calculated metric multiple times, since it shows all calculated metrics for all report suites in the same window. Here is how the same calculated metric looks in the more updated version:

In this case, you would see the same metric for as many report suites as it was associated with in your implementation. While you could keep all of these different versions, doing so presents the following potential risks:

1. It can be confusing to novice end-users
2. If someone makes a change to one of the calculated metrics (in one report suite), it can deviate from the others, so that you lose integrity of the metric across your implementation/organization
3. If you want to make a change to a calculated metric in the future, you have to do it multiple times

In addition to these risks, in the newest version of Adobe Analytics, there are some cool new ways to share metrics that don’t require this duplication of the same metric hundreds of times.

Sharing Calculated Metrics – The New Way

If you were to make a new calculated metric now, using the latest version of Adobe Analytics, you could create the metric once and simply share it to all users or groups of users. Once you have created your metric, you use the share feature and select “All” as shown here:

Doing this allows you to see the calculated metric in every report suite, without having multiple versions of it. As shown here, you will still see the calculated metric when you click “Show Metrics” from within the Adobe Analytics interface:

Therefore, if you have twenty calculated metrics across fifty report suites, you would have twenty rows in your calculated metric manager instead of one thousand! This makes your life as an administrator much easier in the future.

Moving From The Old to the New

So what if you already have a lot of metrics and they are shown multiple times in your calculated metrics manager? If you decide you want to trim things down and go to the newer approach, you would want to do the following:

1. I suggest creating a corporate login as outlined in this blog post. This is a centralized admin login that the core analytics team maintains
2. Review all of your shared bookmarks and dashboards to find all cases in which calculated metrics you are about to remove are used
3. Copy each of the existing calculated metrics using the corporate login ID (described in step 1) and share it across all or designated users
4. Once this is done, you can delete all of the duplicate versions of the calculated metric
5. Go back to the shared bookmarks and dashboards using the old version of calculated metrics and replace them with the newly created shared version

While this may take some time, it will free up time in the future, since it will minimize the number of calculated metrics you have to maintain in the long run. I also find that it is beneficial to periodically review all of your shared reports and calculated metrics and do a clean-up. This process forces you to do this and you may be amazed how many you have, how many you can remove and how many are wrong!

When using Adobe Analytics, one of the things that has been a bit annoying historically is that when you sort by a calculated metric, you often see really high percentages for rows that have very little data. For example, if you create a click-through rate metric or a bounce rate metric and sort by it, you may see items of 100% float to the top, but when looking at the raw instances, the volume is so low that it is insignificant. Here is an example of a case where you may be capturing onsite searches by term (search criteria) and clicks on search results for the same term (as outlined in this post):

In this case, it is interesting to see that certain terms have a highly disproportionate number of clicks per search, but if each is searched only once, it isn’t that statistically relevant. To get around this, Adobe Analytics customers have had to export all data to Microsoft Excel, re-create the calculated metrics, delete all rows with fewer than x items (searches in this case) and then sorted by the click-through rate. What a pain! That is a lot of extra steps!

But now, thanks to the new Derived Metrics feature of Adobe Analytics, this is no longer required. It is now possible to use more complex functions and formulas to narrow down your data such that you can sort by a calculated metric and only see the cases where you have a higher volume of instances. In this post, I will demonstrate exactly how this is done.

Using the Percentile Function

The key to sorting on a calculated metric is the use of the new PERCENTILE function in Adobe Analytics. This function allows you to choose a percentile within a list of values and use that in a formula. To illustrate this, I will continue the onsite search example from above. While the click-through rate formula used above is accurate, we want to create a report that only shows the click-through rate when search criteria have at least x number of searches. However, since the number of unique search criteria will vary greatly, we cannot simply pick a number (like 50 or more), because we don’t know how many searches will be performed for the chosen date range. For example, one of your users may choose one day, in which greater than 5o searches is unlikely, but another user may choose a full year, which will make greater than 50 a huge number of items with a very long tail. To deal with all scenarios, you can use the PERCENTILE function, which will look at all of the rows for the selected date range and allow you to calculate the xth percentile of that list. Hence, the threshold is relative to the date range chosen with respect to the number of instances that have to take place for it to show up in the calculated metric. Since this can be a bit confusing, let’s look at an example:

To start, you can build a new calculated metric that shows you what the PERCENTILE formula will return at a specific percentile. To do this, open the Calculated Metrics builder and make the following formula:

Since there may be a LOT of unique values for search criteria, I am starting off with a very high percentile (99.5%) to see how many searches it takes to be in the 99.5% percentile. This is done by selecting the core metric (Searches in this case) and then making the “k” value 99.5 (Note: You can also figure out the correct “k” value by using the PERCENTILE function in Microsoft Excel with the same data if you find that easier). Once you are done, save this formula and add it to your search criteria report so you see this:

This formula will have the same value for every row, but this is ok since we are only using it temporarily to figure out if 99.5% is the right value. In this case, what we see is that at a 99.5% percentile, anything with over 18 searches will show us the search click-through rate and anything below 18 searches will not. Now it is up to you to make your judgement call. Is 18 too high in this case?  Too low? If you want to raise, it, simply raise the “k” value in the formula to 99.8 or something similar.

While doing this, keep in mind, that changing your date range (say choosing just one day), will change the results as well. The above report is for 30 days of data, but look what happens when we change this to just one day of data:

As you can see, the threshold changed from 18 to 6, but the number of overall searches also went down, so the 99.5% percentile seems to be doing its job!

Once you have determine what your ideal percentile “k” value is, it is now time to use this formula in the overall click-through rate formula. To do this, you need to create an IF statement and use a GREATER THAN function as well. The goal is to tell Adobe Analytics that you want it to show you the search click-through rate only in cases where the number of searches is greater than the 99.5% percentile. In other cases, you want to set the value to “0” so that when you sort in descending order, you don’t have crazy percentages showing up at the top, even though there are low values. Here is what the formula will look like:

While this may look a bit intimidating at first, if you look at its individual components, all it is really doing is calculating a click-through rate only when the number of searches is above our chosen threshold. Now you can add this metric to your report and see the results:

As you can see, this new calculated metric is no different from the existing one in cases where the number of searches is greater than the 99.5% threshold. But look what happens when we sort by this new Weighted Click-Through Rate metric:

Unlike the report shown at the beginning of this post, we don’t see super high percentages for items with low numbers of searches. All of the results are above the threshold, which makes this report much more actionable. If you want, you can verify this by paging down to the point where our new weighted metric is 0% (in this case when searches are under 18):

Here you can see that searches are less than 18 and that the previous click-through rate metric is still calculating, but our new metric has hard-coded these values to “0” for sorting purposes.

Final Thoughts

As you can see, using the PERCENTILE function can be a real time-saver. While this example is related to onsite search, the same concept can be applied to any calculated metric you have in your Adobe Analytics implementation. In fact, Adobe has created a similar metric for Weighted Bounce Rate that is publicly available for all customers to use for marketing campaigns. So any time you want to sort by a calculated metric and not see rows with low numbers of data, consider using this technique.

Product Finding Methods, is a topic that I have touched upon briefly in past blog posts, but not in great detail. Some others have also talked about it, but in a quick Google search, the most relevant post I found on the subject was this post from back in 2008. Therefore, in this post, I thought I would explore the topic and how it can be applied to both eCommerce and non-eCommerce sites.

What is Product Finding Methods?

I define Product Finding Methods as the way that website/app visitors use to find products that they ultimately purchase. For example, if a visitor comes to your website and conducts a search and then finds a product they like, then search would be the Product Finding Method that should be associated with that product. Most websites have about 5-10 different Product Finding Methods, such as:

• Internal Search
• Navigation/Browsing
• Internal Campaigns/Promos
• Wishlist/Favorites/Registries
• Collections
• Cross-Sell
• Campaign Landing Pages

These are usually the main tools that you use to drive visitors to products, with the goal of getting them to add items to their online shopping cart. Having a Product Finding Methods report is useful when you want to see a holistic view of how visitors are getting to your products, but it can also be used to see how each product or product category is found. In most cases, the KPI that you care about for Product Finding Methods is Orders or Revenue because, while it may be interesting to see which methods get visitors to add items to the shopping cart, you make money when they order products and pay you!

Why Implement Product Finding Methods?

So why should you care about Product Finding Methods? Most organizations implementing this do so to identify which method is most successful at driving revenue. Since websites can be tweaked to push visitors to one finding method over another, if you have one that works better than another, you can work with your design team to either fix the lagging one or push people to the better one. For example, if your internal search functionality rarely produces orders, there may be something inherently wrong with it. Even if you track the internal search click-through rate and that is ok, without the Product Finding Methods report, you may not know that those clicking on results are not ultimately buying. The same may be true for your internal promotions and other methods. I once had a client that devoted large swaths of their product pages to product cross-sell, but never had a Product Finding Methods report to show them that cross-sell wasn’t working and that they were just wasting space.

Another use of Product Finding Methods is to see if there are specific products or product categories that lend themselves to specific finding methods. For example, you may have products that are “impulse” buys that do very well when spotlighted in a promotion on the home page, but don’t do so well when found in search results (or vice versa). Having this information allows you to be more strategic in what you display in internal promotions. By creating a “Look to Book” Calculated Metric (Orders/Product Views) within a Product Finding Methods report, it is easy to see which products do well/poorly for each finding method.

Finally, once you have implemented Product Finding Methods, you can use them in Segments. If you have a need to see all visits or visitors who have used both Internal Search and a Registry to find products, you can do this easily by selecting those two methods in the Segment builder. Without Product Finding Methods being implemented, creating a viable segment would be very cumbersome, likely involving  a massive amount of Page-based containers.

How Do You Implement Product Finding Methods?

So let’s say that you are intrigued and want to see how visitors are finding your products. How would you implement this in Adobe Analytics?

Obviously, if you are looking to breakdown Orders and Revenue (which are Success Events) by a dimension, you are going to need a Conversion Variable (eVar). This eVar would capture the most recent Product Finding Method that the user interacted with, regardless of whether that Finding Method led directly or indirectly to a product. To do this, you would work with your developers to identify all of your Product Finding Methods and determine when each should be passed to the Product Finding Methods eVar. For example, the “Internal Search” product finding method would always be set on the search results page.

However, before we go any further, we need to talk about Product Merchandising. If you are not familiar with the Product Merchandising feature of Adobe Analytics, I suggest that you read my post on that now. So why can we not use a traditional eVar to capture the Product Finding Method? The following example will illustrate why. Imagine that Joe visits our site and clicks on a home page promotional campaign and you have your developer set the Product Finding Methods eVar with a value of “Internal Campaign.” Then Joe finds a great product and adds it to the shopping cart. Next Joe does a search on our site, so you have your developer set the Product Finding Methods eVar with a value of “Internal Search.” Joe proceeds to add another product to the shopping cart and then checks out and purchases both products. In this scenario, which Product Finding Method will get credit for each product? If you said “Internal Search” you would be correct because it was the “most recent” value prior to the Purchase Success Event firing. Unfortunately, that is not correct. In this scenario, product #1 should have “Internal Campaign” as its finding method and product #2 should have “Internal Search” as the finding method. Because we need each product to have its own eVar value, we need to use the Product Merchandising feature, which allows us to do that.

For Product Finding Methods, it is common to use the “Conversion Syntax” methodology of Product Merchandising since we often don’t know when the visitor will ultimately get to a product or which product it will be. By using the “Conversion Syntax” method, we can simply set the value when the Product Finding Method occurs and have it persist until the visitor engages in some action that tells us which product they are interested in (the “binding” Success Events). Normally, I recommend that you “bind” (or associate the Product Finding Method with the product) when visitors view the Product Detail Page, perform a Cart Addition, engage with a Product Quick View or other similar product-related actions. These can be configured in the Administration Console as needed.

Once you have navigated the murky waters of Product Merchandising, set-up your Product Finding Methods eVar and worked with your developers to pass the values to the eVar, you will see a report that looks something like this:

From this report, you can then use breakdowns to breakdown each Product Finding Method by Product ID:

If you have SAINT Classifications setup for your Products Variable, you can see the same reports for things like Product Category, Product Name, Product Type, etc… All of these reports are bi-directional, so if you wanted to see the most popular Product Finding Methods for a specific product, all you would do is open the Products report and then break it down by the Product Finding Method eVar.

What About Deep-Links?

One question you may be asking yourself is: “What happens if visitors deep-link directly to a product page on my website and there is no Product Finding Method?” Great question! If you don’t account for this scenario, you would see a “None” row in the Product Finding Methods eVar report for those situations. In that case, the “None” row can be explained as “No Product Finding Method.” But one tip that I will share with you is that you can set a default value of the referring marketing channel in your Product Finding Methods eVar on the first page of the visit. If you can identify the marketing channel (Paid Search, SEO, E-mail, etc…) from which visitors arrive to your site, you can pass that channel to the Product Finding Methods eVar when the visit begins. Doing so will establish a default value so that if a product “binding” event takes place before any of your onsite product finding methods are activated, those products will be bound to your external marketing channel. This gives you more information than the “None” row, but still allows you to quantify what percent of your products have an internal vs. external product finding method. Obviously, to use this tip, you have to be able to identify the external marketing channel of each visit so that it can be passed to the eVar. I tend to do this by some basic rules in JavaScript that analyze the referrer and any campaign tracking codes I am already using. You can see a version of this by looking at the first report above in row five labeled “external campaign referral,” and notice that no “None” row exists in that report.

Non-eCommerce Uses

So what if you don’t sell products on your website? Does that mean you cannot use Product Finding Methods? Of course not! Even if you don’t sell stuff, there are likely uses for the above implementation. For example, if you manage a research site, you probably have visitors looking for content. In that case, your content is your product and you may be storing your content ID’s in the Products variable. This means that you can capture the different methods that your visitors use to find your content and assign each Content ID with the correct Product Finding Method.

If you manage a B2B website, you have product pages, but you may not sell the actual product online. In this case, the implementation for eCommerce will work the same way, but instead of Orders and Revenue, you may make the final Success Event Leads Completed. You can also see how visitors find your product videos, pricing sheets, etc…

Similar approaches can be employed for non-profit sites, government sites and so on. If you work with a non-eCommerce site, you may just have to think a bit more creatively about what your finding methods might be, what your “products” are and which binding events make sense. As long as you understand the general concept: Figuring out how visitors make their way to the stuff you care about, you will be able to find a way to use Product Finding Methods in your implementation.

If you have questions or other approaches related to this topic, feel free to leave a comment here or ping me at @adamgreco. Thanks!

In the last year, Adobe Analytics introduced the ability to have over 1,000 Success Events. That was a pretty big jump from 100 previously. As I work with clients I see some who struggle with what having this many Success Events really means. Does it mean you can now track more stuff? At a more granular level? Should you track more? Etc… Therefore, in this post, I am going to share some of my thoughts and opinions on what having 1,000 Success Events means and doesn’t mean for your Adobe Analytics implementation.

Knee Jerk Reaction

For most companies, I am seeing what I call a “knee jerk reaction” when it comes to all of the new Success Events. This reaction is to immediately track more things. But as my partner Tim Wilson blogged about, more is not necessarily better. Just because you can track something, doesn’t mean you should. But let’s take a step back and consider why Adobe enabled more Success Events. While I cannot be 100% certain, since I am not an Adobe Analytics Product Manager, it is my hunch that the additional Success Events were added for the following reasons:

1. It is easier to increase Success Events than other variables (like eVars) due to the processing that happens behind the scenes
2. Success Events are key to Data Connector integrations and more clients are connecting more non web-analytics data into the Adobe Marketing Cloud
3. Some of the Adobe product-to-product integrations use additional Success Events
4. Having more Success Events allows you to push more metrics into the Adobe Marketing Cloud

I don’t think that Adobe was saying to itself, “our clients can now only track 100 metrics related to their websites/apps and they need to be able to track up to 1,000.”

This gets to my first big point related to the 1,000 Success Events. I don’t think that companies should track additional things in Adobe Analytics just because they have more Success Events. If the data you want to collect, has business benefit, then you should track it, but if you ever say to yourself, “we have 1,0000 Success Events, so why not use them?” there is a good chance you are going down a bad path. For example, if you have thirty links on a page and you want to know how often each link gets clicked, I would not advocate assigning a Success Event to each of the thirty links. If you wouldn’t do it when you had 100 Success Events, I would not suggest doing it just because you have more.

But there will be legitimate reasons to use these new Success Events. If your organization is doing many Data Connector integrations, the amount of Success Events required can grow rapidly. If you have a global organization with 300 different sites and each wants to have a set of Success Events that they can use for their own purposes, you may decide to allocate a set of Success Events to each site (though you can also double-up on Success Events and not send those to the Global report suite as well). In general, my advice is to not have a “knee jerk reaction” and change your implementation approach just because you have more Success Events.

Use Multiple Success Events vs. an eVar

Another thing that I have seen with my clients is the idea of replacing or augmenting eVars with multiple versions of Success Events. This is a bit complex, so let me try and illustrate this with an example. Imagine that one of your website KPI’s is Orders and that your stakeholder wants to see Orders by Product Category. In the past, you would set an Order (using the Purchase Event) and then use an eVar to break those Orders down by Product Category. But with 1,000 Success Events, it is theoretically possible for you to set a different Success Event for each Product Category. In this example, that would mean setting the Orders metric and at the same time setting a new custom Success Event named Electronics Orders (or Apparel Orders depending upon what is purchased). The latter would be a trend of all Electronics Orders and would not require using an eVar to generate a trend by Product Category. If you have fifty Product Categories, you could use fifty of your 1,000 Success Events to see fifty trends.

This raises the question, is doing what I just described a good thing or a bad thing? I am sure many different people will have different opinions on that. Here are the pros and cons of this approach from my point of view:

Pros

1. While I would not recommend removing the Product Category eVar, in theory, you could get rid of it since you have its core value represented in fifty separate Success Events. This could help companies that are running out of eVars, but still not something I would advocate because you can lose the great attribution benefits of eVars (especially across multiple visits).
2. Today, it is only possible to view one metric in a trended report, so if you want to see more than just Orders for a specific Product Category (say Orders, Revenue and Units for Electronics), you can’t do so using an eVar report. But if each of these metrics were tied to a separate Product Category event, you could use the Key Metrics report to get up to five metrics trended for a particular Product Category. But keep in mind that you would need fifty Success Events for each metric you want to see together, which can make this a bit un-scalable. Also keep in mind that you can trend as many metrics as you want using the Adobe ReportBuilder tool.
3. You can create some cool Calculated Metrics if you have all of these additional Success Events, such as Electronics Orders divided by (Electronics Orders + Apparel Orders) that may be more difficult to produce in Adobe Analytics proper without using Derived Metrics or Adobe ReportBuilder.
4. Having additional metrics allows you to have Participation enabled on each, which can provide more granular Participation analysis. For example, if you enable Participation on the Orders event, you can see which pages lead to Orders. But if you enable Participation on a new Electronics Orders event, you will be able to see which pages lead to orders of Electronics products. The latter is something that isn’t possible (easily) without having a separate Electronics Orders Success Event.
5. If you want to pass Adobe Analytics data to another back-end system using a Data Feed, there could be some advantage to having a different metric (in this example for each Product Category) vs. one metric and an eVar in terms of mapping data to an external database.

Cons

1. Setting so many Success Events can be a nightmare for your developers and a pain to maintain in the Administration Console. It may require extra time, logic, TMS data mappings and so on. In the preceding example, developers may have to write additional code to check for fifty product categories. In some cases, developers may only know the Product ID and not the category (which they had planned on being a SAINT Classification), but setting the additional Success Events forces them to write more code to get the product category. If visitors purchase products from multiple product categories, developers have to start defining rules that makes things more complex than originally anticipated. And if product categories change (new ones are added or old ones are removed), that can mean more development work vs. simply passing in different values to an eVar. While using a Tag Management System can make some of this easier, it still creates a lot more work for developers, who are normally already stretched to their limits!
2. Having different versions of the same Success Events can be confusing to your end-users (i.e. Orders vs. Electronics Orders) and can make your entire implementation a bit more confusing
3. Employing this approach too often can force you to eventually run out of Success Events, even with the increased number available. For example, if you set Orders, Revenue, Units and Cart Additions for each Product Category, you are already looking at 200 Success Events. Setting these same events for a different dimension (eVar), would require another 200 Success Events!

While I can see some merits in the benefits listed above, my opinion is that blowing out different Success Events instead of using an eVar is something that can have value in some targeted situations, but is not for everyone. Call me old fashioned, but I like having a finite number of metrics and dimensions (eVars) that break them down. If there is a short list of metrics that are super critical to be seen together in the Key Metrics report and the number of times they would have to be duplicated by dimension is relatively small, then perhaps I would consider adding an extra 10-20 Success Events for each dimension value. But I see this as a bit of a slippery slope and wouldn’t advocate going crazy with this concept. Perhaps my opinion will change in the future, but for now, this is where I land on the subject.

Derived Metrics

Tangentially related to this topic is the concept of Derived Metrics. Derived Metrics are the new version of Calculated Metrics in which you can add advanced formulas, functions and segments. The reason I bring these up is that Derived Metrics can be used to create multiple versions of metrics by segmenting on eVar or sProp values. For example, instead of creating fifty versions of the Orders metric as described above, you could have one Orders metric and then create fifty “Derived” metrics that use the Orders metric in combination with a segment based upon a Product Category eVar. This requires no extra development effort and can be done by any Adobe Analytics user. The end result would be similar to having fifty separate Success Events as each can be trended and up to five can be added to the Key Metrics report. The downsides of this approach is that these Derived Metrics will not be easily fed into a data feed if you want to send data directly to an external database and won’t have Participation metrics associated with them.

It is somewhat ironic that shortly after Adobe provided 1,000 Success Events, it also provided a great Derived Metrics tool that actually reduces the need for Success Events if used strategically with Segments! My advice would be to start with using the Derived Metrics and if you later find that you have reasons to need a native stream of data for each version of the event (i.e. Data Feed) or Participation, then you can hit up your developers and consider creating separate events.

Final Thoughts

So there you have some of my thoughts around the usage of 1,000 Success Events. While I think there can be some great use cases for taking advantage of this new functionality, I caution you to not let it change your approach to tracking what is valuable to your stakeholders. I am all for Adobe adding more variables (I wish the additional eVars didn’t cost more $$$!), but remember that everything should be driven by business requirements (to learn more about this check out my Adobe white paper: http://apps.enterprise.adobe.com/go/701a0000002IvLHAA0).

If you have a different opinion or approach, please leave a comment here.  Thanks!

Few features were more hotly anticipated following Adobe Summit than the arrival of the newly expanded calculated metrics to Adobe Analytics. Within one week of its release, it is already paying off big time for one of my clients. I’m going to share a use case for how these advanced calculated metrics fixed some pretty broken revenue data.

Our example for this case study is KittensSweaters.com*, an ecommerce business struggling with their Adobe Analytics data. Over the past few months, KittenSweaters has dealt with a number of issues with their revenue data, including:

• “Outlier” orders where the revenue recorded was grossly inflated or even negative
• A duplicate purchase event firing prior to the order confirmation page, that double counted revenue and
• Donations to their fundraiser counting as sweaters revenue, instead of in a separate event

For example, here you can see the huge outliers and negative revenue numbers they saw in their data:

Historically, this would require segmentation be layered upon all reports (and ensuring that all users knew to apply this segmentation before using the data!)

However, using the new Calculated Metrics in Adobe Analytics, KittenSweaters was able to create a corrected Revenue metric, and make it easily available to all users. Here’s how:

First, create a segment that is limited only to valid orders.

In the case of KittenSweaters, this segment only allows in orders where:

1. The product category was “sweaters”; and
2. The purchase was fired on the proper confirmation page; and
3. The order was not one of the known “outlier” orders (identified by the Purchase ID)

You can test this segment by applying it on the current Revenue report and seeing if it fixes the issues. Historically, this would have been our only route to fix the revenue issues – layer our segment on top of the data. However, this requires all users to know about, and remember to apply, the segment.

So let’s go a step further and create our Calculated Metric (Components > Manage Calculated Metrics.)

Let’s call our new metric “Revenue (Corrected)”. To do so, drag your new segment of “Valid Sweater Orders” into the Definition of your metric, then drag the Revenue metric inside of the segment container. Now, the calculated metric will only report on Revenue where it matches that segment.

Voila! A quick “Share” and this metric is available to all KittenSweaters.com employees.

You can use this new metric in any report by clicking “Show Metrics” and adding it to the metrics displayed:

Now you’ll get to see the new Metrics Selector rather than the old, clunky pop-up. Select it from the list to populate your report. You can also select the default Revenue metric, to view the two side by side and see how your corrections have fixed the data.

You can quickly see that our new Revenue metric removes the outliers and negative values we saw in the default one, by correcting the underlying data. (YAY!)

But why not make it even easier? Why make busy KittenSweaters employees have to manually add it to their reports? Under Admin Settings, we can update our corrected metric to be the default:

You can even use these new calculated metrics in Report Builder! (Just be sure to download the newest version.)

It’s a happy day in the KittenSweaters office! While this doesn’t replace the need for IT to fix the underlying data, this definitely helps us more easily provide the necessary reporting to our executive team and make sure people are looking at the most accurate data.

Keep in mind one potential ‘gotcha’: If the segment underlying the calculated metric is edited, this will affect the calculated metric. This makes life easier while you’re busy building and testing your segment and calculated metric, but could have consequences if someone unknowingly edits the segment and affects the metric.

Share your cool uses of the new calculated metrics in the comments! If you haven’t had a chance to play around with them yet, check out this series of videos to learn more.

* Obviously KittenSweaters.com isn’t actually my client, but how puuuurrfect would it be if they were?!