Author: Josh West

Adobe Analytics, General, google analytics, Technical/Implementation

Fork in the Road: The Big Questions Organizations are Trying to Answer

Posted on by Josh West

In a normal year, we’d be long past the point in the calendar where I had written a blog post on all of the exciting things I had seen at Adobe Summit. Unfortunately, nothing about this spring has been normal other than that Summit was in person again this year (yay!), because I was unable to attend. Instead, it was my wife and 3 of my kids that headed to Las Vegas the last week in March; they saw Taylor Swift in concert instead of Run DMC, and I stayed home with the one who had other plans.

And boy, does it sound like I missed a lot. I knew something was up when Adobe announced a new product analytics-based solution to jump into what has already been a pretty competitive battle. Then, another one of our partners, Brian Hawkins, started posting excitedly on Slack that historically Google-dominant vendors were gushing about the power of Analytics Workspace and Customer Journey Analytics (CJA). Needless to say, it felt a bit like three years of pent-up remote conference angst went from a simmer to a boil this year, and I missed all the action. But, in reading up on everyone else’s takes from the event, it sure seems to track with a lot of what we’ve been seeing with our own clients over the past several months as well.

Will digital analytics or product analytics win out?

Product analytics tools have been slowly growing in popularity for years; we’ve seen lots of our clients implement tools like Heap, Mixpanel, or Amplitude on their websites and mobile apps. But it has always been in addition to, not as a replacement for traditional digital analytics tools. 2022 was the year when it looked like that might change, for two main reasons:

  • Amplitude started adding traditional features like marketing channel analysis into its tool that had previously been sorely lacking from the product analytics space;
  • Google gave a swift nudge to its massive user base, saying that, like it or not, it will be sunsetting Universal Analytics, and GA4 will be the next generation of Google Analytics.

These two events have gotten a lot of our clients thinking about what the future of analytics looks like for them. For companies using Google Analytics, does moving to GA4 mean that they have to adopt a more product analytics/event driven approach? Is GA4 the right tool for that switch?

And for Adobe customers, what does all this mean for them? Adobe is currently offering Customer Journey Analytics as a separate product entirely, and many customers are already pretty satisfied with what they have. Do they need to pay for a second tool? Or can they ditch Analytics and switch to CJA without a ton of pain? The most interesting thing to me about CJA is that it offers a bunch of enhancements over Adobe Analytics – no limits on variables, uniques, retroactivity, cross-channel stitching – and yet many companies have not yet decided that the effort necessary to switch is worth it.

Will companies opt for a simple or more customizable model for their analytics platform?

Both GA4 and Amplitude are on the simpler side of tools to implement; you track some events on your website, and you associate some data to those events. But the data model is quite similar between the two (I’m sure this is an overstatement they would both object to, but in terms of the data they accept, it’s true enough). On the other hand, for CJA, you really need to define the data model up front – even if you leverage one of the standard data models Adobe offers. And any data model is quite different from the model used by Omniture SiteCatalyst / Adobe Analytics for the better part of the last 20 years – though it probably makes far more intuitive sense to a developer, engineer, or data scientist.

Will some companies answer to the “GA or Adobe” question be “both?”

One of the more surprising things I heard coming out of Summit was the number of companies considering using both GA4 and CJA to meet their reporting needs. Google has a large number of loyal customers – Universal Analytics is deployed on the vast majority of websites worldwide, and most analysts are familiar with the UI. But GA4 is quite different, and the UI is admittedly still playing catchup to the data collection process itself. 

At this point, a lot of heavy GA4 analysis needs to be done either in Looker Studio or BigQuery, which requires SQL (and some data engineering skills) that many analysts are not yet comfortable with. But as I mentioned above, the GA4 data model is relatively simple, and the process of extracting data from BigQuery and moving it somewhere else is straightforward enough that many companies are looking for ways to keep using GA4 to collect the data, but then use it somewhere else.

To me, this is the most fascinating takeaway from this year’s Adobe Summit – sometimes it can seem as if Adobe and Google pretend that the other doesn’t exist. But all of a sudden, Adobe is actually playing up how CJA can help to close some of the gaps companies are experiencing with GA4.

Let’s say you’re a company that has used Universal Analytics for many years. Your primary source of paid traffic is Google Ads, and you love the integration between the two products. You recently deployed GA4 and started collecting data in anticipation of UA getting cut off later this year. Your analysts are comfortable with the old reporting interface, but they’ve discovered that the new interface for GA4 doesn’t yet allow for the same data manipulations that they’ve been accustomed to. You like the Looker Studio dashboards they’ve built, and you’re also open to getting them some SQL/BigQuery training – but you feel like something should exist between those two extremes. And you’re pretty sure GA4’s interface will eventually catch up to the rest of the product – but you’re not sure you can afford to wait for that to happen.

At this point, you notice that CJA is standing in the corner, waving both hands and trying to capture your attention. Unlike Adobe Analytics, CJA is an open platform – meaning, if you can define a schema for your data, you can send it to CJA and use Analysis Workspace to analyze it. This is great news, because Analysis Workspace is probably the strongest reporting tool out there. So you can keep your Google data if you like it – keep it in Google, leverage all those integrations between Google products – but also send that same data to Adobe and really dig in and find the insights you want.

I had anticipated putting together some screenshots showing how easy this all is – but Adobe already did that for me. Rather than copy their work, I’ll just tell you where to find it:

  • If you want to find out how to pull historical GA4 data into CJA, this is the article for you. It will give you a great overview on the process.
  • If you want to know how to send all the data you’re already sending to GA4 to CJA as well, this is the article you want. There’s already a Launch extension that will do just that.

Now maybe you’re starting to put all of this together, but you’re still stuck asking one or all of these questions:

“This sounds great but I don’t know if we have the right expertise on our team to pull it off.”

“This is awesome. But I don’t have CJA, and I use GTM, not Launch.”

“What’s a schema?”

Well, that’s where we come in. We can walk you through the process and get you where you want to be. And we can help you do it whether you use Launch or GTM or Tealium or some other tag management system. The tools tend to be less important to your success than the people and the plans behind them. So if you’re trying to figure out what all this industry change means for your company, or whether the tools you have are the right ones moving forward, we’re easy to find and we’d love to help you out.

Photo credits: Thumbnail photo is licensed under CC BY-NC 2.0

Adobe Analytics, Featured, google analytics

Switching from Adobe to Google? What you Should Know (Part 2)

Posted on by Josh West

Last week, I went into detail on four key differences between Adobe and Google Analytics. This week, I’ll cover four more. This is far from an exhaustive list – but the purpose of these posts is not to cover all the differences between the two tools. There have been numerous articles over the years that go into great detail on many of these differences. Instead, my purpose here is to identify key things that analysts or organizations should be aware of should they decide to switch from one platform to another (specifically switching from Adobe to Google, which is a question I seem to get from one of my clients on a monthly basis). I’m not trying to talk anyone out of such a change, because I honestly feel like the tool is less important than the quality of the implementation and the team that owns it. But there are important differences between them, and far too often, I see companies decide to change to save money, or because they’re unhappy with their implementation of the tool (and not really with the tool itself).

Topic #5: Pathing

Another important difference between Adobe and Google is in path and flow analysis. Adobe Analytics allows you to enable any traffic variable to use pathing – in theory, up to 75 dimensions, and you can do path and next/previous flow on any of them. What’s more, with Analytics Workspace, you can also do flow analysis on any conversion variable – meaning that you can analyze the flow of just about anything.

Google’s Universal Analytics is far more limited. You can do flow analysis on both Pages and Events, but not any custom dimensions. It’s another case where Google’s simple UI gives it a perception advantage. But if you really understand how path and flow analysis work, Adobe’s ability to path on many more dimensions, and across multiple sessions/visits, can be hugely beneficial. However, this is an area Google has identified for improvement, and GA4 is bringing new capabilities that may help bring GA closer to par.

Topic #6: Traffic Sources/Marketing Channels

Both Adobe and Google Analytics offer robust reporting on how your users find your website, but there are subtle differences between them. Adobe offers the ability to define as many channels as you want, and define the rules for those channels you want to use. There are also pre-built rules you can use if you need. So you can accept Adobe’s built-in way of identifying social media traffic, but also make sure your paid social media links are correctly detected. You can also classify your marketing channel data into as many dimensions as you want.

Google also allows you as many channels as you want to use, but its tool is built around 5 key dimensions: source, medium, campaign, keyword, and content. These dimensions are typically populated using a series of query parameters prefixed with “utm_,” though they can also be populated manually. You can use any dimension to set up a series of channel groupings as well, similar to what Adobe offers.

For paid channels, both tools offer more or less the same features and capabilities; Adobe offers far more flexibility in configuring how non-paid channels should be tracked. For example, Adobe allows you to decide that certain channels should not overwrite a previously identified channel. But Google overwrites any old channel (except direct traffic) as soon as a new channel is identified – and, what’s more, immediately starts a new session when this happens (this is one of the quirkiest parts of GA, in my opinion).

Both tools allow you to report on first, last, and multi-touch attribution – though again, Adobe tends to offer more customizability, while Google’s reporting is easier to understand and navigate, GA4 offers some real improvements to make attribution reporting even easier. Google Analytics is also so ubiquitous that most agencies are immediately familiar with and ready to comply with a company’s traffic source reporting standards.

One final note about traffic sources is that Google’s integrations between Analytics and other Google marketing and advertising tools offer real benefits to any company – so much so that I even have clients that don’t want to move away from Adobe Analytics but still purchase GA360 just to leverage the advertising integrations.

Topic #7: Data Import / Classifications

One of the most useful features in Adobe Analytics is Classifications. This feature allows a company to categorize and classify the data captured in a report into additional attributes or metadata. For example, a company might capture the product ID at each step of the purchase process, and then upload a mapping of product IDs to names, categories, and brands. Each of those additional attributes becomes a “free” report in the interface. You don’t need to allocate an additional variable for it, but every attribute becomes its own report. This allows data to be aggregated or viewed in new ways. These classifications are also the only truly retroactive data in the tool – you can upload and overwrite classifications at any time, overwriting the data that was there previously. In addition, Adobe also has a powerful tool allowing you to not just upload your metadata, but also write matching rules (even using regular expressions) and have the classifications applied automatically, updating the classification tables each night.

Google Analytics has a similar feature, called Data Import. On the whole, Data Import is less robust than Classifications – for example, every attribute you want to enable as a new report in GA requires allocating one of your custom dimensions. However, Data Import has one important advantage over Classifications – the possibility to process the metadata in two different ways:

  • Query Time Data Import: Using this approach, the metadata you upload gets mapped to the primary dimension (the product ID in my example above) when you run your report. This is identical to how Adobe handles its classification data.
  • Processing Time Data Import: Using this approach, the metadata you upload gets mapped to the primary dimension at the time of data collection. This means that Google gives you the ability to report on your metadata either retroactively or non-retroactively.

This distinction may not be initially obvious, so here’s an example. Let’s say you capture a unique ID for your products in a GA custom dimension, and then you use data import to upload metadata for both brand name and category. The brand name is unlikely to change; a query time data import will work just fine. However, let’s say that you frequently move products between categories to find the one where they sell best. In this case, a query time data import may not be very useful – if you sold a pair of shoes in the “Shoes” category last month but are now selling it under “Basketball,” when you run a report over both months, that pair of shoes will look like it’s part of the Basketball category the entire time. But if you use a processing time data import, each purchase will be correctly attributed to the category in which it was actually sold.

Topic #8: Raw Data Integrations

A few years ago, I was hired by a client to advise them on whether they’d be better off sticking with what had become a very expensive Adobe Analytics integration or moving to Google Analytics 360. I found that, under normal circumstances, they would have been an ideal candidate to move to Google – the base contract would save them money, and their reporting requirements were fairly common and not reliant on Adobe features like merchandising that are difficult to replicate with Google.

What made the difference in my final recommendation to stick with Adobe was that they had a custom integration in place that moved data from Adobe’s raw data feeds into their own massive data warehouse. A team of data scientists relied heavily on integrations that were already built and working successfully, and these integrations would need to be completely rebuilt if they switched to Google. We estimated that the cost of such an effort would likely more than make up the difference in the size of their contracts (it should be noted that the most expensive part of their Adobe contract was Target, and they were not planning on abandoning that tool even if they abandoned Analytics).

This is not to say that Adobe’s data feeds are superior to Google’s BigQuery product; in fact, because BigQuery runs of Google’s ubiquitous cloud platform, it’s more familiar to most database developers and data scientists. The integration between Universal Analytics and BigQuery is built right into the 360 platform, and it’s well structured and easy to work with if you are familiar with SQL. Adobe’s data feeds are large, flat, and require at least cursory knowledge of the Adobe Analytics infrastructure to consume properly (long, comma-delimited lists of obscure event and variable names cause companies all sorts of problems). But this company had already invested in an integration that worked, and it seemed costly and risky to switch.

The key takeaway for this topic is that both Adobe and Google offer solid methods for accessing their raw data and pulling it into your own proprietary databases. A company can be successful integrating with either product – but there is a heavy switching cost for moving from one to the other.

Here’s a summary of the topics covered in this post:

FeatureGoogle AnalyticsAdobe
PathingAllows pathing and flow analysis only on pages and events, though GA4 will improve on thisAllows pathing and flow analysis on any dimension available in the tool, including across multiple visits
Traffic Sources/Marketing ChannelsPrimarily organized around use of “utm” query parameters and basic referring domain rules, though customization is possible

Strong integrations between Analytics and other Google marketing products

 

Ability to define and customize channels in any way that you want, including for organic channels

Data Import/ClassificationsData can be categorized either at processing time or at query time (query time only available for 360 customers)

Each attribute/classification requires use of one of your custom dimensions

Data can only be categorized at query time

Unlimited attributes available without use of additional variables

Raw Data IntegrationsStrong integration between GA and BigQuery

Uses SQL (a skillset possessed by most companies)

Data feeds are readily available and can be scheduled by anyone with admin access

Requires processing of a series of complex flat files

In conclusion, Adobe and Google Analytics are the industry leaders in cloud-based digital analytics tools, and both offer a rich set of features that can allow any company to be successful. But there are important differences between them, and too often, companies that decide to switch tools are unprepared for what lies ahead. I hope these eight points have helped you better understand how the tools are different, and what a major undertaking it is to switch from one to the other. You can be successful, but that will depend more on how you plan, prepare, an execute on your implementation of whichever tool you choose. If you’re in a position where you’re considering switching analytics tools – or have already decided to switch but are unsure of how to do it successfully, please reach out to us and we’ll help you get through it.

Photo credits: trustypics is licensed under CC BY-NC 2.0

Adobe Analytics, Featured, google analytics, Uncategorized

Switching from Adobe to Google? What You Should Know (Part 1)

Posted on by Josh West

In the past few months, I’ve had the same conversation with at least 5 different clients. After the most recent occurrence, I decided it was time to write a blog post about it. This conversation has involved a client either having made the decision to migrate from Adobe Analytics to Google Analytics 360 – or deciding to invest in both tools simultaneously. This isn’t a conversation that is new to me – I’ve had it at least a few times a year since I started at Demystified. But this year has struck me particularly because of both the frequency and the lack of awareness among some of my clients at what this undertaking actually means to a company as large as those I typically work with. So I wanted to highlight the things I believe anyone considering a shift like this should know before they jump. Before I get into a discussion about the feature set between the tools, I want to note two things that have nothing to do with features and the tools themselves.

  • If you’re making this change because you lack confidence in the data in your current tool, you’re unlikely to feel better after switching. I’ve seen far too many companies that had a broken process for implementing and maintaining analytics tracking hope that switching platforms would magically fix their problems. I have yet to see a company actually experience that magical change. The best way to increase confidence in your data is to audit and fix your implementation, and then to make sure your analysts have adequate training to use whichever tool you’ve implemented. Switching tools will only solve your problem if it is accompanied by those two things.
  • If you’re making this change to save money, do your due diligence to make sure that’s really the case. Google’s pricing is usually much easier to figure out than Adobe’s, but I have seen strange cases where a company pays more for Google 360 than Adobe. You also need to make sure you consider the true cost of switching – how much will it take to start over with a new tool? Have you included the cost of things like rebuilding back-end processes for consuming data feeds, importing data into your internal data warehouse, and recreating integrations with other vendors you work with?

As we take a closer look at actual feature differences between Adobe and Google, I want to start by saying that we have many clients successfully using each tool. I’m a former Adobe employee, and I have more experience with Adobe’s tool’s than Google’s. But I’ve helped enough companies implement both of these tools to know that a company can succeed or fail with either tool, and a company’s processes, structure, and culture will be far more influential in determining success than which tool you choose. Each has strengths and features that the other does not have. But there are a lot of hidden costs in switching that companies often fail to think about beforehand. So if your company is considering a switch, I want you to know things that might influence that decision; and if your management team has made the decision for you, I want you to know what to expect.

A final caveat before diving in…this series of posts will not focus much on GA4 or the Adobe Experience Platform, which represent the future of each company’s strategy. There are similarities between those two platforms, namely that both open allow a company to define its own data schema, as well as more easily incorporate external data sources in the reporting tool (Google’s Analysis tool or Adobe’s Analysis Workspace). I’ll try to call out points where these newer platforms change things, but my own experience has shown me that we’re still a ways out from most companies being ready to fully transition from the old to the new platforms.

Topic #1: Intended Audience

The first area I’d like to consider may be more opinion than fact – but I believe that, while neither company may want to admit it, they have targeted their analytics solutions to different markets. Google Analytics takes a far more democratic approach – it offers a UI that is meant to be relatively easy for even a new analyst to use. While deeper analysis is possible using Data Studio, Advanced Analysis, or BigQuery, the average analyst in GA generally uses the reports that are readily available. They’re fast, easy to run, and offer easily digestible insights.

On the other hand, I frequently tell my clients that Adobe gives its customers enough rope to hang themselves. There tend to be a lot more reports at an analyst’s fingertips in Adobe Analytics, and it’s not always clear what the implications are for mixing different types of dimensions and metrics. That complexity means that you can hop into Analysis Workspace and pretty quickly get into the weeds.

I’ve heard many a complaint from analyst with extensive GA experience that join a company that uses Adobe, usually about how hard it is to find things, how unintuitive the UI is, etc. It’s a valid complaint – and yet, I think Adobe kind of intends for that to be the case. The two tools are different – but they are meant to be that way.

Topic #2: Sampling

Entire books have been written on Google Analytics’ use of sampling, and I don’t want to go into that level of detail here. But sampling tends to be the thing that scares analysts the most when they move from Adobe to Google. For those not familiar with Adobe, this is because Adobe does not have it. Whatever report you run will always include 100% of the data collected for that time period (one exception is that Adobe, like Google, does maintain some cardinality limits on reports, but I consider this to be different from sampling).

The good news is that Google Analytics has dramatically reduced the impact of sampling over the years, to the point where there are many ways to get unsampled data:

  • Any of the default reports in Google’s main navigation menus is unsampled, as long as you don’t add secondary dimensions, metrics, or breakdowns.
  • You always have the option of downloading an unsampled report if you need it.
  • Google 360 customers have the ability to create up to 100 “custom tables” per property. A custom table is a report you build in advance that combines all the dimension and metrics you know you need. When you run reports using a custom table you can apply dimensions, metrics, and segments to the report in any way you choose, without fear of sampling. They can be quite useful, but they must be built ahead of time and cannot be changed after that.
  • You can always get unsampled data from BigQuery, provided that you have analysts that are proficient with SQL.

It’s also important to note that most companies that move from Adobe to Google choose to pay for Google 360, which has much higher sampling thresholds than the free version of Google Analytics. The free version of GA turns on sampling once you exceed 500,000 sessions at the property level for the date range you are using. But GA 360 doesn’t apply sampling until you hit 100,000,000 sessions at the view level, or start pulling intra-day data. So not only is the total number much higher, but you can also structure your views in a way that makes sampling even less of an issue.

Topic #3: Events

Perhaps one of the most difficult adjustments for an analyst moving from Adobe to Google – or vice-versa – is event tracking. The confusion stems from the fact that the word “event” means something totally different in each tool:

  • In Adobe, an event usually refers to a variable used by Adobe Analytics to count things. A company gets up to 1000 “success events” that are used to count either the number of times something occurred (like orders) or a currency amount associated with a particular interaction (like revenue). These events become metrics in the reporting interface. The equivalent would be a goal or custom metric in Google Analytics – but Adobe’s events are far more useful throughout the reporting tools than custom metrics. They can also be serialized (counted only once per visit, or counted once for some unique ID).
  • In Google, an event refers to an interaction a user performs on a website or mobile app. These events become a specific report in the reporting interface, with a series of different dimensions containing data about the event. Each event you track has an associated category, action, label, and value. There really is no equivalent in Adobe Analytics – events are like a combination of 3 props and a corresponding success event, all rolled up into one highly useful report (unlike the custom links, file download, and exit links reports). But that report can often become overloaded or cluttered because it’s used to report on just about every non-page view interaction on the site.

If you’ve used both tools, these descriptions probably sound very unsophisticated. But it can often be difficult for an analyst to shift from one tool to the other, because he or she is used to one reporting framework, and the same terminology means something completely different in the other tool. GA4 users will note here that events have changed again from Universal Analytics – even page and screen views are considered to be events in GA4, so there’s even more to get used to when making that switch.

Topic #4: Conversion and E-commerce Reporting

Some of the most substantial differences between Adobe and Google Analytics are in their approach to conversion and e-commerce reporting. There are dozens of excellent blog posts and articles about the differences between props and eVars, or eVars and custom dimensions, and I don’t really want to hash that out again. But for an Adobe user migrating to Google Analytics, it’s important to remember a few key differences:

  • In Adobe Analytics, you can configure an eVar to expire in multiple ways: after each hit, after a visit/session, to never expire, after any success event occurs, or after any number of days. But in Google Analytics, custom dimensions can only expire after hits, sessions, or never (there is also the “product” option, but I’m going to address that separately).
  • In Adobe Analytics, eVars can be first touch or last touch, but in Google Analytics, all custom dimensions are always last touch.

These are notable differences, but it’s generally possible to work around those limitations when migrating to Google Analytics. However, there is a concept in Adobe that has virtually no equivalent in Google – and as luck would have it, it’s also something that even many Adobe users struggle to understand. Merchandising is the idea that an e-commerce company might want to associate different values of a variable with each product the customer views, adds to cart, or purchases. There are 2 different ways that merchandising can be useful:

  • Method #1: Let’s consider a customer that buys multiple products, and wants to use a variable or dimension to capture the product name, category, or some other common product attribute. Both Adobe and Google offer this type of merchandising, though Google requires each attribute to be passed on each hit where the product ID is captured, while Adobe allows an attribute to be captured once and associated with that product ID until you want it to expire.
  • Method #2: Alternatively, what if the value you want to associate with the product isn’t a consistent product attribute? Let’s say that a customer finds her first product via internal search, and her second by clicking on a cross-sell offer on that first product. You want to report on a dimension called “Product Finding Method.” We’re no longer dealing with a value that will be the same for every customer that buys the product; each customer can find the same product in different ways. This type of merchandising is much easier to accomplish with Adobe than with Google I could write multiple blog posts about how to implement this in Adobe Analytics, so I won’t go into additional detail here. But it’s one of the main things I caution my Adobe clients about when they’re considering switching.

At this point, I want to highlight Google’s suite of reports called “Enhanced E-commerce.” This is a robust suite of reports on all kinds of highly useful aspects of e-commerce reporting: product impressions and clicks, promotional impressions and clicks, each step of the purchase process from seeing a product in a list, to viewing a product detail page, all the way through checkout. It’s built right into the interface in a standardized way, using a standard set of dimensions which yields a et of reports that will be highly useful to anyone familiar with the Google reporting interface. While you can create all the same types of reporting in Adobe, it’s more customized – you pick which eVars you want to use, choose from multiple options for tracking impressions and clicks, and end up with reporting that is every bit of useful but far less user-friendly than in Google’s enhanced e-commerce reporting.

In the first section of this post, I posited that the major difference between these tools is that Adobe focuses on customizability, while Google focuses on standardization. Nowhere is that more apparent than in e-commerce and conversion reporting: Google’s enhanced e-commerce reporting is simple and straightforward. Adobe requires customization to accomplish a lot of the same things, but while layering on complex like merchandising, offers more robust reporting in the process.

One last thing I want to call out in this section is that Adobe’s standard e-commerce reporting allows for easy de-duplication of purchases based on a unique order ID. When you pass Adobe the order ID, it checks to make sure that the order hasn’t been counted before; if it has, it does not count the order a second time. Google, on the other hand, also accepts the order ID as a standard dimension for its reporting – but it doesn’t perform this useful de-duplication on its own. If you want it, you have to build out the functionality as part of your implementation work.

Here’s a quick recap on what we’ve covered so far:

FeatureGoogle AnalyticsAdobe
SamplingStandard: Above 500,000 sessions during the reporting period

360: Above 100,000,000 sessions during the reporting period

Does not exist
CardinalityStandard: 50,000 unique values per report per day, or 100,000 uniques for multi-day tables

360: 1,000,000 unique values per report per day, or 150,000 unique values for multi-day tables

500,000 unique values per report per month (can be increased if needed)
Event TrackingUsed to track interactions, using 3 separate dimensions (category, action, label)Used to track interactions using a single dimension (i.e. the “Custom Links” report)
Custom Metrics/Success Events200 per property

Can track whole numbers, decimals, or currency

Can only be used in custom reports

1,000 per report suite

Can track whole numbers, decimals, or currency

Can be used in any reports

Can be serialized

Custom Dimensions/Variables200 per property

Can be scoped to hit, session, or user

Can only be used in custom reports

Can only handle last-touch attribution

Product scope allows for analysis of product attributes, but nothing like Adobe’s merchandising feature exists

250 per report suite

Can be scoped to hit, visit, visitor, any number of days, or to expire when any success event occurs

Can be used in any report

Can handle first-touch or last-touch attribution

Merchandising allows for complex analysis of any possible dimension, including product attributes

E-Commerce ReportingPre-configured dimensions, metrics, and reports exist for all steps in an e-commerce flow, starting with product impressions and clicks and continuing through purchasePre-configured dimensions and metrics exist all steps in an e-commerce flow, starting with product views and continuing through purchase

Product impressions and clicks can also be tracked using additional success events

This is a good start – but next week, I’ll dive into a few additional topics: pathing, marketing channels, data import/classifications, and raw data integrations. If it feels like there’s a lot to keep track of, it should. Migrating from one analytics tool to another is a big job – and sometimes the people who make a decision like this aren’t totally aware of the burden it will place on their analysts and developers.

Photo credits: trustypics is licensed under CC BY-NC 2.0

Technical/Implementation

Ways to Minimize the Impact of ITP 2.1 on your Analytics Practice

Posted on by Josh West

Demystified Partner Tim Patten also contributed to this blog post.

Earlier this week, I shared our team’s thoughts about Apple’s Intelligent Tracking Protocol (ITP), specifically version 2.1 and its impact on digital analytics. These changes have been exhaustively covered, and we likely didn’t add anything new on the topic. But we thought those comments were important to lead into what I think is the tactical discussion that every company needs to have to ensure they deal with ITP 2.1 (or 2.2, or any future version as well) in a way that is appropriate for its business.

Admittedly, we haven’t done a ton of research on the impact of ITP to digital marketing in general – how it impacts paid search, or display, or social media marketing tools. I mostly work with clients on deploying analytics tools, and generally using either Adobe or Google Analytics – so that’s where most of our thoughts have been since Apple announced ITP 2.1. So most of what follows will focus on those 2 vendors. But since the impact of these changes is not limited to traditional “web analytics,” we’ll also share a few thoughts on a few more all-encompassing potential solutions.

Adobe Analytics (and other Adobe tools)

Omniture’s earliest implementation used a third-party cookie, set when analytics requests were sent to its servers (first using the 2o7.net domain, followed by omtrdc.net). The negative aspects of third-party cookies led Omniture to then introduce a new approach. For nearly a decade, the new best practices recommendation for implementing Omniture’s (and then Adobe’s) analytics tool was to work with them to make it appear as if your analytics data were being sent to your own servers. This would be done by creating a CNAME, and then specifying that subdomain as your tracking server (like metrics.example.com). The first request by a new visitor to your site would be sent to that subdomain, followed by a response with a “set cookie” header that would make your analytics visitor ID a first-party cookie. All analytics requests would be sent to that subdomain as well – without the header, since the cookie was already set.

About five years ago, Adobe decided that was a bit of a cumbersome approach, and as part of its new Experience Cloud ID Service, began setting a first-party cookie using JavaScript. While you could still work with Adobe to use the CNAME approach, it became less critical – and even when using a CNAME, the cookie was still set exclusively with JavaScript. This was a brilliant approach – right up until ITP 2.1 was announced, and all of a sudden, a very high percentage of Safari website visitors now had a visitor ID cookie that would be deleted in 7 days, with nothing they could do about it.

As of May 15, Adobe now has a workaround in place for its customers that had been leveraging the Experience Cloud ID. Customers that already had a CNAME were immediately ready to use this solution, but the rest are required to introduce a CNAME to take advantage. In addition to the CNAME, you must update to the latest version of the Experience Cloud Visitor ID service (version 4.3). In most cases, this can be done through your tag management system – though not all TMS tools have offered this update yet.

It’s important to note that this solution acts like a workaround – it will set an additional cookie called “s_ecid” in the user’s browser using your CNAME tracking server. It does not reissue the older AMCV cookie that was previously used for visitor identification; instead, it uses the s_ecid cookie as a fallback in case the ACMV cookie has expired. The total number of cookies is frequently a concern for IT teams, so make sure you know this if you opt for this approach. You can read more about this implementation in Adobe’s help documentation.

The last important thing to be aware of is that this fix is only for the Experience Cloud visitor ID. It does not address Adobe Target’s mbox cookie, or any other cookies used by other products in the Adobe Marketing Cloud that were previously set with JavaScript. So it solves the biggest problem introduced by ITP 2.1 – but not all of them.

Google Analytics

Ummm…how do I put this nicely?

To this point, Google has offered very little in the way of a recommendation, solution, or anything else when it comes to ITP 2.1. Google’s stance has been that if you feel it’s a problem, you should figure out how to solve it yourself.

This may be somewhat unfortunate – but it makes sense when you think that not only is Google Chrome a competitor to Safari, but other tools in Google’s marketing suite have been heavily impacted each time ITP has introduced new restrictions – and Google didn’t do anything then, either. So this is not a new or unexpected development.

All of this leads to an interesting question: what do I do if I don’t use Adobe Analytics? Or if I use it without a CNAME? Or if I care about other vendors besides Adobe? Luckily there are a few options out there.

Roll Your Own

If you’re looking for a workaround to preserve your cookies, you could always build your own homegrown solution. Simo Ahava discussed several potential ideas here – many of which have real shortcomings. In my opinion, the most viable of these are a series of similar approaches that involve routing some traffic on each page through a type of server-side “gateway” that would “clean” all your cookies for you by re-issuing them with the Set-cookie header. This approach works regardless of how many domains and subdomains your site encompasses, which makes it a fairly robust approach.

It’s not without its challenges, however. The main challenge is that it requires at least some amount of development work and some long-term maintenance of whatever server-side tools you use – a server-side script, a custom CNAME, etc. You’ll encounter another challenge if your site is a single-page application or does any virtual page view tracking – because some vendors will continue to update their cookies as the user interacts with the page, and each cookie update re-corrupts the cookie. So your homegrown solution has to make sure that it continuously cleans the cookies for as long as the page is open in the browser. Another item that you will need to manage on your own is the ability to handle a user’s opt-out settings across all of the different cookies that you manage through this new “gateway.”

Third-Party Tools

If building your own custom solution to solve the problems introduced by ITP 2.1 sounds tedious (at best) or a nightmare (at worst), as luck would have it, you have one last option to consider. There are a handful of companies that have decided to tackle the problem for you. The one I have the most experience with is called Accutics, and you may have seen them at Adobe Summit or heard them on a recent episode of the Digital Analytics Power Hour.

The team at Accutics saw an opportunity in Apple’s ITP 2.1 announcements, and built what they call a “cookie saver” solution that can deal with all of your JavaScript-based cookies. It’s an approach very similar to the solution Adobe deployed a few weeks ago – with the added benefit that it will work for as many cookies as you need it to. They’ve also built their tool to deal with the single-page app considerations I mentioned in the previous section, as they continuously monitor the cookies you tell them you want to preserve to ensure they stay clean (though they do this just like Adobe, so you might notice a few additional cookies show up in your browser as a result). Once you’ve gotten a CNAME set up, the Accutics solution can be quickly deployed through any tag management system in a matter of minutes, so the solution is relatively painless compared to the impact of ITP 2.1.

Conclusion

While Apple’s release of ITP 2.1 may feel a bit like someone tossed a smoke bomb into the entryway of your local grocery store, the good news is that you have options to deal with it. Some of these options are more cumbersome than others – but you don’t have to feel helpless. You can use the analysis of your data to determine the impact of ITP on your business, as well as the potential solutions out there, to identify what the right approach should be as you move forward. ITP won’t be the last – or most problematic – innovation in user privacy that poses a challenge for digital analytics. Luckily, there are workarounds available to you – you just need to decide which solution will allow you to best balance your customers’ privacy with your organization’s measurement goals.

Technical/Implementation

A Less Technical Guide to Apple’s ITP 2.1 Changes

Posted on by Josh West

Demystified Partner Tim Patten also contributed to this blog post.

There are likely very few analysts and developers that have not yet heard that Apple recently introduced some major changes into its Safari web browser. A recent version of Apple’s Intelligent Tracking Protocol (ITP 2.1) has the potential to fundamentally change the way business is done and analyzed online, and this has a lot of of marketers quite worried about the future of digital analytics. You may have noticed that we at Demystified have been pretty quiet about the whole thing – as our own Adam Greco has frequently reminded us over the past few weeks. This isn’t because Kevin, Tim, and I don’t have some strong opinions about the whole thing, or some real concerns about what it means for our industry. Rather, it’s based on 2 key reasons:

  • Apple has released plenty of technical details about ITP 2.1 – what problems Apple sees and is trying to solve, what the most recent versions of Safari do to solve these problems, and what other restrictions may lie ahead. What’s more, the Measure Slack community has fostered robust discussion on what ITP 2.1 means to marketers, and we wholeheartedly endorse all the discussion taking place there.
  • ITP 2.1 is a very new change, and a very large shift – and we’ve all seen that the leading edge of a technological shift sometimes ends up being a bit ahead of its time. While discussing the potential implications of ITP 2.1 with clients and peers, we have been taking a bit of a “wait and see” approach to the whole thing. We’ve wanted to see not just what other browsers will do (follow suit like Firefox? hold steady like Chrome?), but what the vendors impacted by these changes – and that our clients care most about – will decide to do about them.

Now that the dust has settled a bit, and we’ve moved beyond ITP 2.1 to even more restrictions with ITP 2.2 (which lowers the limit from seven days to one if the URL contains query parameters mean to pass IDs from one domain to another), we feel like we’re on a little bit firmer footing and prepared to discuss some of the details with our clients. As Tim and I talked about what we wanted to write, we landed on the idea that most of the developers we talk to have a pretty good understanding about what Apple’s trying to do here – but analysts and marketers are still somewhat in the dark. So we’re hoping to present a bit of a “too long, didn’t read” summary of ITP 2.1. A few days from now, we’ll share a few thoughts on what we think ITP 2.1 means for most of the companies we work with, that use Adobe or Google Analytics, and are wondering most about what it means for the data those vendors deliver. If you feel like you’re still in the dark about cookies in general, you might want to review a series of posts I wrote a few years ago about why they are important in digital marketing. Alternatively, if you find yourself more interested in the very technical details of ITP, Simo Ahava had a great post that really drilled into how it works.

What is the main problem Apple is trying to solve with ITP?

Apple has decided to take a much more proactive stance on protecting consumer privacy than other companies like Facebook or Google. ITP is its plan for these efforts. Early versions of ITP released through its Safari web browser revolved primarily around limiting the spread of third-party cookies, which are generally agreed upon to be intrusive. Basically, Safari limited the amount of time a third-party cookie could be stored unless the user interacted with the site that set the cookie and it was obvious he or she had an interest in the site.

Advertisers countered this effort pretty easily by coming up with ways to pass IDs between domains through the query string, grabbing values from third-party cookies and rewriting them as first-party cookies, and so forth. So Apple has now tightened controls even further with ITP 2.1 – though the end goal of protecting privacy remains the same.

What is different about ITP 2.1?

The latest versions of ITP take these efforts forward multiple levels. Where earlier versions of ITP focused mainly on third-party cookies, 2.1 takes direct aim at first-party cookies. But not all first-party cookies – just those that are set and manipulated with JavaScript (using the document.cookie browser object). Most cookies that contribute to a user’s website experience are set on the server as part of a page load – for example, if a site sets a cookie containing my user ID when I log in, to keep me logged in on subsequent pages. This is done because the server’s response to the browser includes a special header instructing the browser to set a cookie to store a value. Most advertising cookies are set with code in the vendors’ JavaScript tags, and JavaScript cannot specify that header. Apple has made the giant leap to assuming that any cookie set with JavaScript using document.cookie is non-essential and potentially a contributor to cross-site tracking, the elimination of which is the key goal of ITP. Any cookies set in this way will be discarded by Safari after a maximum of 7 days – unless the user returns to the site before the 7 days passes, which resets the timer – but only by another maximum 7 days.

What does this mean for my analytics data?

The side effect of this decision is that website analytics tracking is potentially placed on the same footing as online advertising. Google Analytics sets its unique client ID cookie in this way – as does Adobe Analytics for many implementations. While it may be difficult for a non-developer to understand the details of ITP 2.1, it’s far easier to understand the impact on data quality when user identification is reset so frequently.

If you think this seems a bit heavy-handed on Apple’s part, you’re not alone. But, unfortunately, there’s not a lot that we as analysts and developers can do about it. And Apple’s goal is actually noble – online privacy and data quality should be a priority for each of us. The progressive emphasis by Apple is a result of so many vendors seeking out workarounds to stick with the status quo rather than coming up with new, more privacy-focused ways of doing business online.

Before you decide that ITP 2.1 is the end of analytics or your career as you know it, there are some things to think about that might help you talk yourself off the ledge. You can put your data to the test to see how big of a deal ITP is for you:

  • How much of your traffic comes from mobile devices? Apple is the most common manufacture of mobile devices, so if you have a lot of mobile traffic, you should be more concerned with ITP.
  • How much of your traffic comes from webkit browsers (Safari being by far the largest)? Safari still has a pretty small share of desktop web traffic – but makes up a much larger share of mobile traffic because it is the default browser on iOS devices. While other browsers like Firefox have shown signs they might follow Apple’s lead, there still isn’t a critical mass of other browsers giving the indication they intend to implement the same restrictions.
  • Does your website require authentication to use? If the answer is yes, all of the major analytics providers offer means to use your own unique identifier rather than the default ones they set via JavaScript-based cookies.
  • Does your website have a high frequency of return visits? If your user base returns to the site very frequently within a 7-day window, the impact to you may be relatively low (though Apple also appears to be experimenting with a window as low as 1 day).

After reading all of these questions, you may still be convinced ITP 2.1 is a big deal for your organization – and you’re probably right. Unique visitor counts will likely be inflated, and attribution analytics will be heavily impacted if the window is capped at 7 days – and these are just the most obvious effects of the changes. There are several different paths you can take from here – some of which will reduce or eliminate your problems, and others that will ignore it and hope it goes away. We’ll follow up later this week to describe these options – and specifically how they relate to Adobe and Google Analytics, since they are the tools most of our clients rely on to run their businesses.

Tag Management, Technical/Implementation

Single-Page Apps: Dream or Nightmare?

Posted on by Josh West

A few months ago, I was discussing a new project with a prospective client, and they described what they needed like this: “We have a brand new website and need to re-implement Adobe Analytics. So far we have no data layer, and we have no developer resources in place for this project. Can you help us re-implement Adobe Analytics?” I generally avoid projects just like this – not because I can’t write server-side application code in several languages, but because even if I am going to write code for a project like that, I still need a sharp developer or two to bounce ideas off of, and ask questions to find out where certain files are located, what standards they want me to follow, and other things like that. In an effort to do due diligence, I asked them to follow up with their IT team on a few basics. Which platform was their site built on? Which programming languages would be required?

When they followed up by saying that the site was built on Websphere using ReactJS, I was sure this project was doomed to failure – every recent client I had worked with that was using either of these technologies struggled mightily, and here was a client using both! In addition, while I understand the premise behind using ReactJS and can generally work my way through a ReactJS application, having to do all the heavy lifting myself was a terrifying thought. In an effort to do due diligence, I agreed to discuss this project with some members of their IT team.

On that call, I quickly realized that there had been a disconnect in how the marketing folks on the project had communicated what the IT folks wanted me to know. I learned that a data layer already existed on the site – and it already contained pretty much everything identified in the solution design that needed to be tracked. We still had to identify a way to track a few events on the website (like cart adds), but I felt good enough about the project to take it on.

This project, and a handful of others over the past year, have challenged some strong opinions I’ve held on single page applications (SPAs for short). Here are just a few of those:

  • SPAs have just as many user experience challenges as the page-based applications they are designed to replace.
  • SPAs are present a major measurement challenge for traditional analytics tools like Adobe or Google Analytics.
  • Most companies move to an SPA-based website because they look and sound cool – they’re just the latest “shiny object” that executives decide they have to have.

While I still hold each of these opinions to some degree, the past few months have given me a much more open mind about single-page applications and frameworks like React or Angular. Measurement of SPAs is definitely a challenge – but it’s not an insurmountable one. If your company is thinking about moving to a single-page application, you need to understand that – just like the site itself is going to be fundamentally different than what you’re used to – the way you measure it will be as well. I’d like to offer a few things you’ll want to strongly consider as you decide how to track your new SPA.

A New Data Architecture

In many ways, SPAs are much better equipped to support a data layer than the old, Frankenstein-ish website you’re moving away from. Many companies I know have such old websites that they pre-date their adoption of a tag management system. Think about that – a tool you probably purchased at least six years ago still isn’t as old as your website itself! So when you implemented your TMS, you probably bolted on your data layer at the same time, grabbing data wherever you could find it.

Migrating to an SPA – even for companies that do this one page at a time – requires a company to fundamentally rethink its approach to data. It’s no longer available in the same ways – which is a good thing. Rather than building the data layer one template at a time like in the past, an SPA typically accesses the data it needs to build a page through a series of APIs that are exposed by back-end development teams. For example, data related to the authenticated user is probably retrieved as the page loads from a service connected to your CRM; data relevant to the contents of a customer’s shopping cart may be accessed through an API integrated with your e-commerce platform; and the content for your pages is probably accessed through an integration with your website CMS. But unlike when you implemented your data layer the first time – when your website already had all that data the way it needed it and in the right locations on the page – your development team has to rethink and rebuild all of that data architecture. You both need the data this time around – which should make collaboration much easier and help you avoid claims that they just can’t get you the data you need.

Timing Challenges for Data Availability

As part of this new approach to data, SPAs typically also introduce a shift in the way they make this data accessible to the browser. The services and APIs I mentioned in the previous section are almost always asynchronous – which introduces a new challenge for measurement teams implementing tracking on SPAs.

On a traditional website, the page is generated on the server, and as this happens, data is pulled into the page from appropriate systems. That data is already part of the page when it is returned to the browser. On an SPA, the browser gets an almost “empty” page with a bunch of instructions on where to get the relevant data for the page; then, as the user navigates, rather than reloading a new page, it just gets a smaller set of instructions for how to update the relevant parts of the page to simulate the effect of navigation.

This “set of instructions” is the API calls I mentioned earlier – the browser is pulling in user data from one service, cart data from another, and product/content data from yet another. As data is made available, it is inserted into the page in the appropriate spot. This can have a positive impact on user experience, because less-relevant data can be added as it comes back, rather than holding up the loading of the entire page. But let’s just say it presents quite a challenge to analytics developers. This is because most tag management systems were built and implemented under the assumption that you’d want to immediately track every page as it loads, and that every new page would actually be a new page. SPAs don’t work like that – if you track an SPA on the page load, or even the DOM ready event, you’re probably going to track it before a significant amount of data is available. So you have to wait to track the initial page load until all the data is ready – and then you have to track subsequent page refreshes of the SPA as if a new page had actually loaded.

You may have experienced this problem before with a traditional website – many companies experiment with the idea of an SPA by trying it out on a smaller part of their website, like user authentication or checkout. Or you’ve maybe seen it with certain third-party tools like your recommendation engine – which, while not really an SPA, have similar timing issues because they feed content onto the page asynchronously. The good news is that most companies that go all-in on SPAs do so all at once, rather than trying to migrate single sections over a longer period of time. They undertake a larger replatforming effort, which probably makes it easier to solve for most of these issues.

Figuring out this timing is one of the most important hurdles you’ll need to coordinate as you implement tracking on an SPA – and it’s different for every site. But the good news is that – as long as you’re using one of the major tag management systems – or planning to migrate from Adobe DTM to Launch as part of your project – it’s the hard part. Because every major TMS has a solution to this problem built right in that allows you to fire any tag on any event that occurs on the page. So your web developers just need to notify your analytics developers when the page is truly “ready.” (Again, if you’re still using Adobe DTM, I can’t emphasize strongly enough that you should switch to Launch if you’re building an SPA. DTM has a few notable “features” that pose major problems for SPAs.)

A New Way of Tracking Events

Another major shift between traditional websites and SPAs is in how on-page events are most commonly tracked. It’s likely that when you first implemented a tag management system, you used a combination of CSS selectors and custom JavaScript you deployed in the TMS, along with events you had your web developers tag that would “trigger” the TMS to do something. Because early sales teams for the major TMS companies used a pitch along the lines of “Do everything without IT!” many companies tried to implement as much tracking as they could using hacks and one-offs in the TMS. All of this custom JavaScript may have had the effect on your TMS of moving all your ugly, one-off tracking JavaScript from your website code to your TMS – without making the actual tracking any cleaner or more elegant.

The good news is that SPAs will force you to clean up your act – because many of the traditional ways of tracking fall down. Because an SPA is constantly updating the DOM without loading a new page, you can’t just add a bunch of event listeners that bind when the page loads (or on DOM ready). You’d need to turn off all your listeners on each page refresh and turn on a bunch of new ones, which can be tedious and prone to error. Another option that will likely not work in every case is to target very broad events (like a body “click”) and then within those handlers just see which element first triggered the event. This approach could also potentially have a negative impact on the user’s experience.

Instead, many teams developing an SPA also develop a new model for listening and responding to events that, just like the data layer, can be leveraged by analytics teams as well.

The company I mentioned at the beginning of this post had an entire catalog of events they already needed to listen for to make the SPA work – for example, they needed to listen for each cart add event so that they could send data about that item to their e-commerce system. The e-commerce system would then respond with an updated version of all the data known about a future order. So they built an API for this – and then, the analytics team was able to use it as well. Without any additional development, we were able to track nearly every key interaction on the website. This was all because they had taken the time to think about how events and interactions should work on the website, and they built something that was extensible to other things on the website than just its core purpose.

This is the kind of thing that a company would almost never do with an old website – it’s a large effort to build this type of event service, and it has to be done inside an old, messy codebase. But when you build an SPA, you have to do it anyway – so you might as well add a little bit more work up front to save you a ton of time later on. Developers figure these kinds of things out as they go – they learn tricks that will save time in the future. SPAs can offer a chance to put some of these tricks into action.

Conclusion

There are many other important things to consider when building a single-page application, and it’s a major undertaking that can take longer than a company plans for. But while I still feel that it’s more difficult to implement analytics on an SPA than any other type of web-based application, it doesn’t have to be the nightmare that many companies encounter. Just remember to make sure your development team is building all this new functionality in a way that everyone can benefit from:

  • While they’re making sure all the data necessary for each view (page) of the website is available, make sure they provide hooks so that other teams (like analytics) can access that data.
  • Consider the impact on your website of all of that data showing up at different times.
  • Develop an event model that makes it easy to track key interactions on the site without relying on fragile CSS selectors and DOM hacks.

A few weeks ago at our ACCELERATE conference, I led a roundtable for the more technical minded attendees. The #1 challenge companies were dealing with when it came to analytics implementation was SPAs. But the key is to take advantage of all the opportunities an SPA can offer – you have to realize it gives you the chance to fix all the things that have broken and been patched together over the years. Your SPA developers are going to spend a lot of time getting the core functionality right – and they can do it in a way that can make your job easier, too, if you get out in front of them and push them to think in innovative ways. If you do, you might find yourself wondering why some folks complain so much about tracking single-page apps. But if you don’t, you’ll be right there complaining with everyone else. If you’re working with SPAs, I’d love to hear from you about how you’re solving the challenges they present – or where you’re stuck and need a little help.

Photo Credit: www.gotcredit.com

0
Adobe Analytics, Tag Management, Technical/Implementation, Testing and Optimization

Adobe Target + Analytics = Better Together

Posted on by Josh West

Last week I wrote about an Adobe Launch extension I built to familiarize myself with the extension development process. This extension can be used to integrate Adobe Analytics and Target in the same way that used to be possible prior to the A4T integration. For the first several years after Omniture acquired Offermatica (and Adobe acquired Omniture), the integration between the 2 products was rather simple but quite powerful. By using a built-in list variable called s.tnt (that did not count against the 3 per report suite available to all Adobe customers), Target would pass a list of all activities and experiences in which a visitor was a participant. This enabled reporting in Analytics that would show the performance of each activity, and allow for deep-dive analysis using all the reports available in Analytics (Target offers a powerful but limited number of reports). When Target Standard was released, this integration became more difficult to utilize, because if you choose to use Analytics for Target (A4T) reporting, the plugins required to make it work are invalidated. Luckily, there is a way around it, and I’d like to describe it today.

Changes in Analytics

In order to continue to re-create the old s.tnt integration, you’ll need to use one of your three list variables. Choose the one you want, as well as the delimiter and the expiration (the s.tnt expiration was 2 weeks).

Changes in Target

The changes you need to make in Target are nearly as simple. Log into Target, go to “Setup” in the top menu and then click “Response Tokens” in the left menu. You’ll see a list of tokens, or data elements that exist within Target, that can be exposed on the page. Make sure that activity.id, experience.id, activity.name, and experience.name are all toggled on in the “Status” column. That’s it!

Changes in Your TMS

What we did in Analytics and Target made an integration possible – we now have a list variable ready to store Target experience data, and Target will now expose that data on every mbox call. Now, we need to connect the two tools and get data from Target to Analytics.

Because Target is synchronous, the first block of code we need to execute must also run synchronously – this might cause problems for you if you’re using Signal or GTM, as there aren’t any great options for synchronous loading with those tools. But you could do this in any of the following ways:

  • Use the “All Pages – Blocking (Synchronous)” condition in Ensighten
  • Put the code into the utag.sync.js template in Tealium
  • Use a “Top of Page” (DTM) or “Library Loaded” rule (Launch)

The code we need to add synchronously attaches an event listener that will respond any time Target returns an mbox response. The response tokens are inside this response, so we listen for the mbox response and then write that code somewhere it can be accessed by other tags. Here’s the code:

	if (window.adobe && adobe.target) {
		document.addEventListener(adobe.target.event.REQUEST_SUCCEEDED, function(e) {
			if (e.detail.responseTokens) {
				var tokens = e.detail.responseTokens;
				window.targetExperiences = [];
				for (var i=0; i<tokens.length; i++) {
					var inList = false;
					for (var j=0; j<targetExperiences.length; j++) {
						if (targetExperiences[j].activityId == tokens[i]['activity.id']) {
							inList = true;
							break;
						}
					}
					
					if (!inList) {
						targetExperiences.push({
							activityId: tokens[i]['activity.id'],
							activityName: tokens[i]['activity.name'],
							experienceId: tokens[i]['experience.id'],
							experienceName: tokens[i]['experience.name']
						});
					}
				}
			}
			
			if (window.targetLoaded) {
				// TODO: respond with an event tracking call
			} else {
				// TODO: respond with a page tracking call
			} 
		});
	}
	
	// set failsafe in case Target doesn't load
	setTimeout(function() {
		if (!window.targetLoaded) {
			// TODO: respond with a page tracking call
		}
	}, 5000);

So what does this code do? It starts by adding an event listener that waits for Target to send out an mbox request and get a response back. Because of what we did earlier, that response will now carry at least a few tokens. If any of those tokens indicate the visitor has been placed within an activity, it checks to make sure we haven’t already tracked that activity on the current page (to avoid inflating instances). It then adds activity and experience IDs and names to a global object called “targetExperiences,” though you could push it to your data layer or anywhere else you want. We also set a flag called “targetLoaded” to true that allows us to use logic to fire either a page tracking call or an event tracking call, and avoid inflating page view counts on the page. We also have a failsafe in place, so that if for some reason Target does not load, we can initiate some error handling and avoid delaying tracking.

You’ll notice the word “TODO” in that code snippet a few times, because what you do with this event is really up to you. This is the point where things get a little tricky. Target is synchronous, but the events it registers are not. So there is no guarantee that this event will be triggered before the DOM ready event, when your TMS likely starts firing most tags.. So you have to decide how you want to handle the event. Here are some options:

  • My code above is written in a way that allows you to track a pageview on the very first mbox load, and a custom link/event tracking call on all subsequent mbox updates. You could do this with a utag.view and utag.link call (Tealium), or trigger a Bootstrapper event with Ensighten, or a direct call rule with DTM. If you do this, you’ll need to make sure you configure the TMS to not fire the Adobe server call on DOM ready (if you’re using DTM, this is a huge pain; luckily, it’s much easier with Launch), or you’ll double-count every page.
  • You could just configure the TMS to call a custom link call every time, which will probably increase your server calls dramatically. It may also make it difficult to analyze experiences that begin on page load.

What my Launch extension does is fire one direct call rule on the first mbox call, and a different call for all subsequent mbox calls. You can then configure the Adobe Analytics tag to fire an s.t() call (pageview) for that initial direct call rule, and an s.tl() call for all others. If you’re doing this with Tealium, make sure to configure your implementation to wait for your utag.view() call rather than allowing the automatic one to track on DOM ready. This is the closest behavior to how the original Target-Analytics integration worked.

I’d also recommend not limiting yourself to using response tokens in just this one way. You’ll notice that there are tokens available for geographic data (based on an IP lookup) and many other things. One interesting use case is that geographic data could be extremely useful in achieving GDPR compliance. While the old integration was simple and straightforward, and this new approach is a little more cumbersome, it’s far more powerful and gives you many more options. I’d love to hear what new ways you find to take advantage of response tokens in Adobe Target!

Photo Credit: M Liao (Flickr)

0
Adobe Analytics, Tag Management, Technical/Implementation

My First Crack at Adobe Launch Extension Development

Posted on by Josh West

Over the past few months, I’ve been spending more and more time in Adobe Launch. So far, I’m liking what I see – though I’m hoping the publish process gets ironed out a bit in coming months. But that’s not the focus of this post; rather, I wanted to describe my experience working with extensions in Launch. I recently authored my first extension – which offers a few very useful ways to integrate Adobe Target with other tools and extensions in Launch. You can find out more about it here, or ping me with any questions if you decide to add the extension to your Launch configuration. Next week I’ll try and write more about how you might something similar using any of the other major tag management systems. But for now, I’m more interested in how extension development works, and I’d like to share some of the things I learned along the way.

Extension Development is New (and Evolving) Territory for Adobe

The idea that Adobe has so freely opened up its platform to allow developers to share their own code across Adobe’s vast network of customers is admittedly new to me. After all, I can remember the days when Omniture/Adobe didn’t even want to open up its platform to a single customer, much less all of them. Remember the days of usage tokens for its APIs? Or having to pay for a consulting engagement just to get the code to use an advanced plugin like Channel Manager? So the idea that Adobe has opened things up to the point where I can write my own code within Launch, programmatically send it to Adobe, and have it then available for any Adobe customer to use – that’s pretty amazing. And for being so new, the process is actually pretty smooth.

What Works Well

Adobe has put together a pretty solid documentation section for extension developers. All the major topics are covered, and the Getting Started guide should help you get through the tricky parts of your first extension like authentication, access tokens, and uploading your extension package to the integration environment. One thing to note is that just about everything you define in your extension is a “type” of that thing, not the actual thing. For example, my extension exposes data from Adobe Target for use by other extensions. But I didn’t immediately realize that my data element definitions didn’t actually define new data elements for use in Launch; it only created a new “type” of data element in the UI that can then be used to create a data element. The same is true for custom events and actions. That makes sense now, but it took some getting used to.

During the time I spent developing my extension, I also found the Launch product team is working continuously to improve the process for us. When I started, the documentation offered a somewhat clunky process to retrieve an access token, zip my extension, and use a Postman collection to upload it. By the time I was finished, Adobe had released a Node package (npm) to basically do all the hard work. I also found the Launch product team to be incredibly helpful – they responded almost immediately to my questions on their Slack group. They definitely seem eager to build out a community as quickly as possible.

I also found the integration environment to be very helpful in testing out my extension. It’s almost identical to the production environment of Launch; the main difference is that it’s full of extensions in development by people just like me. So you can see what others are working on, and you can get immediate feedback on whether your extension works the way it should. There is even a fair amount of error logging available if you break something – though hopefully this will be expanded in the coming months.

What Could Work Better

Once I finished my extension, I noticed that there isn’t a real natural spot to document how your extension should work. I opted to put mine into the main extension view, even though there was no other configuration needed that would require such a view. While I was working on my extension, it was suggested that I put instructions in my Exchange listing, which doesn’t seem like a very natural place for it, either.

I also hope that, over time, Adobe offers an easier way to style your views to match theirs. For example, if your extension needs to know the name of a data element it should populate, you need a form field to collect this input. Making that form look the same as everything else in Launch would be ideal. I pulled this off by scraping the HTML and JavaScript from one Adobe’s own extensions and re-formatting it. But a “style toolkit” would be a nice addition to keep the user experience the same.

Lastly, while each of the sections in the Getting Started guide had examples, some of the more advanced topics could use some more additional exploration. For example, it took me a few tries to decide whether my extension would work better with a custom event type, or with just some custom code that triggered a direct call rule. And figuring out how to integrate with other extensions – how to access other extensions’ objects and code – wasn’t exactly easy and I still have some unanswered questions because I found a workaround and ended up not needing it.

Perhaps the hardest part of the whole process was getting my Exchange listing approved. The Exchange covers a lot of integrations beyond just Adobe Launch, some of which are likely far more complex than what mine does. A lot of the required images, screenshots, and details seemed like overkill – so a tiered approach to listings would be great, too.

What I’d Like to See Next

Extension development is in its infancy still, but one thing I hope is on the roadmap is the ability to customize an extension to work the way you need it. For a client I recently migrated, they used both Facebook and Pinterest, though the extensions didn’t work for their tag implementation. There were events and data they needed to capture that the extension didn’t support. I hope that in a future iteration, I’ll be able to “check out” an extension from the library and download the package, make it work the way I need and either create my own version of the extension or contribute to an update of someone else’s extension that the whole community can benefit from. The inability to customize tag templates has plagued every paid tag management solution except Tealium (which has supported it from the beginning) for years – in my opinion, it’s what turns tag management from a tool used primarily to deploy custom JavaScript into a powerful digital marketing toolbelt. It’s not something I’d expect so early in the game, but I hope it will be added soon.

In conclusion, my hat goes off to the Launch development team; they’ve come up with a really great way to build a collaborative community that pushes Launch forward. No initial release will ever be perfect, but there’s a lot to work with and a lot of opportunity for all of us in the future to shape the direction Launch takes and have some influence in how it’s adopted. And that’s an exciting place to be.

Photo Credit: Rod Herrea (Flickr)

0
Adobe Analytics, Featured, Tag Management, Technical/Implementation

A Coder’s Paradise: Notes from the Tech Track at Adobe Summit 2018

Posted on by Josh West

Last week I attended my 11th Adobe Summit – a number that seems hard to believe. At my first Summit back in 2008, the Great Recession was just starting, but companies were already cutting back on expenses like conferences – just as Omniture moved Summit from the Grand America to the Salt Palace (they moved it back in 2009 for a few more years). Now, the event has outgrown Salt Lake City – with over 13,000 attendees last week converging on Las Vegas for an event with a much larger footprint than just the digital analytics industry.

With the sheer size of the event and the wide variety of products now included in Adobe’s Marketing and Experience Clouds, it can be difficult to find the right sessions – but I managed to attend some great labs, and wanted to share some of what I learned. I’ll get to Adobe Launch, which was again under the spotlight – only this year, it’s actually available for customers to use. But I’m going to start with some of the other things that impressed me throughout the week. There’s a technical bent to all of this – so if you’re looking for takeaways more suited for analysts, I’m sure some of my fellow partners at Demystified (as well as lots of others out there) will have thoughts to share. But I’m a developer at heart, so that’s what I’ll be emphasizing.

Adobe Target Standard

Because Brian Hawkins is such an optimization wizard, I don’t spend as much time with Target as I used to, and this was my first chance to do much with Target Standard besides deploy the at.js library and the global mbox. But I attended a lab that worked through deploying it via Launch, then setting up some targeting on a singe-page ReactJS application. My main takeaway is that Target Standard is far better suited to running an optimization program on a single-page application than Classic ever was. I used to have to utilize nested mboxes and all sorts of DOM trickery to delay content from showing until the right moment when things actually took place. But with Launch, you can easily listen for page updates and then trigger mboxes accordingly.

Target Standard and Launch also makes it easier to handle a common issue with frameworks like ReactJS where the data layer is being asynchronously populated with data from API calls – so you can run a campaign on initial page load even if it takes some time for all the relevant targeting data to be available.

Adobe Analytics APIs

The initial version of the Omniture API was perhaps the most challenging API I’ve ever used. It supported SOAP only, and from authentication to query, you had to configure everything absolutely perfectly for it to work. And you had to do it with no API Explorer and virtually no documentation, all while paying very close attention to the number of requests you were making, since you only had 2,000 tokens per month and didn’t want to run out or get charged for more (I’m not aware this ever happened, but the threat at least felt real!).

Adobe adding REST API support a few years later was a career-changing event for me, and there have been several enhancements and improvements since, like adding OAUTH authentication support. But what I saw last week was pretty impressive nonetheless. The approach to querying data is changed significantly in the following ways:

  • The next iteration of Adobe’s APIs will offer a much more REST-ful approach to interacting with the platform.
  • Polling for completed reports is no longer required. It will likely take several more requests to get to the most complicated reports, but each individual request will run much faster.
  • Because Analytics Workspace is built on top of a non-public version of the API, you truly will be able to access any report you can find in the UI.
  • The request format for each report has been simplified, with non-essential parameters either removed or at least made optional.
  • The architecture of a report request is fundamentally different in some ways – especially in the way that breakdowns between reports work.
  • The ability to search or filter on reports is far more robust than in earlier versions of the API.

Launch by Adobe

While Launch has been available for a few months, I’ve found it more challenging than I expected to talk my clients into migrating from DTM to Launch. The “lottery” system made some of my clients wonder if Launch was really ready for prime-time, while the inability to quickly migrate an existing DTM implementation over to Launch has been prohibitive to others. But whatever the case may be, I’ve only started spending a significant amount of time in Launch in the last month or so. For customers who were able to attend labs or demos on Launch at Summit, I suspect that will quickly change – because the feature set is just so much better than with DTM.

How Launch Differs from DTM

My biggest complaint about DTM has always been that it hasn’t matched the rest of the Marketing Cloud in terms of enterprise-class features. From a limited number of integrations available, to the rigid staging/production publishing structure, I’ve repeatedly run into issues where it was hard to make DTM work the way I needed for some of my larger clients. Along the way, Adobe has repeatedly said they understood these limitations and were working to address them. And Launch does that – it seems fairly obvious now that the reason DTM lagged in offering features other systems did is because Adobe has been putting way more resources into Launch over the past few years. It opens up the platform in some really unique ways that DTM never has:

  • You can set up as many environments as you want.
  • Minification of JavaScript files is now standard (it’s still hard to believe this wasn’t the case with DTM).
  • Anyone can write extensions to enhance the functionality and features available.
  • The user(s) in charge of Launch administration for your company have much more granular control over what is eventually pushed to your production website.
  • The Launch platform will eventually offer open APIs to allow you to customize your company’s Launch experience in virtually any way you need.

With Great Power Comes Great Responsibility

Launch offers a pretty amazing amount of control that make for some major considerations to each company that implements it. For example, the publishing workflow is flexible to the point of being a bit confusing. Because it’s set up almost like a version control system like Git, any Launch user can set up his or her own development environment and configure in any number of ways. This means each user has to then choose which version of every single asset to include in a library, promote to staging/production, etc. So you have to be a lot more careful than when you’re publishing with DTM.

I would hope we’ve reached a point in tag management where companies no longer expect a marketer to be able to own tagging and the TMS – it was the sales pitch made from the beginning, but the truth is that it has never been that easy. Even Tealium, which (in my opinion) has the most user-friendly interface and the most marketer-friendly features, needs at least one good developer to tap into the whole power of the tool. Launch will be no different; as the extension library grows and more integrations are offered, marketers will probably feel more comfortable making changes than they were with DTM – but this will likely be the exception and not the rule.

Just One Complaint

If there is one thing that will slow migration from DTM to Launch, it is be the difficulty customers will face in migration. One of the promises Adobe made about Launch at Summit in 2017 was that you would be able to migrate from DTM to Launch without updating the embed code on your site. This is technically true – you can configure Launch to publish your production environment to an old DTM production publishing target. But this can only be done for production, and not any other environment – which means you can migrate without updating your production embed code, but you will need to update all your non-production codes. Alternatively, you can use a tool like DTM Switch or Charles Proxy for your testing – and that will work fine for your initial testing. But most enterprise companies want to accumulate a few weeks of test data for all the traffic on at least one QA site before they are comfortable deploying changes to production.

It’s important to point out that, even if you do choose to migrate by publishing your Launch configuration to your old production DTM publishing target, you still have to migrate everything currently in DTM over to Launch – manually. Later this year, Adobe has said that they will release a true migration tool that will allow customers to pull rules, data elements, and tags from a DTM property into a new Launch property and migrate them without causing errors. Short of such a tool, some customers will have to invest quite a bit to migrate everything they currently have in DTM over to Launch. Until then, my recommendation is to figure out the best migration approach for your company:

  1. If you have at least one rockstar analytics developer with some bandwidth, and a manageable set of rules and tags in DTM, I’d start playing around with migration in one of your development environments, and put together an actual migration plan.
  2. If you don’t have the resources yet, I’d probably wait for the migration tool to be available later in the year – but still start experimenting with Launch on smaller sites or as more resources become available.

Either way, for some of my clients that have let their DTM implementations get pretty unwieldy, moving from DTM to Launch offers a fresh start and a chance to upgrade to Adobe’s latest technology. No matter which of these two situations you’re in, I’d start thinking now (if you haven’t already) about how you’re going to get your DTM properties migrated to Launch. It is superior to DTM in nearly every way, and it is going to get nearly all of the development resources and roadmap attention from Adobe from here on out. You don’t need to start tomorrow – and if you need to wait for a migration tool, you’ll be fine. But if your long-term plan is to stay with DTM, you’re likely going to limit your ability in the future to tap into additional features, integrations and enhancements Adobe makes across its Marketing and Experience Cloud products.

Conclusion

We’ve come a long ways from the first Summits I attended, with only a few labs and very little emphasis on the technology itself. Whether it was new APIs, new product features announcements, or the hands-on labs, there was a wealth of great information shared at Summit 2018 for developers and implementation-minded folks like me – and hopefully you’re as excited as I am to get your hands on some of these great new products and features.

Photo Credit: Roberto Faccenda (Flickr)

0
Adobe Analytics, Featured, General, google analytics, Technical/Implementation

Can Local Storage Save Your Website From Cookies?

Posted on by Josh West

I can’t imagine that anyone who read my last blog post set a calendar reminder to check for the follow-up post I had promised to write, but if you’re so fascinated by cookies and local storage that you are wondering why I didn’t write it, here is what happened: Kevin and I were asked to speak at Observepoint’s inaugural Validate conference last week, and have been scrambling to get ready for that. For anyone interested in data governance, it was a really unique, and great event. And if you’re not interested in data governance, but you like outdoor activities like mountain biking, hiking, fly fishing, etc. – part of what made the event unique was some really great networking time outside of a traditional conference setting. So put it on your list of potential conferences to attend next year.

My last blog post was about some of the common pitfalls that my clients see that are caused by an over-reliance on cookies. Cookies are critical to the success of any digital analytics implementation – but putting too much information in them can even crash a customer’s experience. We talked about why many companies have too many cookies, and how a company’s IT and digital analytics teams can work together to reduce the impact of cookies on a website.

This time around, I’d like to take a look at another technology that is a potential solution to cookie overuse: local storage. Chances are, you’ve at least heard about local storage, but if you’re like a lot of my clients, you might not have a great idea of what it does or why it’s useful. So let’s dive into local storage: what it is, what it can (and can’t) do, and a few great uses cases for local storage in digital analytics.

What is Local Storage?

If you’re having trouble falling asleep, there’s more detail than you could ever hope to want in the specifications document on the W3C website. In fact, the W3C makes an important distinction and calls the actual feature “web storage,” and I’ll describe why in a bit. But most people commonly refer to the feature as “local storage,” so that’s how I’ll be referring to it as well.

The general idea behind local storage is this: it is a browser feature designed to store data in name/value pairs on the client. If this sounds a lot like what cookies are for, you’re not wrong – but there are a few key differences we should highlight:

  • Cookies are sent back and forth between client and server on all requests in which they have scope; but local storage exists solely on the client.
  • Cookies allow the developer to manage expiration in just about any way imaginable – by providing an expiration timestamp, the cookie value will be removed from the client once that timestamp is in the past; and if no timestamp is provided, the cookie expires when the session ends or the browser closes. On the other hand, local storage can support only 2 expirations natively – session-based storage (through a DOM object called sessionStorage), and persistent storage (through a DOM object called localStorage). This is why the commonly used name of “local storage” may be a bit misleading. Any more advanced expiration would need to be written by the developer.
  • The scope of cookies is infinitely more flexible: a cookie could have the scope of a single directory on a domain (like http://www.analyticsdemystified.com/blogs), or that domain (www.analyticsdemystified.com), or even all subdomains on a single top-level domain (including both www.analyticsdemystified.com and blog.analyticsdemystified.com). But local storage always has the scope of only the current subdomain. This means that local storage offers no way to pass data from one subdomain (www.analyticsdemystified.com) to another (blog.analyticsdemystified.com).
  • Data stored in either localStorage or sessionStorage is much more easily accessible than in cookies. Most sites load a cookie-parsing library to handle accessing just the name/value pair you need, or to properly decode and encode cookie data that represents an object and must be stored as JSON. But browsers come pre-equipped to make saving and retrieving storage data quick and easy – both objects come with their own setItem and getItem methods specifically for that purpose.

If you’re curious what’s in local storage on any given site, you can find out by looking in the same place where your browser shows you what cookies it’s currently using. For example, on the “Application” tab in Chrome, you’ll see both “Local Storage” and “Session Storage,” along with “Cookies.”

What Local Storage Can (and Can’t) Do

Hopefully, the points above help clear up some of the key differences between cookies and local storage. So let’s get into the real-world implications they have for how we can use them in our digital analytics efforts.

First, because local storage exists only on the client, it can be a great candidate for digital analytics. Analytics implementations reference cookies all the time – perhaps to capture a session or user ID, or the list of items in a customer’s shopping cart – and many of these cookies are essential both for server- and client-side parts of the website to function correctly. But the cookies that the implementation sets on its own are of limited value to the server. For example, if you’re storing a campaign ID or the number of pages viewed during a visit in a cookie, it’s highly unlikely the server would ever need that information. So local storage would be a great way to get rid of a few of those cookies. The only caveat here is that some of these cookies are often set inside a bit of JavaScript you got from your analytics vendor (like an Adobe Analytics plugin), and it could be challenging to rewrite all of them in a way that leverages local storage instead of cookies.

Another common scenario for cookies might be to pass a session or visitor ID from one subdomain to another. For example, if your website is an e-commerce store that displays all its products on www.mystore.com, and then sends the customer to shop.mystore.com to complete the checkout process, you may use cookies to pass the contents of the customer’s shopping cart from one part of the site to another. Unfortunately, local storage won’t help you much here – because, unlike cookies, local storage offers no way to pass data from one subdomain to another. This is perhaps the greatest limitation of local storage that prevents its more frequent use in digital analytics.

Use Cases for Local Storage

The key takeaway on local storage is that there are 2 primary limitations to its usefulness:

  • If the data to be stored is needed both on the client/browser and the server, local storage does not work – because, unlike cookies, local storage data is not sent to the server on each request.
  • If the data to be stored is needed on multiple subdomains, local storage also does not work – because local storage is subdomain-specific. Cookies, on the other hand, are more flexible in scope – they can be written to work across multiple subdomains (or even all subdomains on the same top-level domain).

Given these considerations, what are some valid use cases when local storage makes sense over cookies? Here are a few I came up with (note that all of these assume that neither limitation above is a problem):

  • Your IT team has discovered that your Adobe Analytics implementation relies heavily on several cookies, several of which are quite large. In particular, you are using the crossVisitParticipation plugin to store a list of each visit’s traffic source. You have a high percentage of return visitors, and each visit adds a value to the list, which Adobe’s plugin code then encodes. You could rewrite this plugin to store the list in the localStorage object. If you’re really feeling ambitious, you could override the cookie read/write utilities used by most Adobe plugins to move all cookies used by Adobe (excluding visitor ID cookies of course) into localStorage.
  • You have a session-based cookie on your website that is incremented by 1 on each page load. You then use this cookie in targeting offers based on engagement, as well as invites to chat and to provide feedback on your site. This cookie can very easily be removed, pushing the data into the sessionStorage object instead.
  • You are reaching the limit to the number of Adobe Analytics server calls or Google Analytics hits before you bump up to the next pricing tier, but you have just updated your top navigation menu and need to measure the impact it’s having on conversion. Using your tag management system and sessionStorage, you could “listen” for all navigation clicks, but instead of tracking them immediately, you could save the click information and then read it on the following page. In this way, the click data can be batched up with the regular page load tracking that will occur on the following page (if you do this, make sure to delete the element after using it, so you can avoid double-tracking on subsequent pages).
  • You have implemented a persistent shopping cart on your site and want to measure the value and contents of a customer’s shopping cart when he or she arrives on your website. Your IT team will not be able to populate this information into your data layer for a few months. However, because they already implemented tracking of each cart addition and removal, you could easily move this data into a localStorage object on each cart interaction to help measure this.

All too often, IT and analytics teams resort to the “just stick it in a cookie” approach. That way, they justify, we’ll have the data saved if it’s ever needed. Given some of the limitations I talked about in my last post, we should all pay close attention to the number, and especially the size, of cookies on our websites. Not doing so can have a very negative impact on user experience, which in turn can have painful implications for your bottom line. While not perfect for every situation, local storage is a valuable tool that can be used to limit the number of cookies used by your website. Hopefully this post has helped you think of a few ways you might be able to use local storage to streamline your own digital analytics implementation.

Photo Credit: Michael Coghlan (Flickr)

0
Adobe Analytics, Featured, google analytics, Technical/Implementation

Don’t Let Cookies Eat Your Site!

Posted on by Josh West

A few years ago, I wrote a series of posts on how cookies are used in digital analytics. Over the past few weeks, I’ve gotten the same question from several different clients, and I decided it was time to write a follow-up on cookies and their impact on digital analytics. The question is this: What can we do to reduce the number of cookies on our website? This follow-up will be split into 2 separate posts:

  1. Why it’s a problem to have too many cookies on your website, and how an analytics team can be part of the solution.
  2. When local storage is a viable alternative to cookies.

The question I described in the introduction to this post is usually posed to me like this: An analyst has been approached by someone in IT that says, “Hey, we have too many cookies on our website. It’s stopping the site from working for our customers. And we think the most expendable cookies on the site are those being used by the analytics team. When can you have this fixed?” At this point, the client frantically reaches out to me for help. And while there are a few quick suggestions I can usually offer, it usually helps to dig a little deeper and determine whether the problem is really as dire as it seems. The answer is usually no – and, surprisingly, it is my experience that analytics tools usually contribute surprisingly little to cookie overload.

Let’s take a step back and identify why too many cookies is actually a problem. The answer is that most browsers put a cap on the maximum size of the cookies they are willing to pass back and forth on each network request – somewhere around 4KB of data. Notice that the limit has nothing to do with the number of cookies, or even the maximum size of a single cookie – it is the total size of all cookies sent. This can be compounded by the settings in place on a single web server or ISP, that can restrict this limit even further. Individual browsers might also have limits on the total number of cookies allowed (a common maximum number is 50) as well as the maximum size of any one cookie (usually that same 4KB size).

The way the server or browser responds to this problem varies, but most commonly it’s just to return a request error and not send back the actual page. At this point it becomes easy to see the problem – if your website is unusable to your customers because you’re setting to many cookies that’s a big problem. To help illustrate the point further, I used a Chrome extension called EditThisCookie to find a random cookie on a client’s website, and then add characters to that cookie value until it exceeded the 4KB limit. I then reloaded the page, and what I saw is below. Cookies are passed as a header on the request – so, essentially, this message is saying that the request header for cookies was longer than what the server would allow.

At this point, you might have started a mental catalog of the cookies you know your analytics implementation uses. Here are some common ones:

  • Customer and session IDs
  • Analytics visitor ID
  • Previous page name (this is a big one for Adobe users, but not Google, since GA offers this as a dimension out of the box)
  • Order IDs and other values to prevent double-counting on page reloads (Adobe will only count an order ID once, but GA doesn’t offer this capability out of the box)
  • Traffic source information, sometimes across multiple visits
  • Click data you might store in a cookie to track on the following page, to minimize hits
  • You’ve probably noticed that your analytics tool sets a few other cookies as well – usually just session cookies that don’t do much of anything useful. You can’t eliminate them, but they’re generally small and don’t have much impact on total cookie size.

If your list looks anything like this, you may be wondering why the analytics team gets a bad rap for its use of cookies. And you’d be right – I have yet to have a client ask me the question above that ended up being the biggest offender in terms of cookie usage on the site. Most websites these days are what I might call “Frankensteins” – it becomes such a difficult undertaking to rebuild or update a website that, over time, IT teams tend to just bolt on new functionality and features without ever removing or cleaning up the old. Ask any developer and they’ll tell you they have more tech debt than they can ever hope to clean up (for the non-developers out there, “tech debt” describes all the garbage left in your website’s code base that you never took the time to clean up; because most developers prefer the challenge of new development to the tediousness of cleaning up old messes, and most marketers would rather have developers add new features anyway, most sites have a lot of tech debt).  If you take a closer look at the cookies on your site, you’d probably find all sorts of useless data being stored for no good reason. Things like the last 5 URLs a visitor has seen, URL-encoded twice. Or the URL for the customer’s account avatar being stored in 3 different cookies, all with the same name and data – one each for mysite.com,  www.mysite.com, and store.mysite.com. Because of employee turnover and changing priorities, a lot of the functionality on a website are owned by different development on the same team – or even different teams entirely. It’s easy for one team to not realize that the data it needs already exists in a cookie owned by another team – so a developer just adds a new cookie without any thought of the future problem they’ve just added to.

You may be tempted to push back on your IT team and say something like, “Come talk to me when you solve your own problems.” And you may be justified in thinking this – most of the time, if IT tells the analytics team to solve its cookie problem, it’s a little like getting pulled over for drunk driving and complaining that the officer should have pulled over another driver for speeding instead while failing your sobriety test. But remember 2 things (besides the exaggeration of my analogy – driving while impaired is obviously worse than overusing cookies on your website):

  1. A lot of that tech debt exists because marketing teams are loathe to prioritize fixing bugs when they could be prioritizing new functionality.
  2. It really doesn’t matter whose fault it is – if your customers can’t navigate your site because you are using too many cookies, or your network is constantly weighed down by the back-and-forth of unnecessary cookies being exchanged, there will be an impact to your bottom line.

Everyone needs to share a bit of the blame and a bit of the responsibility in fixing the problem. But it is important to help your IT team understand that analytics is often just the tip of the iceberg when it comes to cookies. It might seem like getting rid of cookies Adobe or Google sets will solve all your problems, but there are likely all kinds of cleanup opportunities lurking right below the surface.

I’d like to finish up this post by offering 3 suggestions that every company should follow to keep its use of cookies under control:

Maintain a cookie inventory

Auditing the use of cookies frequently is something every organization should do – at least annually. When I was at salesforce.com, we had a Google spreadsheet that cataloged our use of cookies across our many websites. We were constantly adding and removing the cookies on that spreadsheet, and following up with the cookie owners to identify what they did and whether they were necessary.

One thing to note when compiling a cookie inventory is that your browser will report a lot of cookies that you actually have no control over. Below is a screenshot from our website. You can see cookies not only from analyticsdemystified.com, but also linkedin.com, google.com, doubleclick.net, and many other domains. Cookies with a different domain than that of your website are third-party, and do not count against the limits we’ve been talking about here (to simplify this example, I removed most of the cookies that our site uses, leaving just one per unique domain). If your site is anything like ours, you can tell why people hate third-party cookies so much – they outnumber regular cookies and the value they offer is much harder to justify. But you should be concerned primarily with first-party cookies on your site.

Periodically dedicate time to cookie cleanup

With a well-documented inventory your site’s use of cookies in place, make sure to invest time each year to getting rid of cookies you no longer need, rather than letting them take up permanent residence on your site. Consider the following actions you might take:

  • If you find that Adobe has productized a feature that you used to use a plugin for, get rid of it (a great example is Marketing Channels, which has essentially removed the need for the old Channel Manager plugin).
  • If you’re using a plugin that uses cookies poorly (by over-encoding values, etc.), invest the time to rewrite it to better suit your needs.
  • If you find the same data actually lives in 2 cookies, get the appropriate teams to work together and consolidate.

Determine whether local storage is a viable alternative

This is the real topic I wanted to discuss – whether local storage can solve the problem of cookie overload, and why (or why not). Local storage is a specification developed by the W3C that all modern browsers have now implemented. In this case, “all” really does mean “all” – and “modern” can be interpreted as loosely as you want, since IE8 died last year and even it offered local storage. Browsers with support for local storage offer developers the ability to store that is required by your website or web applicaiton, in a special location, and without the size and space limitations imposed by cookies. But this data is only available in the browser – it is not sent back to the server. That means it’s a natural consideration for analytics purposes, since most analytics tools are focused on tracking what goes on in the browser.

However, local storage has limitations of its own, and its strengths and weaknesses really deserve their own post – so I’ll be tackling it in more detail next week. I’ll be identifying specific uses cases that local storage is ideal for – and others where it falls short.

Photo Credit: Karsten Thoms

0
Featured, Tag Management, Technical/Implementation

A Developer’s Perspective on Features Every Tag Management System Should Offer

Posted on by Josh West

Almost 2 years ago I wrote a blog post on some of the major questions companies face when choosing a tag management system. I very carefully crafted a post that talked about the main trends in the industry and some of the strengths of the major products in the industry. I wasn’t prepared for the response that post received, though in hindsight I should have been: I heard from nearly all the major TMS companies, and each seemed to feel a lot more strongly about any perceived weaknesses I mentioned than about any of the strengths. But that post taught me an important lesson about the weight that a Demystified opinion can carry throughout the analytics community, and about the competitive nature of the tag management space.

Since then, I have chosen my words very carefully when mentioning any of the 5 leading tag management systems. I always preface my comments by saying that we have clients using all of them, and doing so quite successfully. I even refer to the companies by listing them in alphabetical order, and then explain the reason for the order I have chosen – lest anyone think it’s an unofficial ranking of my fondness for any of them (in this regard, DTM benefited a lot more from its acquisition by Adobe than Signal did by rebranding!).

However, seeing how I lead Demystified’s tag management practice, it’s probably time to dangle a toe back in treacherous water. I’d like to provide a list of what I call “essential features” that any tag management system should offer. In some cases, the feature is offered by all of them, and in others, by only one or two – but I will leave you to research that, rather than pointing it out for you. A few caveats before I get started:

  • You’ll find no mention at all of delivery network (100% client-side versus client/server hybrid). I find that both approaches offer such a dramatic improvement in page performance over traditional tagging that I have little interest in picking nits one way or the other.
  • I feel similarly about the synchronous/asynchronous argument as well. There are compelling reasons for both, but you can deploy any system either way (though it may go against the vendor’s best practices). Just remember to make it clear to each vendor you talk to if you plan on deploying synchronous tags (like an A/B testing tag) through their system, and find out whether such tags are supported, and any special considerations for implementing them.

Creating a list like this is a bit tricky because some of the tools are free. While it’s obviously much more palatable to forego a particular feature when you’re not paying for the tool, there are some features that are important enough to me that I’d have to have them whether the tool was free or not. Without further ado, here is my list of essential tag management features:

1. Support for versioning and rollbacks. This is the absolute most important feature any tag management system has to support. Any time you’re dealing with remotely hosted JavaScript, you must be able to remove a rogue vendor’s code at a moment’s notice. Don’t assume that an annoying error or warning icon in the corner of the browser is your worst-case scenario; a broken user experience or a cross-site scripting error could carry a huge cost to your business. The closer the system comes to the version control employed by IT teams, the better – the ability to stage changes, save them for a later release while still deploying new tags that are ready and tested, or revert to a previous version in production without completely losing those changes in your development environment are incredibly valuable features. I’ve seen people lose their jobs because the wrong content ended up on a website at the wrong time, and a company just couldn’t remove it fast enough.

2. Customizable integrations with your most common tags. If you’re new to tag management, you probably don’t realize what goes on under the hood when you implement a tag in the TMS. If the TMS offers a supported tag integration with a particular vendor (often called a template, tool, or app), that integration generates a block of JavaScript that represents what the “average” company uses to implement the vendor’s tag. Most of the time, that’s all you need – you just specify which pages the tag goes on, where to find the data the tag needs, and you’re done. But the ability to customize that code block when you need to is important – because every once in awhile, you’re bound to want to work with a vendor in a slightly different way than everyone else does – and the alternative to customizing the integration offered by the TMS is just to write your own completely. Isn’t that one of the main problems you were trying to solve in the first place.

3. The ability to handle frequent, repetitive tasks without a developer. The original promise of tag management was that you could add third-party tags to your site without a developer. The past few years have proven the fallacy of that idea – but it sure is nice to let your marketers make basic changes to tags. If you decide you want to capture the page title in an Adobe eVar, or that you need to pass the product name to Adwords or DFA, those are simple changes you shouldn’t have to send to a developer. It should be easy to get data you already have (and have already configured in your TMS) to other vendors that want it.

4. The ability to send the same data in a slightly different format with little effort. If you’ve spent even the slightest time looking at what data you’re actually sending to your tag vendors, you’ve seen some common threads. They all want the same things: which products a customer purchased, how much they paid, the unique ID generated to a web lead in your CRM, and so on. But they likely want this data in a different format: one vendor may want a list of products delimited by a comma, and another may want them delimited by a pipe. A good TMS has integrations that don’t require you to customize the format of all this common data – it will do it for you.

5. Support for events, interactions, and data that changes throughout a page’s lifecycle. We’re moving from the world of multi-page to single-page web apps. While your TMS likely offers you a way to track these interactions, look for one whose API most closely matches the model used by your IT team – whether it’s JavaScript you’ll have them add when interactions occur, or the ability to “listen” for events they’re already capturing. And make sure that as you configure data elements in the system, those elements can be updated throughout the page’s lifecycle – it’s incredibly annoying to have to develop a hack to something as common as that an unknown user authenticates on your site using Ajax, but your TMS doesn’t know because when the page first loaded the user was still anonymous. Your development team will be more supportive of tag management if they feel the tool supports them – rather than the other way around.

6. Consideration for tag optimization and caching. Besides decoupling your IT release process from your digital marketing and tagging effort, it’s possible that the greatest potential benefit in migrating to a tag management system is the improvement it provides to your website’s performance. But the TMS should allow you the flexibility to fine-tune that performance benefit by loading only the code and logic required for the tags on that page, rather than loading the code and logic that could be required for all tags used across your site. Even if all that logic is cached, it still needs to be run on page after page after page. In other words, there’s no reason for your homepage to load code that doesn’t actually need to run until it’s time to fire order confirmation tags. I also love it when a system allows you to cache code and reuse it throughout the site when you need the same basic tag throughout your site. If you load a Doubleclick tag on 50 pages on your site, and the only difference is the ‘type’ or ‘cat’ parameter, there’s no reason for the TMS to reload an uncached version of that logic on all 50 pages – load it once and have it run again and again from the browser cache. If the TMS allows you to manage those subtle differences in a single place rather than in 50 different tags, this also offers a huge benefit to the folks managing your tags, who now can support a single tag implementation instead of 50. Even small optimization features can make the end-users of your TMS and your website very happy.

So if you’re new to tag management, hopefully this list helps you choose the tool that will be the best fit with your organization. And if you adopted tag management earlier, hopefully it it helps you make sure you’ve got the right system in place – and the right processes to manage it. I’ve tried to come up with a list of features that will appeal to both developer and marketer end users, because both play an important part in a company’s digital marketing efforts. And in the end, that’s what tag management is really about – all these tags serve no useful purpose on your website if they’re not allowing you to run your online business more effectively.

Photo Credit: Bill Dickinson (Flickr)

0
Adobe Analytics, Featured, google analytics, Technical/Implementation

The Hard Truth About Measuring Page Load Time

Posted on by Josh West

Page load performance should be every company’s #1 priority with regard to its website – if your website is slow, it will affect all the KPIs that outrank it. Several years ago, I worked on a project at salesforce.com to improve page load time, starting with the homepage and all the lead capture forms you could reach from the homepage. Over the course of several months, we refactored our server-side code to run and respond faster, but my primary responsibility was to optimize the front-end JavaScript on our pages. This was in the early days of tag management, and we weren’t ready to invest in such a solution – so I began sifting through templates, compiling lists of all the 3rd-party tags that had been ignored for years, talking to marketers to find out which of those tags they still needed, and then breaking them down to their nitty-gritty details to consolidate them and move them into a single JavaScript library that would do everything we needed from a single place, but do it much faster. In essence, it was a non-productized, “mini” tag management system.

Within 24 hours of pushing the entire project live, we realized it had been a massive success. The difference was so noticeable that we could tell the difference without having all the data to back it up – but the data eventually told us the exact same story. Our monitoring tool was telling us our homepage was loading nearly 50% faster than before, and even just looking in Adobe at our form completion rate (leads were our lifeblood), we could see a dramatic improvement. Our data proved everything we had told people – a faster website couldn’t help but get us more leads. We hadn’t added tags – we had removed them. We hadn’t engaged more vendors to help us generate traffic – we were working with exactly the same vendors as before. And in spite of some of the marketing folks being initially hesitant about taking on a project that didn’t seem to have a ton of business value, we probably did more to benefit the business than any single project during the 3 1/2 years I worked there.

Not every project will yield such dramatic results – our page load performance was poor enough that we had left ourselves a lot of low-hanging fruit. But the point is that every company should care about how their website performs. At some point, almost every client I work with asks me some variation of the following question: “How can I measure page load time with my analytics tool?” My response to this question – following a cringe – is almost always, “You really can’t – you should be using another tool for that type of analysis.” Before you stop reading because yet another tool is out of the question, note that later on in this post I’ll discuss how your analytics tool can help you with some of the basics. But I think it’s important to at least acknowledge that the basics are really all those tools are capable of.

Even after several years of hearing this question – and several enhancements both to browser technology and the analytics tools themselves – I still believe that additional tools are required for robust page load time measurement. Any company that relies on their website as a major source of revenue, leads, or even just brand awareness has to invest in the very best technologies to help that website be as efficient as possible. That means an investment not just in analytics and optimization tools, but performance and monitoring tools as well. At salesforce.com, we used Gomez – but there are plenty of other good services as well that can be used on a small or large scale. Gomez and Keynote both simulate traffic to your site using any several different test criteria like your users’ location, browser, and connection speed. Other tools like SOASTA actually involve real user testing along some of the same dimensions. Any of these tools are much more robust than some of the general insight you might glean from your web analytics tool – they provide waterfall breakdowns and allow you to isolate where your problems come from and not just that they exist. You may find that your page load troubles only occur at certain times of the day or in certain parts of the world, or that they are happening in a particular leg of the journey. Maybe it’s a specific third-party tag or a JavaScript error that you can easily fix. In any case, these are the types of problems your web analytics tool will struggle to help you solve. The data provided by these additional tools is just much more actionable and helpful in identifying and solving problems.

The biggest problem I’ve found in getting companies to adopt these types of tools is often more administrative than anything. Should marketing or IT manage the tool? Typically, IT is better positioned to make use of the data and act on it to make improvements, but marketing may have a larger budget. In a lot of ways, the struggles are similar to those many of my clients encounter when selecting and implementing a tag management system. So you might find that you can take the learnings you gleaned from similar “battles” to make it easier this time. Better yet, you might even find that one team within your company already has a license you can use, or that you can team up to share the cost. However, if your company isn’t quite ready yet to leverage a dedicated tool, or you’re sorting through red tape and business processes that are slowing things down, let’s discuss some things you can do to get some basic reporting on page load time using the tools you’re already familiar with.

Anything you do within your analytics tool will likely be based on the browser’s built-in “timing” object. I’m ashamed to admit that up until recently I didn’t even realize this existed – but most browsers provide a built-in object that provides timestamps of the key milestone events of just about every part of a page’s lifecycle. The object is simply called “performance.timing” and can be accessed from any browser’s console. Here are some of the useful milestones you can choose from:

  • redirectStart and redirectEnd: If your site uses a lot of redirects, it could definitely be useful to include that in your page load time calculation. I’ve only seen these values populated in rare cases – but they’re worth considering.
  • fetchStart: This marks the time when the browser first starts the process of loading the next page.
  • requestStart: This marks the time when the browser requests the next page, either from a remote server or from its local cache.
  • responseEnd: This marks the time when the browser downloads the last byte of the page, but before the page is actually loaded into the DOM for the user.
  • domLoading: This marks the time when the browser starts loading the page into the DOM.
  • domInteractive: This marks the time when enough of the page has loaded for the user to begin interacting with it.
  • domContentLoaded: This marks the time when all HTML and CSS are parsed into the DOM. If you’re familiar with jQuery, this is basically the same as jQuery’s “ready” event (“ready” does a bit more, but it’s close enough).
  • domComplete: This marks the time when all images, iframes, and other resources are loaded into the DOM.
  • loadEventStart and loadEventEnd: These mean that the window’s “onload” event has started (and completed), and indicate that the page is finally, officially loaded.

JavaScript timing object

There are many other timestamps available as part of the “performance” object – these are only the ones that you’re most likely to be interested in. But you can see how it’s important to know which of these timestamps correspond to the different reports you may have in your analytics tool, because they mean different things. If your page load time is measured by the “loadEventEnd” event, the data probably says your site loads at least a few hundred milliseconds slower than it actually appears to your users.

The major limitation to using JavaScript timing is exactly what you’d expect: cross-browser compatibility. While IE8 is (finally!) a dying browser, it has not historically been the only one to lack support – mobile Safari has been a laggard as well as well. However, as of late 2015, iOS now supports this feature. Since concern for page load time is even more important for mobile web traffic, and since iOS is still the leader in mobile traffic for most websites, this closes what has historically been a pretty big gap. When you do encounter an older browser, the only way to fill this gap accurately for browsers lacking timing support is to have your development team write its own timestamp as soon as the server starts building the page. Then you can create a second timestamp when your tags fire, subtract the difference, and get pretty close to what you’re looking for. This gets a bit tricky, though, if the server timezone is different than the browser timezone – you’ll need to make sure that both timestamps are always in the same timezone.

This functionality is actually the foundation of both Adobe Analytics’ getLoadTime plugin and Google Analytics’ Site Speed reports. Both have been available for years, and I’ve been suspicious of them since I first saw them. The data they provide is generally sound, but there are a few things to be aware of if you’re going to use them – beyond just the lack of browser support I described earlier.

Adobe’s getLoadTime Plugin

Adobe calculates the start time using the most accurate start time available: either the browser’s “requestStart” time or a timestamp they ask you to add to the top of the page for older browsers. This fallback timestamp is unfortunately not very accurate – it doesn’t indicate server time, it’s just the time when the browser got to that point in loading the page. That’s likely to be at least a second or two later than when the whole process started, and is going to make your page load time look artificially fast. The end time is when the tag loads – not when the DOM is ready or the page is ready for user interaction.

When the visitor’s browser is a modern one supporting built-in performance timing, the data provided by Adobe is presented as a series of numbers (in milliseconds) that the page took to “load.” That number can be classified into high-level groups, and it can be correlated to your Pages report to see which pages load fastest (or slowest). Or you can put that number into a custom event that can be used in calculated metrics to measure the average time a given page takes to load.

Adobe Analytics page load time report

Google’s Site Speed Reports

Google’s reports, on the other hand, don’t have any suspect handling of older browsers – the documentation specifically states that the reports only work for browsers that support the native performance timing object. But Google’s reports are averages based on a sampling pool of only 1% of your visitors (which can be increased) – but you can see how a single visitor making it into that small sample from a far-flung part of the world could have a dramatic impact on the data Google reports back to you. Google’s reports do have the bonus of taking into account many other timing metrics the browser collects besides just the very generic interpretation of load time that Adobe’s plugin offers.

Google Analytics page load time report

As you can see, neither tool is without its flaws – and neither is very flexible in giving you control over which time metrics their data is based on. If you’re using Adobe’s plugin, you might have some misgivings about their method of calculation – and if you’re using Google’s standard reports, that sampling has likely led you to cast a suspicious eye on those reports when you’ve used them in the past. So what do you do if you need more than that? The only real answer is to take matters into your own hands. But don’t worry – the actual code is relatively simple and can be implemented with minimal development effort, and it can be done right in your tag management system of choice. Below is a quick little code snippet you can use as a jumping-off point to capture the page load time on each page of your website using built-in JavaScript timing. 

	function getPageLoadTime() {
		if (typeof(performance) !== 'undefined' && typeof(performance.timing) == 'object') {
			var timing = performance.timing;
			
			// fall back to less accurate milestones
			var startTime = performance.timing.redirectStart ||
					performance.timing.fetchStart ||
					performance.timing.requestStart;
			var endTime = performance.timing.domContentLoadedEventEnd ||
					performance.timing.domInteractive ||
					performance.timing.domComplete ||
					performance.timing.loadEventEnd;
			
			if (startTime && endTime && (startTime < endTime)) {
				return (endTime - startTime);
			}
		}
		
		return 'data not available';
	}

You don’t have to use this code exactly as I’ve written it – but hopefully it shows you that you have a lot of options to do some quick page load time analysis, and you can come up with a formula that works best for your own site. You (or your developers) can build on this code pretty quickly if you want to focus on different timing events or add in some basic support for browsers that don’t support this cool functionality. And it’s flexible enough to allow you to decide whether you’ll use a dimensions/variables or metrics/events to collect this data (I’d recommend both).

In conclusion, there are some amazing things you can do with modern browsers’ built-in JavaScript timing functionality, and you should do all you can to take advantage of what it offers – but always keep in mind that there are limitations to this approach. Even though additional tools that offer dedicated monitoring services carry an additional cost, they are equipped to encompass the entire page request lifespan and can provide much more actionable data. Analytics tools allow you to scratch the surface and identify that problems exist with your page load time – but they will always have a difficult time identifying what those problems are and how to solve them. The benefit of such tools can often be felt across many different groups within your organization – and sometimes the extra cost can be shared the same way. Page load time is an important part of any company’s digital measurement strategy – and it should involve multiple tools and collaboration within your organization.

Photo Credit: cod_gabriel (Flickr)

0
Adobe Analytics

Adobe’s new Marketing Cloud Visitor ID: How Does it Work?

Posted on by Josh West

A few months ago, I wrote a series of posts about cookies – how they are used in web analytics, and how Google and Adobe (historically) identify your web visitors. Those two topics set the stage for a discussion on Adobe’s current best practices approach for visitor identification.

You’ll remember that Adobe has historically used a cookie called “s_vi” to identify visitors to your site. This cookie is set by Adobe’s servers – meaning that by default it is third-party. Many Adobe customers have gone through the somewhat tedious process of allowing Adobe to set that cookie from one of their own subdomains, making it first-party. This is done by having your network operations team update its DNS settings to assign that subdomain to Adobe, and by purchasing (and annually maintaining) an SSL certificate for Adobe to use. If that sounds like a pain to you, you’re not alone. I remember having to go through the process when I worked at salesforce.com – because companies rightly take their websites, networks, and security seriously, what is essentially 5 minutes of actual work took almost 3 months!

So a few years back, Adobe came up with another alternative I discussed a few months ago – the “s_fid” cookie. This is a fallback visitor ID cookie set purely in JavaScript, used when a browser visiting a website still on third-party cookies rejected Adobe’s cookie. That was nice, but it wasn’t a very publicized change, and most analysts may not even know it exists. That may be because, at the time it happened, Adobe already had something better in the works.

The next change Adobe introduced – and, though it happened well over a year ago, only now am I starting to see major traction – was built on top of the Demdex product they acquired a few years ago, now known as Adobe Audience Manager (AAM). AAM is the backbone for identifying visitors using its new “Marketing Cloud” suite, and the Marketing Cloud Visitor ID service (AMCV) is the new best-practice for identifying visitors to your website. Note that you don’t need to be using Audience Manager to take advantage of the Visitor ID service – the service is available to all Adobe customers.

The really great thing about this new approach is that it represents something that Adobe customers have been hoping for for years – a single point of visitor identification. The biggest advantage a company gains in switching to this new approach is a way to finally, truly integrate some of Adobe’s most popular products. Notice that I didn’t say all Adobe products – but things are finally moving in that direction. The idea here is that if you implement the Marketing Cloud Visitor ID Service, and then upgrade to the latest code versions for tools like Analytics and Target, they’ll all be using the same visitor ID, which makes for a much smoother integration of your data, your visitor segments, and so on. One caveat is that while the AMCV has been around for almost 2 years, it’s been a slow ramp-up for companies to implement it. It’s a bit more challenging than a simple change to your s_code.js or mbox.js files. And even if you get that far, it’s then an additional challenge to migrate to the latest version of Target that is compatible with AMCV – a few of my clients that have tried doing it have hit some bumps in the road along the way. The good news is that it’s a major focus of Adobe’s product roadmap, which means those bumps in the road are getting smoothed out pretty quickly.

So, where to begin? Let’s start with the new cookie containing your Marketing Cloud Visitor ID. Unlike Adobe’s “s_vi” cookie, this cookie value is set with JavaScript and will always be first-party to your site. However, unlike Google’s visitor ID cookie, it’s not set exclusively with logic in the tracking JavaScript. When your browser loads that JavaScript, Adobe sends off an additional request to its AAM servers. What comes back is a bit of JavaScript that contains the new ID, which the browser can then use to set its own first-party cookie. But there is an extra request added in (at least on the first page load) that page-load time and performance fanatics will want to be aware of.

The other thing this extra request does is allow Adobe to set an additional third-party cookie with the same value, which it will do if the browser allows. This cookie can then be used if your site spans multiple domains, allowing you to use the same ID on each one of your sites. Adobe’s own documentation says this approach will only work if you’ve set up a first-party cookie subdomain with them (that painful process I discussed earlier). One of the reasons I’ve waited to write this post is that it took awhile for a large enough client, with enough different sites, to be ready to try this approach out. After a lot of testing, I can say that it does work – but that since it is based on that initial third-party cookie, it’s a bit fragile. It works best for brand-new visitors that have no Adobe cookies on any of your website. If you test it out, you’re likely to see most visits to your websites work just like you hoped – and a few where you still get a new ID, instead of the one stored in that third-party cookie. There’s a pretty crazy flow chart that covers the whole process here if you’re more of a visual learner.

Adobe has a lot of information available to help you migrate through this process successfully, and I don’t want to re-hash it here. But the basics are as follows:

  1. Request from Adobe that they enable your account for the Marketing Cloud, and send you your new “Org ID.” This uniquely identifies your company and ensures your visitors get identified correctly.
  2. If you’re using (or want to use) first-party cookies via a CNAME, make sure your DNS records point to Adobe’s latest regional data center (RDC) collection servers. You can read about the details here.
  3. If your migration is going to take time (like if you’re not using tag management or can’t update all your different implementations or sites at the same time), work with Adobe to configure a “grace period” for the transition process.
  4. Update your Analytics JavaScript code. You can use either AppMeasurement or H code – as long as you’re using the latest version.
  5. Deploy the new VisitorAPI JavaScript library. This can happen at the same time you deploy your Analytics code if you want.
  6. Test. And then test again. And just to be safe, test one more time – just to make sure the data being sent back to Adobe looks like you expect it to.

Once you finish, you’re going to see something like this in the Adobe Debugger:

adobe-debugger-amcv

 

There are two things to notice here. The first is a request for Adobe Audience Manager, and you should see your Marketing Cloud Org ID in it. The other is a new parameter in the Analytics request called “mid” that contains your new Marketing Cloud Visitor ID. Chances are, you’ll see both those. Easy, right? Unfortunately, there’s one more thing to test. After helping a dozen or so of my clients through this transition, I’ve seen a few “gotchas” pop up more than once. The Adobe debugger won’t tell you if everything worked right, so try another tool like Charles Proxy or Firebug, and find the request to “dpm.demdex.net.” The response should look something like this if it worked correctly:

charles-amcv-good

However, you may see something like this:

charles-amcv-bad

If you get the error message “Partner ID is not provisioned in AAM correctly,” stop your testing (hopefully you didn’t test in production!). You’ll need to work with Adobe to make sure your Marketing Cloud Org ID is ”provisioned” correctly. I have no idea how ClientCare does this, but I’ve seen this problem happen enough to know that not everyone at Adobe knows how to fix it, and it may take some time. But where my first 4-5 clients all had the problem the first time they tested, lately it’s been a much smoother process.

If you’ve made it this far, I’ve saved one little thing for last – because it has the potential to become a really big thing. One of the less-mentioned features that the new Marketing Cloud Visitor ID service offers you is the ability to set your own unique IDs. Here are a few examples:

  • The unique ID you give to customers in your loyalty program
  • The unique ID assigned by your lead generation system (like Eloqua, Marketo, or Salesforce)

You can read about how to implement these changes here, but they’re really simple. Right now, there’s not a ton you can do with this new functionality – Adobe doesn’t even store these IDs in its cookie yet, or do anything to link those IDs to its Marketing Cloud Visitor ID. But there’s a lot of potential for things it might do in the future. For example, very few tools I’ve worked with offer a great solution for visitor stitching – the idea that a visitor should look the same to the tool whether they’re visiting your full site, your mobile site, or using your mobile app. Tealium’s AudienceStream is a notable exception, but it has less reporting capability than Adobe or Google Analytics – and those tools still aren’t totally equipped to retroactively change a visitor’s unique ID. But creating an “ID exchange” is just one of many steps that would make visitor stitching a realistic possibility.

I’ve intentionally left out many technical details on this process. The code isn’t that hard to write, and the planning and coordination with deploying it is really where I’ve seen my clients tripped up. But the new Marketing Cloud Visitor ID service is pretty slick – and a lot of the new product integration Adobe is working on depends on it. So if you’re an Adobe customer and you’re not using it, you should investigate what it will take to migrate. And if you’ve already migrated, hopefully you’ve started taking advantage of some of the new features as well!

0
Analytics Strategy, Featured, General

A Primer on Cookies in Web Analytics, Part 2: What about Web Storage?

Posted on by Josh West

Last week I wrote about cookies, and how they are used in web analytics.  That post was really just the first in a series of posts I wanted to write about how cookies are used by web analytics tools to identify visitors to your site, and I’ll get to that shortly. But a few readers reached out to me and asked why I hadn’t mentioned web storage, so I wanted to mention it briefly before moving on.

Web storage is a feature built into modern browsers that allows a developer to store information in the browser about the site and the user, without using a cookie. This storage can be persistent (via the localStorage JavaScript object) or session-based (via the sessionStorage JavaScript object). It was included in the HTML5 spec by the W3C years ago, and any browser versions released in the last 5 years or so include it – even Internet Explorer 8, the current bane of web developers everywhere! Note that there are a few exceptions (like Compatibility Mode in IE8), but suffice to say that web storage has been available to developers for quite awhile. However, while it is an incredibly cool idea in theory, my experience has been that it is less widely used in practice – especially when it comes to web analytics. Why is this?

  1. Cookies are included with every HTTP request, but web storage is only available to the browser. So for tools that need to communicate with the server and give it access to client-side data (like many analytics tools), cookies are still the best solution – even though they clutter up the network as they get passed back and forth, and are limited to about 4KB of data.
  2. Cookies can be shared across subdomains of the same website, but web storage is subdomain-specific. That means that a cookie with scope of analyticsdemystified.com can be read by my blog or any of my partners – but a value stored in web storage is available only to my blog, josh.analyticsdemystified.com. For a company that operates many subdomains and tries to track their users across each one, this is a major limitation of web storage.

When I worked at salesforce.com, that second reason had a major impact on the reason we used a cookie to store all sorts of cool personalization data about our visitors, rather than local storage. In fact, at one point I was challenged by my manager with migrating that cookie to local storage – a really exciting project for me because I got to try out some really new technology. But after all our initial testing passed, I remember frantically having to revert all my code prior to our next code release because my own visitor history disappeared once I started browsing our staging site instead of our development site!

So keep that in mind when thinking about whether to use local storage or cookies. Web storage is much more considerate of your users, and much for forward thinking, since it’s utilizing the latest and greatest technology built into browsers. But there are some things it can’t do, so keep it in your toolbelt but remember that there are many times when cookies are still the best way to solve your analytics challenges.

0
Analytics Strategy, General

A Primer on Cookies in Web Analytics

Posted on by Josh West

Some of you may have noticed that I don’t blog as much as some of my colleagues (not to mention any names, but this one, this one, or this one). The main reason is that I’m a total nerd (just ask my wife), but in a way that is different from most analytics professionals. I don’t spend all day in the data – I spend all data writing code. And it’s often hard to translate code into entertaining blog posts, especially for the folks that tend to spend a lot of time reading what my partners have to say.

But I recently came upon a fairly broad topic that I think will lend itself to a nice series of posts, on something that I continually see confuse analysts and developers alike – cookies! Over the next few weeks, I’m hoping to dive into cookies, how they’re used in web analytics, and specifically how the 2 web analytics tools you probably use the most (Adobe Analytics and Google Analytics) use cookies to identify visitors to your website. There are subtle differences in how these tools identify website visitors, and I’ve found that most people really don’t understand those differences very well.

What is a cookie?

From the most reliable and trusted source on the Internet, a cookie is defined as:

a small piece of data sent from a website and stored in a user’s browser while the user is browsing that website

I disagree with a few key points in that definition:

  1. A cookie can be sent from a website (i.e. a web server), but it can also be set or manipulated in the browser once the page has loaded in the browser and communication has more-or-less stopped with the server.
  2. Cookies are stored in a user’s browser – but the duration of this storage can (and usually does) last beyond the time the user browses that website.

So let’s start by redefining what a cookie is – or at least my opinion of what a cookie is: A cookie is a small piece of data about a website, as well as the user browsing that website. It is stored in the user’s browser for an amount of time determined by the code used to set the cookie, and can be set and modified either on the server or in the browser. There are several important attributes to a cookie:

  • A name: This is a friendly name for the cookie, usually reflecting what data it is storing (like a username, or a preferred content language).
  • A value: This is the actual data to be stored, and is always a string – but it could contain a numeric string, or JSON data converted to a string, or any type of data that can easily be converted from a string to some other format.
  • An expiration date/time: This is a readable UTC date string (not a timestamp). If this date is omitted, the cookie expires when the browser closes (i.e. a session cookie). If the date is included, the cookie is persistent.
  • A domain: This is the website in which the cookie has scope (meaning where it can be read and written). For example, on my blog I can set a cookie that has scope of analyticsdemystified.com or josh.analyticsdemystified.com. I cannot set a cookie that has scope of adam.analyticsdemystified.com, or for any other website.
  • A path: Once the domain has been specified, I can also choose to restrict it to a certain path or directory on my website – like maybe only anything in /products. The path is rarely specified.
  • Security and other access controls: These include whether the cookie can be read on both secure and non-secure pages, and (in newer browsers) whether a cookie should be readable by JavaScript. These settings are also rarely used.

 A day in the life of a cookie

You may use a tool like Firebug to inspect your cookies, and be familiar with seeing cookies represented like this:

However, you may be less familiar with what a cookie actually “looks” like. But it’s just a string that is saved as a text file somewhere in your browser’s application data (you can find these text files if you search for where your browser’s cache is located on your computer). For the same example above, the text string looks something like this:

userid=12345; expires=Wed, 31 Dec 2014 23:59:59 UTC; path=/; domain=josh.analyticsdemystified.com

So lets take a deep dive into this particular cookie and how it ends up being available for your analytics tools to use. This cookie gives me a unique user ID on my blog. It is set to expire on the last day of 2014, and has scope anywhere on my blog domain. But what led to this cookie showing up in the list of cookies in my browser?

First, a cookie must be set. This can occur on the web server that responds to your request for a given page – or it can occur with JavaScript code setting a new cookie once the page is rendered in your browser. The code setting the cookie must specify all the details above.

Now, every time I request a new page on my blog, this cookie will be sent back to the server with that request. The server may modify it, and then the cookie will be available to the browser with the rest of the page. Any JavaScript loaded onto the page can then read it, overwrite it, and do whatever it wants to it. But if I leave and go to Eric’s blog, my cookie will not be sent anymore – it will stay stored in my browser, waiting until New Years’ Eve, just hoping that I come back.

cookies

This issue of cookie scope is a particularly tricky topic, so let’s take a closer look. Because I set my cookie with the scope of josh.analyticsdemystified.com, it will be included any time one of my blog posts is served. But if I set it with the scope of webanalyticsdemytified.com, it will be included for any of my partners’ posts as well – because we all share that top-level domain. I cannot set my cookie with any other scope – I can use my subdomain or the top-level domain – but other subdomains or other domains (like google.com) are not allowed. And my code will never be allowed to read cookies on those other domains, either. The same is true of the “path” component of a cookie as well – so you can see how path and domain combine to limit which cookies a a site can read, and which ones it cannot read. They essentially act as a bouncer, controlling which parts of this big party called the web my site has access to.

First-party vs. Third-party

Another major issue of confusion when dealing with cookies in web analytics is the idea of first-party vs. third-party cookies. A first-party cookie simply means that the domain of the cookie matches the domain of the website in the browser. In the example above, a cookie whose domain is josh.analyticsdemystified.com or analyticsdemystified.com is first-party on my blog. It will be included in all requests to the server, and returned to the browser where JavaScript can use it.

However, the average page normally includes many assets – like images or scripts – that are hosted on a different domain than my site. For example, my blog loads the jQuery library from Google’s CDN, ajax.googleapis.com, and our Google Analytics tracking requests are sent back to http://www.google-analytics.com. Any cookies my browser has for either of those domains (note: if you look, you will not see any – this is just an example) will be sent back to each of those servers, because they have scope there – but when the requests come back to my browser, the page waiting to receive the assets has a different domain – so it can neither read  from nor write to any of those cookies. They are what we call third-party.

Third-party cookies have a bad reputation, because some companies use them maliciously, and also because it can be annoying to have cookies you never asked for cluttering up your browser cache and passing data back to places you never knew about. But in most cases, cookies – even third-party cookies – contribute to making the most positive user-experience possible. However, because maintaining privacy is important to everyone, most modern browsers offer users the ability to choose whether third-party cookies should be allowed or not. Most browsers allow third-party cookies by default, and most users never modify those settings. But if third-party cookies are disabled, it means that the server will try to set a cookie, and the browser will immediately reject or delete it. And even when third-party cookies are allowed, they can never be used by the site in the browser – nor can the third-party site read any first-party cookies, either. For example, when this page loaded, a cookie was set by disqus.com, the tool we use for comments and discussions. My blog cannot read or modify that disqus.com cookie – and disqus.com can’t read any analyticsdemystified.com cookies, either. Developers reading this post are now shaking their heads, as there are obviously exceptions to this – but most readers probably aren’t interested in cross-site scripting and other hacking techniques that are the exceptions to this rule.

Some browsers like Safari even allow you to select an option that says “Only accept cookies from sites I visit,” which blocks some, but not all, third-party cookies. For example, when I worked at salesforce.com, http://www.salesforce.com was the entry point to all our websites, but we had other websites (like http://www.force.com) that we linked to on that website. Because http://www.force.com included images from http://www.salesforce.com, those cookies with http://www.salesforce.com scope would not be deleted, even though the domain didn’t match that of the page in the browser – because I had previously been to that site. The browser used my history as an implicit acceptance of those third-party cookies. However, even though the cookies would not be deleted, they couldn’t be used by http://www.force.com, either.

I often have clients ask me how they can take advantage on one site of a cookie they set on another one of their sites. Unfortunately, even though they “own” both sites, and the same developers manage both sites, there is no way of doing this without taking the risk that those cookies will be deleted by your users blocking third-party cookies. There are some “hacks” to accomplish this, but they rarely outweigh the risks you’re taking.

Conclusion

Hopefully, you now understand the basics of cookies a little better – because I’ll be back soon with a closer look about how analytics tools make use of them. The next topic I plan to address is how Adobe and GA use cookies to identify your visitors. Then, the concluding topic in this series will be about Adobe’s new Marketing Cloud visitor ID, and how it differs from the other ways Adobe has historically managed visitor identification.

0
Tag Management

Time for Tag Management Vendors to Grow Up

Posted on by Josh West

As I have written before, Analytics Demystified are big fans of Tag Management Systems, and I believe we are at the point where our entire client base is either on TMS or will be in the very near future. The advantages of TMS, especially when coupled with solid governance and the right resources managing the technology, are so many that we have simply stopped counting, and our most advanced clients are starting to do things with tag management that we’re not sure even the vendors imagined.

Speaking of tag management vendors, sometimes watching the competitive landscape evolve is like watching the entire 10 year evolution of the web analytics sector happen in fast-forward … good, bad, and ugly. Market share is constantly in flux, outside (investor) interest ebbs and flows, prices are falling, employees are moving around, and sadly we are seeing the same snippiness, FUD, and otherwise uncouth behavior that characterized the worst aspects of our industry during the early days.

Sigh.

Fortunately for all involved, many of the TMS vendors have banded together with the World Wide Web Consortium (W3C) and have authored a first-of-its-kind proposed standard for the creation of the digital data layer that forms the foundation of every “best of breed” tag management deployment. The group, chaired by IBM’s Viswanath Srikanth, had 100+ participants and over 50 companies contribute to the creation of the specification which is now available for download via the W3C.

How cool is that?

Built around a common set of definitions and widely used Javascript, the document outlines how data layers can (and should) be built to facilitate the deployment of tag management systems. What’s more, assuming the vendor participants in the W3C working group follow through on their commitment to support this standard, those companies leveraging the W3C work will essentially enable a “plug and play” environment allowing for even greater portability of systems including TMS.

At Analytics Demystified we are excited to start leveraging this document in our client work and are looking forward to years of growth in the TMS sector. We never had any kind of standard in the early days of web analytics — the technology didn’t lend itself to it and the competitive landscape would have never let it happen. Hopefully the participation of many of the major vendors and their commitment to support the W3C standard will demonstrate that the aforementioned “snippiness” is just growing pains and the TMS sector will mature as quickly as they emerged.

If you’d like help interpreting the W3C document and deploying tag management on your sites drop me a line, we’d love to hear from you.

 

0
Tag Management

TMS Best Practices: Get the Data Layer Right

Posted on by Josh West

Over the past year or so, I’ve had the opportunity to see some “do’s” and “dont’s” when implementing a tag management system. I thought today I’d share some thoughts on the most important item on the “do” list: Every good TMS implementation I’ve seen is supported by a carefully planned, well-documented data layer.

The initial sales pitch used by the various vendors – and by marketing folks to their IT departments – is often that a TMS will solve all their problems. Among other things, a TMS can do the following:

  • See any piece of data in the DOM
  • Bind to any DOM event
  • Replace all your tags with a single tag

Companies often assume that these three things mean you’ll never need to engage your development team again. While a good tag management implementation will definitely make the lives of marketing and IT folks easier, this just isn’t true. And if you try to make it true by relying exclusively on what you can see on the page and in the DOM, you’re likely to give yourself major headaches at some point – because what you can see changes way more often than anyone realizes.

Developers are notorious (I speak from experience) for looking for ways to simplify code, make it more semantical, and remove things they think are no longer needed. In almost all cases, this is a good thing. But what is no longer needed for a functioning web application may still be mission-critical to your web analytics implementation. Consider the following code snippet:

 

Lets say you log into your TMS of choice, and you set up your add-to-cart tracking in the following way:

  • Attach an on click handler to all links with class “add”.
  • Find the parent <div> and use its “data-id” attribute as the product ID.
  • Grab the nearest <span> with class “price”, remove the first character (dollar sign), and treat that value as the current price.
  • Track an “add-to-cart” event.

That’s exactly what you bought a TMS for, right? To make your life easier – and boy, was it easy to set up that tracking for your new cart add functionality. But how quickly it can all go wrong:

 

  1. What if your developer decides to change the markup for price, to separate the price from the currency. Now, according to your analytics data, your new price for this item is “.99”. Ouch!
  2. Next, your developer decides that “data-id” is a poor name for your product IDs, because all your IDs start with “SKU-“. Wouldn’t it be better to use “data-sku” instead? Now you’re tracking cart adds, but not cart adds by product ID.
  3. Finally, your development team loves HTML5, and they decide that <article> is more semantically correct than <div>. But your tracking depended on a CSS selector of “div.product”. At this point, your tracking is completely broken.

These are just a few reasons that I strongly recommend that my clients to create a data layer – that is, a catalog or dictionary for all the data points used by all your third-party tags. Your data layer should primarily be made up of a single JavaScript data object, and depending on the vendor you choose, this may be required.. But even if your tool doesn’t require it, you should do it anyway. A data object is a logical, semantical, code-based representation of all the key information we might want to know about a page when it loads:

  • The page’s name
  • Site-related information (site name, country and language)
  • Visitor-related information (login ID, logged in or out)
  • Any events taking place as part of the user loading the page (product view, internal search, etc.)
  • If an e-commerce site, any product-related information (the product being viewed, all products in the cart, etc.)
  • Any other information that might be required or useful to track the page via your analytics tool or any other third-party tags

In code, your data layer may end up looking something like this:

 

Having a good data layer in place makes your life much easier when using your TMS to implement your tags. You’re left with a simple mapping exercise to make sure that all the right data points end up going to the right tags, rather than having to use Firebug to find every possible piece of data you may want in the DOM, and then write code to access it correctly.

Let me leave you with 10 tips to build a data layer that works for you (and not the other way around):

  1. Most importantly, document it. Create a shared, central document that shows each item in your data layer, how it is set, how it can be retrieved, and which tags use it.
  2. Use a dedicated data object. In some cases, you may still rely on query parameters, cookies, or the DOM, but you should have high confidence these values won’t change. Having all your data in one place makes it less likely that a code change will ruin your data.
  3. Make your data layer as granular as possible. For example, your page name may combine country, language, and a friendly page name. Separate those into 3 unique elements in your data object, and then use your TMS to combine them together. This preserves your flexibility if you ever change your mind.
  4. Use terminology that makes sense to your organization. If you use the term “order ID”, you shouldn’t care that SiteCatalyst calls it “purchaseID” or DoubleClick calls it “ord”. Your TMS will take care of that.
  5. Don’t overload or overcomplicate. If an element isn’t used on a page, either don’t include it in the data object for that page, or set it to an empty string. Values like “null” can cause problems in your code and yield strange values in your reports.
  6. Use string representations of “true” and “false” rather than booleans. This might drive your developers crazy, but it’s much easier to check for certain strings than booleans in JavaScript (while very cool, the difference between “==” and “===” in JavaScript isn’t the easiest concept for non-developers to understand).
  7. Consider setting numeric values like prices to strings. This may require some extra code in the event that calculations are necessary, but you’ll have fewer times when you need to calculate than when you’ll want actual strings, so it’s easier just to stay consistent.
  8. Understand the trade-offs between using nested objects and arrays. For example, your developer might prefer an array of product objects to represent your shopping cart, but your TMS vendor may prefer several arrays of each product attribute (one for IDs, one for names, etc.). A single array of objects may be more semantically correct, but it’s much faster to loop through several arrays that have all their items in the same order. Your tool may make this decision for you – but know why you’ve chosen the approach you have.
  9. Don’t be afraid of quick fixes – but don’t be afraid to “fix your fixes”, either. Sometimes you’ll be in a hurry and might need to rely on DOM manipulation or quick fixes for your tracking. Make sure these are clearly documented, and work to move that data into the data object as quickly as possible.
  10. Hold everyone in your organization accountable. Make sure your data layer stays accurate over time, and that it remains the single source of truth for all your tags. If your data layer is accurate, your data will be accurate, too.

Getting your data layer right is the most important thing you’ll do when implementing your TMS. It’s what allows your development team to be website experts, without needing to be experts in every tag you might use. But it also forces marketers adhere to the centralization that was likely a key reason IT went along with the TMS in the first place. So get your data layer right – and keep everyone happy, all the way around!

0
Reporting

The "K" in "KPI" is not for "1,000"

Posted on by Josh West

At the core of any effective performance measurement process are key performance indicators, or KPIs.

Did you catch the redundancy in that statement? Performance measurement uses performance indicators. What gets my goat — because it drives report bloat and the scheduled production of an unnecessary sea of data — is how often the “K” in KPI gets ignored. More times than I can count, I’ve been sent a “list of KPIs,” that, rather than being a set of 3-5 measures with targets established, is a barfed out list of metrics and data:

Barfing Metrics

I had a self-humoring epiphany last week that, perhaps, marketers get confused by the acronym and think that “K” stands for “1,000” rather than for “key!” Through that lens, perhaps they’re falling short — lists of 20 or 30 KPIs are still well short of 1,000! My favorite response to my idle ephiphany (shared on Twitter, of course, because that’s what Twitter is for, right?) was from Eric Matisoff:

kkpi_matisoff

Not only did I realize that I’d seen the phrase “key KPIs” used myself…I saw this phrase in writing two days later!

NO, people! No. No. NO!!!

This bothers me (obviously!) — not just when it happens, but the fact that it happens so. So, why does it happen, and what can we do about it?

The History of Digital Analytics Does Not Help

As an industry, we are stuck with a pretty persistent albatross of history. When I started in web analytics, the data we had access to was generated once a month when our web analytics platform (Netgenesis) crunched through the server log files and published several hundred reports as static HTML pages. The analysts needed to know what those reports were so that they could quickly find the ones that would be most useful in answering the business questions at hand. When no such report was in the monthly list of published reports, we would either dive into a cumbersome (hours to run a simple query) ad hoc analysis tool, configure a new report to be added to the monthly list, or both.

We might look at the new report once or twice over the next few months…but the report never went away.

It got to the point where it took the first 10 days of each month for the ever-growing list of monthly reports to be published by the tool. In many cases, data from those reports was getting pulled into other reports with data from other sources. We got to that dreaded point where the report for any given month was often not published until 3 weeks into the following month. Egad!

But, in some ways, it was our only option. We didn’t have quick and efficient access to ad hoc queries of the data that we now have on many front. So, the reports were, really, mini data marts. High latency, expensive, and low value mini data marts, but mini data marts nonetheless. Somehow, though, we often still seem to be stuck with that mindset: a recurring report is the one shot we have to pull all the data we might want to look at. That’s silly. And inefficient. Our monthly (or on-demand) performance measurement reports need to be short (one screen), clear (organized around business goals), and readily intepreted (“at a glance” read of whether goals are being met or not).

KPIs Are Actually Quite Simple (if not Easy) to Identify

KPIs are the core of performance measurement. They’re not there for analysis (although they may be the jumping off point that triggers analysis). They’re not the only data that anyone can ever look at. They’re not even the only data that will go on a dashboard (but they will get much more prominent treatment than other metrics on the dashboard). I use the “two magic questions” to identify KPIs:

  1. What are we trying to achieve?
  2. How will we know if we’re doing that?

The answer to the second question is our list of KPIs, but we have to clearly and concisely articulate what we’re trying to achieve first! And that question gets skipped as often as Lindsey Lohan dons an ankle monitor.

I like to think of the answer to the first question as the conversation I would have with a company executive when we find ourselves riding on an elevator and making idle chit chat. She asks, “What are you working on these days?” I (the marketer) respond:

  • “Rolling out our presence on Twitter.”
  • “Creating a new microsite for our latest campaign.”
  • “Redesigning the home page of the site.”
  • “Expanding our paid media investment to Facebook.”)

She then asks, “What’s that going to do for us?” (This is the first of the two magic questions.) I’m not going to start spouting metrics. I’m going to answer the question succinctly in a way that expresses the value to the business:

  • “With Twitter, we’re working to put our brand and our brand’s personality in the minds of more consumers by engaging with them in a positive, timely, and meaningful way.”
  • “We will be giving consumers who find out about our new product through any channel a place to go to get more detailed information so that they can purchase with confidence.”
  • “We will make visitors to our home page more aware of the services we offer, rather than just the products we sell.”
  • “We will introduce potential customers to our brand efficiently by targeting consumers who have a profile and interests that make them likely targets for our products.”

As marketers, we actually tend to suck at having a ready and repeatable answer to that  question. If we have that, then we’re 75% of the way to identifying a short list of meaningful KPIs, because the KPIs are then viewable through the lens of whether they are actually metrics appropriate for measuring what we’re trying to achieve.

A KPI Without a Target Is Not a KPI

“Visits is one of our KPIs, and we had 225,000 visits to the site last month.”

Is that good? Bad? Who knows? In the absence of an explicitly articulated target, we simply look at how the KPI changed from the prior month and, perhaps, how it compared to the same month in the prior year. That’s fine…if the target established for the KPI was based on one of these historical baselines. All too often, though, there is no agreement and alignment around what the target is.

If we accept that KPIs have to explicitly have targets set (and those targets aren’t necessarily fixed numbers — they can be based on some expected growth percentage or compare), then the list of KPIs automatically gets shorter. Setting targets takes thought and effort, so it’s not practical to set targets for 25 different metrics. If we hone in on 3-5 KPIs, then we can gnash our teeth about the lack of historical baselines or industry benchmarks to use in setting targets…and then set targets anyway! We will roll up our sleeves, get creative, realize that there is a SWAG aspect of setting the target…and then set a target that we will use as an appropriate frame of reference going forward. It’s not an impossible exercise, nor is it one that takes an undue amount of time.

Did I Mention that “K” is for “Key?”

Perhaps it is a quixotic quest, but I’ll take any company I can get in this battle for sanity. Let’s get the “key” back in KPIs! If you’re up for saddling up and tilting at this particular windmill, feel free to snag a copy of my performance measurement planning template as one of your armaments!

donquixotic

0
Analytics Strategy, Tag Management

Tag management systems: How do I choose?

Posted on by Josh West

Since I joined Analytics Demystified a few months ago, I’ve spent quite a bit of time deep-diving into the various tag management tools. I had worked with most of these tools previously, but I’ve had the chance to meet with their leadership and development teams, and really get a feel for where they’ve come from, where they’re heading, and where the industry is at generally – so I figured I’d share a few thoughts.

In the early days of tag management, the leading tools either offered only professional services and no product (“Hey, just let us write all your analytics code for you in a central place”), or just made a developer manage tags in yet another environment that they weren’t familiar with. These tools are much better now, with the idea of a data layer, “tag templates”, and the ability to easily connect your data to your tags.

Still, there’s a lot of room for improvement, and in my view there isn’t one single tool that addresses all of the challenges that a dynamic, data-driven organization faces in their quest for rapid deployment, data integrity, and marketing agility. Here are my top three questions that a company must answer when choosing a tag management vendor:

  1. Am I trying to remove my development team from the tagging process – or am I trying to free my tagging process from my development team’s deployment cycle?
  2. What role does testing and optimization play in my organization?
  3. How do I prioritize scalability, speed, and tool usability in my selection of a tag management system?

Question #1: Am I trying to remove my development team from the tagging process – or am I trying to free my tagging process from my development team’s deployment cycle?

The historical schism between Marketing and IT becomes the central question when it comes to tag management: how can the enterprise improve their analytical efficiency? On one hand, the development team doesn’t want to “waste” time and effort on tagging anymore; on the other, the marketing team is frustrated by the lengthy release cycles their development team offers them to release new tags.

If you’re trying to solve for the first part of the question, you may be in for an uphill battle and might end up disappointed. Even the most user-friendly tools don’t totally remove the need for a developer. After implementation, a marketer may be able to add the simplest conversion or remarketing tags, but web analytics tags like Adobe Analytics (formerly SiteCatalyst) are complicated, especially with dynamic interactions to be tracked after a page has loaded. You should plan on needing some development help even after implementing a TMS, though it will be less than before.

It’s a much more realistic goal to address the second question and separate your tagging from traditionally lengthy release cycles. Your development team may still need to help you write some code to target a particular page event, or write some custom JavaScript to manipulate a particular value in your data layer for a new tag – but (ideally) they can do it without being tied to a release several months in the future.

Regardless, these questions aren’t as much a TMS selection question as an internal process and governance issue. When you have answered the question your TMS choices will narrow, but if you don’t address this question “head on” no TMS will help you.

Question #2: What role does testing and optimization play in my organization?

Optimization is becoming increasingly critical for data-driven organizations, and unfortunately, it’s the most difficult use case for a tag manager to address. This is because according to best practices, testing tags must load synchronously on a page to function properly. For example, the Adobe Target (formerly Test&Target) “mBox” blocks everything below it from loading, by design, until Adobe returns the new content.

On this specific point I see the current TMS vendors as being differentiated. Ensighten is the current leader in providing easy integration with optimization tools; it is also the only tag management system whose best-practice implementation starts with a synchronous tag. Conversely, Tealium and BrightTag use an asynchronous tag, which has several advantages, the most common being no page blocking. This is ideal for almost any tag except optimization tags. Both these tools offer a synchronous deployment approach and other alternatives to prevent “flicker” when the page loads, but they are not as simple to implement.

On the question of testing, at least for now you will need to decide if you value an asynchronous tag or the ability to stick with a standard tagging strategy for your optimization program.

Question #3: How do I prioritize scalability, speed, and tool usability in my selection of a TMS?

As I mentioned earlier, each of the tag management leaders have strengths and weaknesses. Three important factors to consider when choosing a TMS are scalability, speed, and usability – and there isn’t a single tool currently available that has cornered the market on all three.

  • Scalability: You want to choose a tool that can grow with your organization. Ensighten and BrightTag both utilize a tag that combines client-side and server-side technology. IT departments and security teams often worry about overloading the DOM – and while there is a limit to what you can do with a purely client-side solution, I’ve found it to be more of a “theoretical” limit. If your TMS-deployed tags are still slowing down your site, it’s probably not the tool’s fault! You can succeed with either approach if you plan well and execute on that plan – but it’s up to you to decide what works best for your tagging roadmap.
  • Speed: The flip-side to that additional server-side request is speed. Every bit of code delivered by Tealium can be served from a static CDN (like Akamai), which means it can potentially load much faster than a tool requiring server-side technology – and they, BrightTag, and Ensighten support the bundling of your most commonly-used tags, which speeds things up even more. With careful planning around your tool’s scalability and speed constraints, any of these tools can likely yield a successful implementation – but you need to be aware of the strengths and weaknesses of your tool.
  • Usability: The first releases of the leading TMS tools were clunky from a usability point-of-view. Instead of being able to manage your tagging code within your CMS, in an environment you were comfortable with, you had to cut and paste your tags into a less familiar tool. Now, all the major tools offer “tag templates”, which simplify and standardize the implementation of the leading tags. Right now, Tealium’s template library and collection of helpful extensions is the richest and most user-friendly, and now everyone is heading in that direction. A good tag manager will have the capability to access critical data on the page – BrightTag’s data binding features, for example, are great at this.

Two last points before I wrap up:

  1. I’ve intentionally tried to focus primarily on strengths, not weaknesses, of the products I’ve mentioned – to identify areas where a particular product does something really well. But don’t read too much into not seeing a product mentioned with a particular topic. Plus, I value our partnerships with the various tag management companies – and I obviously have lots of friends and former colleagues scattered throughout the industry.
  2. It’s quite possible your company uses or has researched a tag management tool I didn’t specifically address. For the most part, I tried to stick to the tools our clients use most frequently, because they’re what I’m most familiar with. But there are a few other tools I really need to get my hands on, and as soon as I do, I’ll share my thoughts on them as well.

In summary, when you choose your TMS, it will be critical to identify your top requirements and make sure you pick a solution that covers them best. You’ll likely encounter some trade-offs along the way, because there’s not a perfect tool out there. But any of the solutions can make your life easier and improve the speed with which you can deploy your tags.

At Analytics Demystified, we’re excited about the trends in the industry – and we’d love to talk more with you as you consider which tool works best for you!

0
General

Josh West: The Latest Partner at Demystified

Posted on by Josh West

I am super-excited to join the incredibly talented team of Demystified partners! My background is in development, and in full disclosure I considered beginning my initial post with “Hello analytics world!”, until better judgment prevailed. Needless to day, I’m thrilled to be joining the brightest minds in digital analytics, and looking forward to adding my skillset to the wide range of services we offer our clients.

I was most recently a member of the front-end development team at Salesforce.com, where I managed all technical aspects of their web analytics, optimization, and advertising efforts. Initially with our own Adam Greco, and later with another group of rockstars, we rebuilt our implementations from the ground up in a way that that allowed the entire online marketing organization to have an “apples-to-apples” view of what each individual team was doing. Prior to that, I worked in the Omniture Consulting group with some of the brightest individuals and most successful companies in the industry today.

My focus at Demystified will be in tag management, implementation, and in custom development. I enjoy turning broken implementations into world-class ones, leveraging vendor APIs in unique ways, and enabling clients to maximize the significant investment they make in their online presence. And I couldn’t be more thrilled to work side-by-side with Adam, Kevin, and Brian again – and with Eric, John, and Michelle for the first time.

I’m the second Demystified partner in Utah, where I live with my wife and 3 young kids – a boy and 2 girls. Our youngest was born the week I started at Demystified – which made for a pretty crazy week! I enjoy playing basketball and following almost every sport (none of which I’m very good at), camping, and spending time with family. We enjoy making homemade pizza, which I learned when I spent a few years living in Italy. I enjoy all things Italian – in fact, I chose my Twitter handle (@joshovest) because “Ovest” is “West” in Italian, and all the common variations of my name were taken when I signed up.

Please feel free to reach out to me via email (josh@analyticsdemystified.com) or follow me on Twitter (@joshovest), especially if you have any technical questions about your analytics. I look forward to engaging with all of you in the analytics community very soon.

0