My interview with Megan Burns of Forrester Research

I recently had the pleasure of chatting with Megan Burns at Forrester Research. Megan covers, among other things, web analytics. Despite her being with Forrester for just about a year now, she is one of the leading analysts thinking about how companies actually deploy and use web analytics technology.

Megan hit the ground running, filling some pretty big shoes, and has published nearly a half-dozen reports directly relevant to the web analytics market. I don’t normally interview people in my weblog but when Forrester analysts talk, people listen. The transcript of our conversation follows:

Eric T. Peterson: First question: You’ve been in the job at Forrester Research for just over a year now. What would you say the biggest thing you’ve learned about the web analytics industry is so far?

Megan Burns: There are probably two things that stand out. First, the fact that measuring a Web sites is tougher than many people think.

There’s so much data to chose from, and all of it’s imperfect. Plus there are multiple ways to solve most measurement problems. Deciding which approach to take isn’t always straightforward and for many people measurement is just one part of their job, so even though they’d like to dedicate time to thinking about the best way to leverage all the data they could be collecting, they have to make some tough priority calls.

But the second thing I’ve learned is that many people believe very passionately in the power of data, and they’re committed to figuring out the tough problems.

Eric T. Peterson: So on one side of the coin you have the complexity of measurement as a function of expertise, approach, and time, and on the other side of the coin a strong desire to make it work.

Megan Burns: Absolutely.

Eric T. Peterson: So the folks you talk to who are being successful with web analytics, is there something that sets them apart? Something quantifiable?

Megan Burns: I’m not sure if it’s quantifiable, but there’s an understanding that metrics are a means to an end not an end unto themselves. They constantly think about what they’re trying to do, and how data can act as a tool to help them do it.

Eric T. Peterson: In your experience, does success with analytics improve with company size or does it appear to be tied to motivation?

Megan Burns: I haven’t seen a correlation to company size, but I haven’t looked at that relationship specifically.

I don’t think company size is a factor, though. I think it has more to do with attitude and approach to the problem. Often it’s about teaching people in the company about data and what it can do for them so that they change the way they make decisions. Any time you’re dealing with people and change it takes a good dose of both patience and time.

Eric T. Peterson: You and I have talked in the past about the importance of “process” to web analytics. You commented once that “people think process is a four letter word” which made me laugh and wince at the same time.

First, can you describe your position on the need for process in web analytics? And second, do you have any advice to help companies get past their fear of the “P” word?

Megan Burns: Sure. I think process is important in any discipline as a way to help people make sure the right things get done by the right people at the right time. We’re all trying to do so much these days, it helps to have a process that reminds us what needs to get done. It also sets clear lines within the organization as to what each person or group is responsible for and who they are dependent on. Web analytics is no different.

People who design sites need to understand that others in the business have to be able to measure the impact and success of those sites. They need to factor measurement requirements in to the process. But they have many other people asking them to build in other requirements, so it helps to have a checklist to make sure you’ve thought about all the different types of requirements you need to capture before you build something. That checklist is part of the process.

But it’s important to remember that the “Web analytics process” is really a sub-process of the larger eBusiness process. That data is needed by certain people in the firm at certain points in their decision making cycle. If the two aren’t integrated properly, things break down. People don’t get the data they need when they need it.

Changing people’s perceptions of process can be tough, depending on their experience with it. But I think the most important thing to remember is that process != bureaucracy.

When the only thing people are trying to do is check off boxes on the process so they’re “in compliance”, you’ve totally missed the point. It needs to be detailed enough that it’s useful and insures key steps don’t get missed, but it shouldn’t impose unnecessary restrictions or red tape. That’s a very fine line — one that’s not easy to get right.

Eric T. Peterson: So tell me the truth and don’t hold back … in my presentation at Emetrics where I advised our community to go so far as to draw business process diagrams for how web analytics integrates into the bigger picture … good idea or a superfulous waste of time?

Megan Burns: Somewhere in between. I think people responsible for analytics should start by looking at the larger site design/interactive marketing process (which probably isn’t written down anywhere, by the way ) and see where and when the data needs arise. Then look at where they need to be involved (i.e. requirments, development) in order to get what they need to meet those requirements. To me, measurement is a section in the business and technical requirements documents that must always be filled out. Even if it just says “No Impact” or “No new requirements”. But at least that way you know someone thought about what new data might be needed, or what code might have to change to maintain existing measurement.

Metrics need to be considered in the project planning process, like any other feature of the site. How long will it take to define requirements for this? To implement and test them? The process is no different, but it’s not something customers use so often it gets missed.

Or skipped intentionally to save time. But then it takes twice as long to add in after the fact, so you didn’t really save any time.

Eric T. Peterson: Excellent points, all.

You have a background in software development process, don’t you?

Megan Burns: Yes, I do.

Eric T. Peterson: Okay, new direction here: You’ve written a ton on web analytics since joining Forrester Research. I especially enjoyed your work on the ROI of dedicated headcount for analysis. What was the overall response to that report?

Megan Burns: The response was extremely positive. So many people I talk to tell me that resources are their biggest obstacle to using and interpreting the Web analytics data they’re collecting. The report helped them explain to senior management what analysts do and how they add value to the organization.

In a quantitative way, that is.

Eric T. Peterson: When companies ask you where to find experienced web analytics talent, what kind of advice do you give?

Megan Burns: That’s a tough question, because experienced Web analytics talent is so hard to find these days. My advice is usually to engage professional services consultants from either their vendor or an independent consulting firm to act as mentors for existing staff.

If they really want to hire, I suggest networking, networking, networking.

This is such an active community with the Yahoo group, blogs, Web Analytics Wednesdays … there are plenty of ways to meet others who might be able to lead them to a qualified candidate.

Eric T. Peterson: We’re just about out of time and I want to thank you for being so generous in allowing me to interview you.

Megan Burns: Quite welcome. Glad we could finally arrange it.

Eric T. Peterson: Last few questions … what book or books are currently on your nightstand?

Megan Burns: There are so many books … Information Dashboard Design, by Stephen Few is one.

Eric T. Peterson: What music would we find on your iPod?

Megan Burns: My iPod’s full of all sorts of music. Everything from rock to oldies to show tunes.

Eric T. Peterson: Who are some of your favorite bloggers?

Megan Burns: I wish I had time to read as many blogs as I’d like to.

I try to keep up regularly with Charlene Li and the other Forrester blogs, you, Avinash, and Om Malik. A few others, too. It also depends on what I’m working on.

Eric T. Peterson: Megan thanks very much for taking the time to chat with me today. I hope to see you in San Francisco in May and look forward to your upcoming research on Rich Internet Applications.

Megan Burns: You’re quite welcome. Take care, and I’ll see you in CA (if not sooner).