The "Right" Talent: an MVT-Meets-Fractional-Factorial-Design Anecdote

When it comes to business/management books, one of my favorites is First, Break All the Rules. When I first read it, it didn’t strike me as particularly profound. I was relatively new to managing people, and I was being “forced” to read it for an internal class, so my natural reaction was to view it cynically. I’m not proud of it, but that’s how I roll.

Over time, I’ve found myself quoting and recommending the book again and again (I can’t say the same for the follow-up book — Now, Discover Your Strengths — but that’s a topic for another post). The fundamental premise of First, Break All the Rules, goes something like:

1. There is a difference between skills and talents — the former can be taught, whereas the latter are more innate characteristics, the combination of which make each person unique
2. Conventional wisdom has managers focussing on hiring for skills and then focussing on employees’ weaknesses and trying to “fix” them
3. The “weaknesses” are, all too often, talents the employee simply does not have
4. It’s much better to identify each employee’s talents/strengths and then help them figure out how to capitalize on those strengths rather than simply managing to their weaknesses

The book also explains the fallacy of spending a disproportionate amount of time with your weakest employees, which is where day-to-day management tends to pull you. 

And, I’m possibly butchering the book in my summary — it’s been a few years since I reread it!

When it comes to data-driven job roles — think most roles where the word “analyst” appears in the job title — the real challenge is finding people who have the right mix of talents. A big part of what we worked on in the Business Intelligence department at National Instruments was building a capability to operate effectively in two different dimensions:

• As business experts — we recognized that we needed to know our business and the business of sales and marketing as well or better than our internal customers. We had to genuinely want to understand the business problems they were facing — know them well enough that we could articulate them effectively on our own.
• As data geeks — at the end of the day, we were expected to be able to pull data, analyze it, explain the results, and present them effectively.

What made our team effective, in my view, is that everyone in the department was very strong in one of these areas, and at least competent in the other. And, we paired up people with complementary skills when it came to tackling any project. On the one hand, this sounds like it was inefficient, but it really wasn’t — it didn’t mean that these teams were joined at the hip and never operated alone. Rather, it meant that they collaborated — both directly with our internal customers as well as offline with each other — to come at each project from multiple angles.

Now, here I am several years later, taking an Intermediate Statistics class through The Ohio State University. The class is taught on-site at my company, and it’s taught by a professor who spent a big chunk of his career in applied statistics working for Battelle. He’s a good professor, and I particularly like that he beats a pretty hard drum when it comes to the parallel talents needed to effectively use data in a business setting: subject matter expertise, effective problem formulation, and statistical/analytical knowledge. He rails against trained statisticians and even “applied mathematicians” who don’t want to really address the first two requirements head-on — those who jump to crunching the data prematurely, relying on their technical tools and skills to route them to “the answer.”

Bravo, I say!

At the same time, even as the professor is eloquently speaking to this issue (and politely patting himself on the back for not falling into the trap), it’s readily apparent that he’s coming at the analysis of data from a heavy background of hardcore statistics. And, while he spent much of his career working on industrial (and defense) processes and problems, he is now mired full-time in the world of marketing and consumer behavior. He is not the first data guru to cross over, by any means, but he is clearly new to the space, and, in many ways, seems hell-bent on retreading ground that has been covered already. 

As one example, there is the case of MVT, or multivariate testing. MVT has been getting a lot of buzz in marketing over the past five years or so. It’s been touted as a way to accurately test many different variables without having to run a gazillion experiments to test every combination of them. One place that MVT gets used these days is with web landing page design — enabling a marketer to test various color schemes, banner ad taglines, and headline placements to derive the optimal combination without breaking the bank with the number of tests that have to be run to make a valid, data-driven decision. That’s all well and good, and it’s clearly gotten enough traction that it works.

An old-school process improvement expert I worked with back when I was first starting to hear about MVT pulled me aside one day and said, “You know, Tim, none of this stuff is really knew — MVT’s been around since World War II. It just wasn’t applied to Marketing until recently, so there are a lot of people capitalizing on it.” And…he was right!

So, back to the present day. In a recent class, this OSU professor was walking through various ways to design experiments and how they could be analyzed, and he kept referencing “fractional factorial design.” We’re only going to touch on the technique in the class, he assured us, but he explained how there were trade-offs you have to make when using that approach. From his explanation, it sounded like fractional factorial design was a lot like MVT, and I asked him about it after class. He had never heard of the “MVT” acronym, but said it sure sounded a lot like fractional factorial design (which was a term that was entirely knew to me). It only took a few seconds on Google to find out that our suspicions were correct.

I was surprised that MVT was a new term for this professor, as it’s hard to do much of any poking around in marketing circles these days without stumbling into it.

At the same time, “fractional factorial design” was a new term for me. But, then again, I’m coming at things from a background much more grounded in marketing and business, rather than deep mathematics. I understand the point of MVT, but I’m still wildly fuzzy on the actual mechanics of it.

And…that’s the short point of this lengthy post: it seems like, in order to truly use data effectively, requires a mix of talents that rarely occur naturally in a single person. This professor has the deep statistical chops and an awareness that he is not a subject matter expert when it comes to marketing and consumer behavior, so he needs to increase that knowledge. Partner him with someone with a deep marketing background (who is almost certainly not also a statistician), and, as long as that person has an awareness of analytics and an interest in applying analysis effectively, you’ve got a winning formula.

It’s a Venn Diagram, really. If there is no intersection of the talents, then chances are it’s a combination of talents doomed to fail.

UPDATE: Check the first comment below for a clarification — MVT is not the same thing as fractional factorial design. Rather, fractional factorial is one approach for conducting MVT experiment analysis. The link on the subject earlier in this post makes this same point. I was unclear/ambiguous. I still think that, in Marketing circles, when MVT has gotten recent play, that it is fractional factorial design and analysis that gets the buzz. But, I don’t know for sure.