"You can make the data say whatever you want it to."
Rack that up as one of those popular, throwaway cliches, stated with a ho-hum air as if to say, “It’s so factual and irrefutable that I can’t believe I’m wasting my body’s energy pumping carbon dioxide converted from oxygen into the atmosphere to say it.”
Drives me nuts.
My personal fantasy? Anyone who makes this statement is banned from accessing or using any data for a year.
Why? Because, as stated this way, it fairly directly implies that any sort of data analysis is just a way to drive someone’s agenda or spin the results of an initiative. And, data can absolutely be used to do this. But it doesn’t have to be.
While I may sound like a broken record, I hope that I more sound like a well-produced album, with a selection of tunes that, while all on a similar theme, approach that theme from various angles.
So, two ways to come at the, “whatever you want it to” comment.
Situation 1
Someone makes this statement when discussing data they have looked at or the results of an analysis that was undertaken, directly or indirectly, at their behest. I had this happen today, and the fellow’s initial beef was that the analysis that we had done did not aggressively and vividly back up his own strongly held beliefs about a certain business situation. What he was looking for was an analysis that simply supported his assertion as to the current state of affairs. What temperature does blood boil at? When he dropped the, “You can make data…” comment, it was an intellectual slap in the face of sorts (not that he saw it that way — this is not a fellow who is particularly self-aware as to the impact of his words, and he was by no means trying to insult…um…my chosen career). I pushed back (rather calmly; please hold while I pat myself on the back) that, if he already knew what he wanted the data to say, and if he was just going to push for multiple iterations on the analysis until it said that, then what’s the point? Analysis should be all about answering questions. In this sort of situation, the only question is, “Can I dip my advocacy brush into a bucket of data and paint the picture that already exists in my mind’s eye?”
Situation 2
This is the case when someone presents data, and someone that is seeing it presented doesn’t buy into what it supposedly supports. Ironically, the person who points out the spinnability of data is the same person who would spin it for his/her own purposes (see Situation 1). This is, quite simply, sad. It’s a waste of a company’s money to pay for someone to take this on. More often than not, though, what happens in this situation is that the user/presenter of the data simply didn’t know how to effectively use the data. It’s sooooo tempting to start with the data. If you do that, you have an infinite number of ways to pivot it, plot it, and print it. And, after making several dozen charts, and realizing you’ve got a real snoozer in the works if you present all of them, you narrow down to 2 or 3 that seem relevant. And, human nature says, the more they seem to show something positive, the more of a relevancy boost they’ll get. The way to avoid this is to have the pre-data discipline to articulate what you’re trying to get at. Publicize those (or, at least, write them down for yourself — it will keep you on the right track!). Doggonit! Seems like half of the songs in this blog-album wind up referring back to one of my first posts. It’s a “how to avoid Situation 2” prescription.
Yes, you can make the data say whatever you want. But, that’s an awfully jaded view of the world. And, with some proper up-front discipline, you won’t be wasting your time trying to make it say what you want, and you won’t be providing much of an opportunity for the people who are receiving the analysis or report to have that cliche pop into their heads.
