One of the great ironies of our industry is all the time wasted talking about “Big Data” or “Governance” or a thousand other small wasted catch phrases. All of these are simple attempts to show a “maturation” or growth of the online data world. While there is enough backlash to point out that none of this is really new, it is simply the online matching the offline Business Intelligence path. A world that never has produced 1/50th of the value it also pretends to. The irony lies not in what is being debated or not debated, or if online and offline are the same, but in the fact that we all talk around the real issue and do nothing to address it.

The reality is that most BI work is a wasted effort, designed to facilitate a predetermined wish of some executive, and that the people rewarded are never the ones that actually provide the best analysis, but the ones that produce the analysis that best helps make others agenda’s move forward. The online world is simply following suit, to the point that we no longer look at how many reports you can create as justification for existence, but now how can you create a fancy graphic or data to support one groups agenda or another. This evolution follows a normal path from creation and storage, access, and now delivery of data, without one time dealing with the real issues at hand. Human beings are both awful at understanding or leveraging data, but also most people (especially those in marketing) are awful at their jobs.

If you think about it, it’s not that shocking that marketers are horrendous at their jobs; they make a living telling stories and trying to convince others of things that have no basis in reality. This means to exist in this world, you are left with two options: Act like a sociopath, or unconsciously acquiesce to some sick permanent version of the prisoner’s dilemma, but in this version as long as no one points out how full of it the person speaking is, they will return the favor. Both diseases leave the same outcome, a group of people who exist to propagate the work of the same group of people, and who seek outside justification, be it awards (from the same group), data that is only searched for one way to support them, or case studies of people who did the same thing and likewise lied their way to “success”.

The scare and threat of data is that when used in a rational manner, it can shed light on the real value of the day to day actions we hang our hat on. We can find out just how horrendously off our preconceptions are. One of the great ways to succeed at testing is simply to bet against people, as people are so rarely right that just picking the other direction creates a winning streak that would allow you to live as a billionaire if you could translate it to Vegas. You will quickly find that most experts are nothing more then storytellers, and that most of the largest gains companies made are often the least publicized, but those that are shared are often subconscious attempts to get others to fall pray to the same mistake that they too wasted months on. With almost no effort you can prove that most actions taken provide no value whatsoever to the company, or are so inefficient that they are far worse. The data and evidence is easy to get, but we avoid it in order to cope within this world.

Why can’t people act rationally more often? Why is data accepted and abused, why do we seek confirmation not information? Why do we not worship those that get results instead of those that tell stories? The answer is simple, we fear most that which may hurt our own world view. If everyone was willing to search for the right answer, we would all be better off, but as soon as one weak person accepts the word of one sociopath, we are all set on down this path, or suffer silently the fight against the tide.

This is not a new problem, Kant, Engels and many others have been talking about this problem for hundreds of years; we just find new names for the same human weakness. We seek out people to do attribution, and then believe it tells you anything about generation. We seek out those that confirm our hypothesis, not those that disprove them, despite the fact that the disproven hypothesis inherently has a better outcome. We want people to speak at conferences and point out why everyone else is screwed up but why we can change and do the exact same things that they were railing on, but now with a new name. We want to find a site that tells us “which test won”, not one that helps us be better tests or to achieve any value whatsoever. We are constantly searching for the next affirmation to justify who we are, not improve who we are.

“Reality” is not a kind mistress for those that are even slightly interested in it. Empirical Realism is looked at and talked about, but practiced by so few that it is almost as meaningless a buzzword as “personalization”. While it helps companies, it rarely helps those that which to exist in a corporate environment (see prisoner’s dilemma). We are forced to make a Sophie’s choice, do what makes others happy or that which helps our company. We all try and find ways to convince ourselves and others that we are not faced with this choice, yet we only succeed when we stop caring about or thinking about anything other then our own gain. In order to facilitate this, we find every way possible to make the mental pain go away and to find others that will tell us it will all be ok.

So if this is the sandbox in which we play, is it any wonder that our “heroes” are those that best project the best ways to make others believe what is being done matters? We worship at the altar of Avinash, or Peterson, or Eisenberg’s or anyone else we can find that as justification for what we were already doing. We have no way of knowing if what they say is correct, and personal experience shows that following most of that advice leads to immensely fallible results. Far be it from an inquisitive mind to question if the current action is the right one, or if there is a better way to think about and tackle problems. We instead allow others to dictate to us, so that we can avoid cognitive dissonance and rest easy at night… ok, on second thought, most marketers are both suffering from living in a prisoner’s dilemma and are also sociopaths. The data shows that these are not mutually exclusive but complimentary. Glad we got that squared away…

If you want to really make a difference, if you are tired of this same old world or of those charlatans who propagate it, are you prepared to fight the tide? Are you able to evaluate your own work, to go past the comfort and to find out how wrong you are, in just about everything you do? Are you then able to get past that mental scarring to do the same for others? Will you back down the first time someone pushes back, or will you make it your quest to do the right thing when it is neither profitable or easy to do so?

The history of business shows that rarely if ever does this problem truly go away, or does the better answer win. While history is written by the winners to justify their existence, randomness and the trampling of others are bred into every page of this twisted form of storytelling. And yet, until we deal with this real problem, until we are more interested in doing the right thing then the easy one, what will really change?

We will continue to waste time and effort on data, in order to justify wasted time and effort in most other efforts. We will continue to seek new words for old problems, and we will continue to make heroes those that most hold us back. Until we stop propagating the lie and until we look at ourselves first, how can we ever really deal with the real problem of data. Not the collection, not the sharing, not the presentation, but the people who are wired to use that data in the least efficient and most self-serving way possible. You want to solve big data, you want to change the industry, stop wasting time on tag management or Hadoop, solve the people, since there is where all problems lie. Don’t solve how do you share your point, but how do they think about and are they rationally using data to find an answer, or only to justify one?