Category: Psychology

Why we do what we do: You are Not so Special — The Forer Effect

While the quest for personalization may be newer to some members of the online marketing world, the reality is that is a concept that is old as sales. People have been trying to convince others that they alone were getting a special deal or that their message was meant just for them. One of the great practitioners of this concept was P.T. Barnum, who famously billed his circus as “we’ve got something for everyone.” On some level everyone understands the appeal of being special and of having someone take the time to tell me something that is unique to just me. The greatest salespeople though understood one of the great ironies of personalization, which is that general statements, when given in context, often are treated as deeply personal and are extremely powerful. This concept is known as the Forer effect, or more directly, the tendency of people to interpret statements as being accurate for them personally, even when they are not.

The Forer effect gets its name from B.R. Forer and came about from a series of experiments that he performed in 1948. His famous study involved giving a personality test to all of his students. He told them that they were all receiving unique personality analysis, and they were to rate that analysis on a scale of 0 to 5. All of the students actually received the exact same results, using such lines as, “While you have some personality weaknesses you are generally able to compensate for them.” and “You also pride yourself as an independent thinker; and do not accept others’ statements without satisfactory proof.“. Despite the exact same statements being made, his students average score for his “analysis” was 4.26.

The favorite trick of psychics, conference speakers, and astrologists, this psychological bias is important to understand, especially when thinking about the concept of personalization. General statements hidden to look like targeted messages have much greater impact than direct statements, and are far more likely to increase belief in the speaker. Personalization, as it turns out, is about not being personal, or at least not to the Nth degree. Personalization is about the match of the general and the pseudo specific, and it is about taking that message to the largest group possible, not just the ones that directly match the message. The more we can measure different types of messages, and the more we can find the largest groups that respond to them, the better our results, since functionally the more targeted the message, the less overall gain we get to improving total site performance.

So you might ask why is this so important in the quest for personalization? This bias tells us that overly thinking personalization and designing a large number of specific messages is both a waste of resources, but also far less likely to create a positive outcome. It also tells us that the message itself does not have to match the rules that dictate the outcome; general statements have an impact for a large variety of people, not just a specific targeted group.

As you start thinking about and tackling your personalization programs, it is important to understand the nature of why you are doing these actions. Barnum knew that he was there to sell his circus, and every action he did had only that outcome in mind. He was one of the most famous practitioners of a single success definition, and he knew that no matter what he did needed to drive more to spend more on his circus. The same is true of all online efforts. Your goal in the end is to make more money, and the key is not to focus on a specific message, or to over rely on experts or correlative information to tell you when and how to target. The key is to test out all sorts of possible content, and to see how you can best present them to people to allow you the efficiency of largest group of people possible.

This is why a message about a specific product may work best for Firefox users, why time of day may be the best match for your re-targeting content, and why smaller segments are so inefficient. It is also one of the main reasons why targeting content without the discovery process of the value is far more likely to lose you revenue than generate more. It turns out the more you try to narrow a message or assume an outcome, the worse your results will be. Somewhat specific messages work for far larger groups than you could ever imagine, and you only know the true power when you let go of your own ego and preconceived notions and explore.

Stop thinking of personalization as trying to build a one on one message with a customer, that does not work and is extremely inefficient. Instead explore the various ways that you can create different content, and then explore who the largest groups are that you can present that to. This means always going through a discovery process of figuring out what matters, and then figuring out for whom. You may want to target to only people who looked at brand X, or page Y, or who have done a come to your site 3 times without purchasing, but that in no way means that you should limit the message to just that group. The less control you exert on the specifics of a message, and the more you are open to new possibilities, the more likely you are to find larger and more meaningful outcomes.

Explore what the value is of different messages and of taking it to different groups. You have powerful tools at your disposal to do just that, to discover and take these more general statements to large groups. From simple A/B tests all the way to automated machine learning, the real key to value comes from how you think about the problems, not in your ability to just find a group and target to it. Not only that, but you have the ability to measure the efficiency of various discoveries and techniques against each other. You are not limited to creating these stories, or just targeting to a specific persona, you have so much more at your disposal if you just allow yourself and others the flexibility to learn and grow.

P.T. Barnum is also famous for how he could get people to pay for anything, with the most famous example being the egress. It wasn’t meant for anyone specific, but he could get just about anyone to fall prey to the mystery. He didn’t have to target that message to just one group, or to offer it for only people who were on their way out, he figured out how to take that message to everyone. He understood that just because a group might be inclined for something, that just limiting your message to that group was a waste of his time. He was the ultimate salesman, but he knew that the key was to make it look like you were walking a fine line and being extremely specific, while at the same time in no way going that far.

So the question comes down, as you explore personalization, or you selling the egress? Or are you the one on your way out that door?


The Great Dark Truth of Analytics: Sociopath or Prisoner’s Delimma?

One of the great ironies of our industry is all the time wasted talking about “Big Data” or “Governance” or a thousand other small wasted catch phrases. All of these are simple attempts to show a “maturation” or growth of the online data world. While there is enough backlash to point out that none of this is really new, it is simply the online matching the offline Business Intelligence path. A world that never has produced 1/50th of the value it also pretends to. The irony lies not in what is being debated or not debated, or if online and offline are the same, but in the fact that we all talk around the real issue and do nothing to address it.

The reality is that most BI work is a wasted effort, designed to facilitate a predetermined wish of some executive, and that the people rewarded are never the ones that actually provide the best analysis, but the ones that produce the analysis that best helps make others agenda’s move forward. The online world is simply following suit, to the point that we no longer look at how many reports you can create as justification for existence, but now how can you create a fancy graphic or data to support one groups agenda or another. This evolution follows a normal path from creation and storage, access, and now delivery of data, without one time dealing with the real issues at hand. Human beings are both awful at understanding or leveraging data, but also most people (especially those in marketing) are awful at their jobs.

If you think about it, it’s not that shocking that marketers are horrendous at their jobs; they make a living telling stories and trying to convince others of things that have no basis in reality. This means to exist in this world, you are left with two options: Act like a sociopath, or unconsciously acquiesce to some sick permanent version of the prisoner’s dilemma, but in this version as long as no one points out how full of it the person speaking is, they will return the favor. Both diseases leave the same outcome, a group of people who exist to propagate the work of the same group of people, and who seek outside justification, be it awards (from the same group), data that is only searched for one way to support them, or case studies of people who did the same thing and likewise lied their way to “success”.

The scare and threat of data is that when used in a rational manner, it can shed light on the real value of the day to day actions we hang our hat on. We can find out just how horrendously off our preconceptions are. One of the great ways to succeed at testing is simply to bet against people, as people are so rarely right that just picking the other direction creates a winning streak that would allow you to live as a billionaire if you could translate it to Vegas. You will quickly find that most experts are nothing more then storytellers, and that most of the largest gains companies made are often the least publicized, but those that are shared are often subconscious attempts to get others to fall pray to the same mistake that they too wasted months on. With almost no effort you can prove that most actions taken provide no value whatsoever to the company, or are so inefficient that they are far worse. The data and evidence is easy to get, but we avoid it in order to cope within this world.

Why can’t people act rationally more often? Why is data accepted and abused, why do we seek confirmation not information? Why do we not worship those that get results instead of those that tell stories? The answer is simple, we fear most that which may hurt our own world view. If everyone was willing to search for the right answer, we would all be better off, but as soon as one weak person accepts the word of one sociopath, we are all set on down this path, or suffer silently the fight against the tide.

This is not a new problem, Kant, Engels and many others have been talking about this problem for hundreds of years; we just find new names for the same human weakness. We seek out people to do attribution, and then believe it tells you anything about generation. We seek out those that confirm our hypothesis, not those that disprove them, despite the fact that the disproven hypothesis inherently has a better outcome. We want people to speak at conferences and point out why everyone else is screwed up but why we can change and do the exact same things that they were railing on, but now with a new name. We want to find a site that tells us “which test won”, not one that helps us be better tests or to achieve any value whatsoever. We are constantly searching for the next affirmation to justify who we are, not improve who we are.

“Reality” is not a kind mistress for those that are even slightly interested in it. Empirical Realism is looked at and talked about, but practiced by so few that it is almost as meaningless a buzzword as “personalization”. While it helps companies, it rarely helps those that which to exist in a corporate environment (see prisoner’s dilemma). We are forced to make a Sophie’s choice, do what makes others happy or that which helps our company. We all try and find ways to convince ourselves and others that we are not faced with this choice, yet we only succeed when we stop caring about or thinking about anything other then our own gain. In order to facilitate this, we find every way possible to make the mental pain go away and to find others that will tell us it will all be ok.

So if this is the sandbox in which we play, is it any wonder that our “heroes” are those that best project the best ways to make others believe what is being done matters? We worship at the altar of Avinash, or Peterson, or Eisenberg’s or anyone else we can find that as justification for what we were already doing. We have no way of knowing if what they say is correct, and personal experience shows that following most of that advice leads to immensely fallible results. Far be it from an inquisitive mind to question if the current action is the right one, or if there is a better way to think about and tackle problems. We instead allow others to dictate to us, so that we can avoid cognitive dissonance and rest easy at night… ok, on second thought, most marketers are both suffering from living in a prisoner’s dilemma and are also sociopaths. The data shows that these are not mutually exclusive but complimentary. Glad we got that squared away…

If you want to really make a difference, if you are tired of this same old world or of those charlatans who propagate it, are you prepared to fight the tide? Are you able to evaluate your own work, to go past the comfort and to find out how wrong you are, in just about everything you do? Are you then able to get past that mental scarring to do the same for others? Will you back down the first time someone pushes back, or will you make it your quest to do the right thing when it is neither profitable or easy to do so?

The history of business shows that rarely if ever does this problem truly go away, or does the better answer win. While history is written by the winners to justify their existence, randomness and the trampling of others are bred into every page of this twisted form of storytelling. And yet, until we deal with this real problem, until we are more interested in doing the right thing then the easy one, what will really change?

We will continue to waste time and effort on data, in order to justify wasted time and effort in most other efforts. We will continue to seek new words for old problems, and we will continue to make heroes those that most hold us back. Until we stop propagating the lie and until we look at ourselves first, how can we ever really deal with the real problem of data. Not the collection, not the sharing, not the presentation, but the people who are wired to use that data in the least efficient and most self-serving way possible. You want to solve big data, you want to change the industry, stop wasting time on tag management or Hadoop, solve the people, since there is where all problems lie. Don’t solve how do you share your point, but how do they think about and are they rationally using data to find an answer, or only to justify one?

Change the Conversation: Technology as the only “solution”?

As the world becomes more and more complicated, the battle between those on both sides of the functional knowledge gape becomes more vital. The need to constantly update your world view; the speed of change and the need to move past no longer relevant concepts leaves many people struggling to keep up and far more willing to listen to any offer to lighten their burden. When people do not know what they do not know, anything sounds like an oasis in the desert. To make up for this, there is a massive amount of groups who promise “solutions” to this fundamental problem, providing an answer of technology as the sole means to to make up for this fundamental inability to adapt to the constantly changing word. The problem comes not from this challenge, but from mistaking the solution being suggested for the sole requirement to the desired end result. Technology is part of a solution, but without understanding that your people must change; no technology will ever provide its promised value and very little will ever be achieved. When we fail to understand that change starts with how you think, not with what you purchase, we lose the ability to gain value from any “solution” that we can ever acquire.

The same cycle plays out time and time again. Senior executive defines a “problem”, such as a lack of clean data or ease of deployment of technology or the need to create a personalized experience. People proceed down a path of trying to both be the one to find a “solution” while at the same time finding ways to pass blame for the current state onto another party, be it internal or external, as any new solution must replace the prior “solution” that did not in fact solve all the world’s problems. They reach out and research and find a provider to give them a solution that makes the largest promise about “ease” or “functionality”, or the one that they have a prior relationship with. From there, it is a process of discover, promises, and then acquisition. The tool then gets shared to all the other groups, and the individuals who now find themselves with the task of trying to get this installed, also must make sure that their boss does not upset others in the company by instituting a change in the status quo. Each group provides “needs” in a one way direction that become part of a massive deployment project road-map. Groups continue to get buy-in and then try resources to deploy, each time acquiescing a little bit to each group they work with. Eventually the solution goes live, activities and tasks enacted, and everyone moves forward. The same problems arise a year or two down the line, agendas get forgotten, and large presentations are held to try and find a positive outcome for all that was invested. Very little has changed, very little has really improved for the organization, just a new piece of technology has been invested in to replace the old technology that went out the door.

I am in no way saying that technology is a bad thing, I work for the top marketing suite in the world, and wouldn’t if I did not feel that the tools themselves were best in class. Technology is simply a magnifying lens, increasing the impact of what your organization does great, but also where it fails. The reality is though that few companies get anywhere close to the value they should from the tools, and often that lack of value is accompanied by magnitudes of increased effort. If groups would start with a real honest change in how they understand the world around them based on each tool, they would find that they are wasting almost all their efforts in the vein attempt to justify their prior actions. Each tool is an opportunity to change and improve the efficiency of your organization, yet in almost all cases this vital task is talked about or ignored, and never enacted in a meaningful way. If you do not start your time with a tool with a discussion around what disciplines define success and failure for that specific tool, than no tool will ever do more than just be window dressing on bad organizational dynamics.

One of the first things I try to teach new analysts and consultants is that there is no such thing as a technical solution. All problems are strategic, they may have a technical solution, but they are truly strategic in nature. It is far easier to find a massively technical work around to do the one thing that senior VP X is asking for then it is to take the time to discover if that effort is needed or if it will provide any actual ROI. The unfortunate truth is that for a vast majority of the “problems” that are being identified, a successful or non-successful answer to that stated problem would have no change in the fact that they are not going to receive value. Slick interfaces don’t make up for poor strategy, integrations between platforms do not make up for not understanding your data. The truth is that in almost all cases the real problems are the ones that we are turning a eye from; they are the elephants sitting in the rooms that we refuse to talk about, so instead we make excuses and sacrifice outcomes in the name of taking credit for change.

This is the nature of confusing the “solution” for the desired outcome. Solutions are a means to an end, not the end itself. Never confuse the need to add functionality with the goal of that functionality. you are not just adding testing to someone’s day to day job, you are asking your entire organization to discover what the value of its own actions are. You do not just find a tag solution for the fun of it, you do it to save resources so that you can then spend them on valuable actions. You do not just start a personalization project from the goodness of your heart; you do it because you believe it will increase revenue. As soon as you keep the conversation about the end result, then you can have a functional conversation about the efficiencies of various paths to arrive at that point. Do you really need 90+ targets active, or would 15 give you a higher lift and much lower costs?

The cycle that technology gets brought into is the problem, as are the egos of those that own that purchase. Like most real world situations, it is far easier to make promises then to fix real problems or to deal with other groups and how they think about and tackle their own problems. Analytics, testing, and marketing are not things that are just done, even if your job is often times just a series of repeat activities. These actions are done to improve performance, which means that the change has to happen with which actions you are taking the resources to do, not just changing technology providers. If more time is not spent on reforming the enviroment around the technology, then all time will end up wasted. Never get caught up in the cycle and the can questions, without constantly keeping a vigilant eye on the should questions of all actions.

No matter if an idea is good or bad, it is always going to be easier to just do what your boss asks, and even easier to find a way to convince yourself that it is somehow valuable. We convince ourselves as others convince us that we are doing the right thing. We do not want to take the time to think about our opportunities to do things in a different way. Sadly most actions commonly done in our world are not valuable or efficient, and in all cases can and should be improved. You must first get over your own fear of doing the right thing before you can ever try and get those above you to do the same. The battles worth fighting when you bring in a piece of technology are not about how many developers can you get to deploy a solution, or how can you get an agency to run it for you, but in how do we find ways to fundamentally change current practices to learn and grow with the technology.

There is no shortage of people who are willing to promise that you don’t need to really look inwards to get value, and in some cases they are able to provide some momentary glimpses of value. Great tools offer you the chance to succeed, but do not guarantee that outcome. No tool will ever be able to make up for the incorrect application of its features, just as no organization will truly change unless change is more important than task. In the end though, every success I have ever seen or had with an organization comes from fundamentally challenging and changing existing practices and in creating simpler ways to get more value. Change is hard, most can not achieve it in a meaningful way, but all value comes from change. Not from creating complex ways to accomplish a task. Complicated will never mean valuable, complicated will always simply mean complicated. Never forget that a solution is a promise as a means to an end, and that the real ability to achieve that end, or more, comes from action, not from just a tag or from a solution being deployed.

Why we do what we do: Separating Fact from Fiction – The Narrative Fallacy

Stories are powerful devices that help get a point across to others. They help us close the distance between abstract thought and the way out brains operate through narrative. They help us add order to events. We can convey very complex ideas and help others understand them with our stories. Even more powerfully, this is how we are wired to understand events and information. But what happens if the story we tell is not the right one? How would you even know? Stories are often far more powerful at bypassing rational decisions then in facilitating them. The human mind is wired to support any conclusion it comes to, even in the face of mounting evidence against our supposition. Nassim Taleb describes this error in logic with what he calls the Narrative Fallacy, or the need for people to create stories, even if we do not have evidence that that story is true or even the best explanation of events.

Taleb’s description is as follows:

The narrative fallacy addresses our limited ability to look at sequences of facts without weaving an explanation into them, or, equivalently, forcing a logical link, an arrow of relationship upon them. Explanations bind facts together. They make them all the more easily remembered; they help them make more sense. Where this propensity can go wrong is when it increases our impression of understanding.

Here is an all too common example of how this plays out in the real world. You run a test and discover the a variant produces a 6% lift in RPV. you also discover that the same variant produces a 5% drop in internal searches. So you tell others that people were obviously finding what they wanted easier, so they didn’t need search and so they were spending more. The data presented does not tell you that, it only tells you that the recipe had a lift for RPV and a drop in search. It doesn’t tell you that search and RPV are related, or why someone spent more, it only presented a single data point for comparative analysis between default and that recipe. Any story you come up with adds nothing to why you should make the decision (it raised RPV) and it can set dangerous precedent for believing that dropping search always results in raising RPV (it might, but a single data point in no way provides any insight into that relationship).

Any set of data can be used to make a story. There doesn’t have to be a connection between the real world and the story we tell, since we are the ones that are filling in the gaps between data points. Randomness and direct cause are confused easily when we start to narrate an action. We love these stories because they make it accessible and easy for someone to hear, first event A happened, then event B, then event C. What happens is our minds instantly race to say, event B happened BECAUSE event A happened. Because event A lead to event B, then naturally event C happened. This might be true, it might not be, but the story we use and tell ourselves grants us the excuse to not understand what really went on. We are eliminating the discovery of the reality of this relationship by granting ourselves the story that fills in those gaps, despite its lack of connection to the real world. This action can completely ignore hundreds of other causes and also rules out the involvement of chance.

The world is a very complex place, and there is almost never as simple an answer as a simple series of events to explain any action, let alone one that would actually be important enough to make a business decision. So why then do we let ourselves fall into this and why do we fall back on stories as a tool to make those decisions? We are not actually adding any real value to the information from these stories, we are simply packaging them in a way to help get across an agenda.

We don’t actually need stories to make decisions, we only need discipline. Often times we find ourselves stuck trying to convey concepts beyond others ability to understand in short period and between many different draws for their attention, but this is not an excuse for us believing the fiction that we narrate. Many people base their entire jobs on their ability to tell these stories, not on their ability to deliver meaningful information or change. The reality is that to make a decision, you simply need the ability to compare numbers and choose the best one. I don’t need to know why variant C was better than B, I simply need to know that it was 5% better. Pattern and anomalies are powerful tools and the analyst best friend, but we can never confuse them with explanations for those events. Often times things happen for very complex and difficult reasons, and while it is nice if we feel that we understand them, it does not change the pattern of events.

One of the main opportunities for groups to grow is to move past this dangerous habit of creating stories, and to instead focus on create disciplined and previously agreed on rules of action in order to enable decisions to be made away from the narrative. This move allows you to stop wasting energy on this discussion and instead use it to think of better more creative opportunities to explore and measure the value of. Any system is only good as the input into it, so start focusing on improving the input, and stop worrying about creating stories for every input into that system.

Because of the difficult nature of this change for some groups, many are turning to more advanced techniques in hope of avoiding this bias. The fundamental goal of machine learning is to remove human interpretation of results, and to instead let an algorithm find the most efficient option. All of these system fail when we lose focus and get back into storytelling, when we let the ego of others dictate an action based on how well they understand the reality of the situation. They also fail when we allow our own biases to make the decisions over the system, instead of letting the system learn and choose the best option. When we free ourselves from storytelling, it allows us the freedom to focus on the other end of that system. We don’t need to worry about acting on the data, or in others understanding it; we can instead focus ours and others energy in trying new things and in feeding the system with more quality input.

Love your stories, and if you need them to get a point across, do not instantly remove them from your arsenal. Just don’t believe that they are conveying anything resembling the cause and effect of the world, and do not let them be the deciding factor in how you view and act on the world. They are color, and they make others feel good, but they add no value to the decisions being made. Be clear with others on how you are to act before you ever get to the storytelling and you will discover that stories are simply color. Every journey is a story, just make sure yours is less fiction and more about making correct decisions.