Category: Psychology
Why we do what we do: Not everything is a nail – Maslow’s Hammer
At any given time you will often hear me quoting many famous quotes about anything and everything; that is how I relate to new experiences by trying to tie them to some bit of knowledge that I had already picked up. Probably my most common refrain lately is the famous Mike Tyson quote, “Everyone has a plan until they get punched in the face.” It’s true that everyone talks doing the right thing and everyone wants things to succeed, but as soon as there is some challenge to the prevailing world view or as soon as a small bump in the road exists, people often revert back to what they know best and turn back in on themselves. Unfortunately this is especially problematic in the business world as the only way to move forward is to change behaviors and tackle existing problems in new ways. Even more distressing is that as people fall back to what they are most comfortable with they turn towards their own disciplines and their own previous experience, limiting the ability for people of disparate talents and backgrounds to work together.
One of the things that defines people is the concept of viewing the world through their own experiences, and the most powerful experiences that we have in the modern world is our professions. Be it marketing, or engineering, management or data, we all view the world through the lens of the things we do and the challenges that we face day to day. We view the challenge of improving numbers by looking to “dialogue with our customers” or “increase efficiency through data analysis” or by “building better tools and a better user experience”. All of these in isolation seem like and often are very good ideas except when they cloud our ability to prioritize and to focus on a single outcome. Each day in the business world is really a Sisyphean climb to the top and each time that boulder rolls back on us we run back to that which we are most comfortable with. This is especially dangerous when we do not even have true accountability for the tie between those concepts and the functional bottom line outcome that we need to generate.
Abraham Maslow is famous for many things, from his hierarchy of needs to his many contributions to modern psychology. What he is often not associated with is a quote that almost everyone is familiar with, “if all you have is a hammer, everything looks like a nail.” We are all carrying hammers in the form of our world views and our professional disciplines. The key is to accept that there are many things outside of what we accept as “true” about the way to do things and about how to tackle problems. Even more when we do get evidence that does not directly correlate with our existing world view we can not dismiss it or try to understand it through that same tired lens.
Optimization at its core is the act of adding accountability to these world views and about challenging assumptions. It is about taking the existing practices of the entire organization and standing them on their end, shaking them, and finding all the holes and least effective parts. It does this not maliciously but as a mutual benefit to everyone to add a different point of view on the functions and actions that they are taking. This is why the discipline of testing is about everything but test ideas. It is about building rational rules of action and building out alternative hypothesis. This is why you focus on efficiency and multiple options and not just on what won and not about what won elsewhere or about some great idea someone had. It is why it is about patterns and not about some artificial reasoning why something won. You serve the organization a great discipline when all you do is regurgitate the nail back to someone so that they can then hit it with the same tired hammer. Optimization is the act of putting any idea and discipline through a system that allows for it to get better and for everyone to learn and to get better results.
At the same time, it is important to understand that everyone else is viewing the world through a very different lens. They are trying to tie their past experiences with new actions and new results. A marketer has always thought in terms of a dialogue with a certain user or a certain persona. That mental model has gotten where they are today. When you come in and show that there might be more effective ways to look at those same users or the the concept of personalization most likely will not work the way they envision, you are creating a very powerful form of cognitive dissonance and you are forcing people outside of that hammer that they so readily wield. Too much and you will cause major push back and possibly form an ongoing barrier to success. Too little push and you are just confirming their biases and not providing any assistance.
The key in this and in all actions is to be firm on discipline but flexible on tactics. Work with the concepts and push them past their existing barriers. This is why it is so vital to not focus on test ideas when building out a successful test. Talk about what people were already focusing on and how best you can test out that concept against many others. You want to do personalization, great, here is how we take what you were doing and serve that and other concepts to everyone. If you are right, we get to see that and if you are wrong then we found something that is better. In reality there is no downside to performance when we tackle a problem that way. It is about reaching the ends, not about the means that get you there.
Another key to this is to get people to vote on what they think will win for each test. If you do this enough and with enough varied options and you will be amazed at just how bad people are at guessing the right answer. In the last 9 tests that I have done we have averaged 8 options for each test, with some variants coming from the team, some from myself, but a large many simply expressions of the various directions that are feasible. I have asked a large team to pick there favorite and second favorite. In those 9 tests, we have had exactly 1 second place vote for all of the winners combined, and the only reason that the option got that vote was because my very talented designer picked up on the pattern and voted her least favorite. The shock of where we are versus where people thought we would be and the impact to the bottom line (over 200% improvement) has helped open doors to new ways of tackling problems, and it has done so organically.
In both tactics you are giving people the chance to tie their world view in with the results and letting them have a stake in the outcome. You are welcoming that hammer they wield but helping them see that there are many different nails to hit.
Keep in mind however that you are just as guilty as they are. Spend too much time in the world of optimization and you will start to feel like no one has any idea what they are doing and that all ideas are going to fail. It is even more important for you to challenge yourself and for you to go beyond your comfort zone in where you let testing going. Make sure you include ideas from others as much as possible, even if you are sure they are not going to work. Make sure you tie optimization in on actions that you feel might not comfortable or worth your time. Remember that the smarter someone is, the more likely they are to be impacted by biases and that you serve no good to the organization or yourself if you are not more vigilant against your own biases then you are against others.
Why we do what we do: Why do we Shoot the Messenger? – Backfire Effect
At some time, everyone that works in data has had to deal with the following scenario:
You run a test or you do an analysis that shows that a member of management has been claiming something or pushing something that is clearly wrong. You present the data, and then they push back even harder saying that you just don’t understand or there must be more to the story. You dive back in, find more and more supporting data, you make charts and breakdowns and present them again. This time instead of just pushing back your recipient start attacking you and everything you do. They may do it overtly or behind the scenes, but they now view you as a problem and a threat. They never change their view of your original point, and now they distrust you and are looking for opportunities to attack your work.
This is a way too common outcome in the business world, and one that is not actually limited to the use of data. What you are experiencing is the Backfire Effect, or the fact that people become stronger in their beliefs when presented with evidence that directly contradicts them.
So why does this happen? Why is the data you are clearly presenting, data that multiple others agree with and buy into not having its desired effect? It is because you have started to attack their world view. Every person you ever work with believes that they do superior work, believes that they make a large impact to the business, and that they hold a deep understanding and correct view of how things work. When you present direct evidence against this, you are not actually attacking the statement, but their self-perception, which creates a level of cognitive dissonance, resulting in an ad hominem attack on the messenger, and a blind ignorance of the evidence.
Like most psychological biases the key is to set the stage for success prior to action, not after. You may not be able to force rationality into individuals or organizations, but you can certainly push discipline. Define rules of action before you start and task, work to get agreement on what will define success, and what follow up action should and will be. Often times these conversations are pushed, ignored, or dismissed, but it is up to you as the one who will ultimately be sharing the news to force this as a priority of a conversation.
No one you work with will want to talk about how you make a decision; they will want to talk about their great idea for a test, or for a group to target to, or their amazing advertising campaign. They have already decided what they want, why it is great, and what you will present in the end. If you only allow or enter the conversation at this point, you role in their subconscious mind is simply to validate their opinion. The job of those that work in data is to never give into this path, no matter how easy it is or how it may help us politically. You can not view success ever as how many actions you fulfill, but instead the value of the ones that you fulfill. The instant you allow for quantity of action to take precedence over quality of outcomes, you are setting yourself and others up for this type of failure. It is instead to be the holders of discipline, to be the ones that help create opportunities to find out the faults in these ideas, to not be the ones to validate held world views.
This is also why changing the conversation about what it means to be “right” and “wrong” is so important. If you shape each conversation to talk about the amazing outcomes of being “wrong”, of going in a not previously encouraged direction and about the impact to the business, you are opening the door for individuals to not have their world view attacked. If you allow others to understand that they have impacted the business, and that they have succeeded in their end goal of finding out people cases where they are wrong, you have enabled them to not fall into the Backfire Effect. Changing the conversation away from the faults of one idea and towards the value of different options and why choosing this action allows you to not attack someone’s world view and instead help them look good by giving them the tools to find an outcome, not just an input to a failed system. It is important that you understand deeply why you need to do this, what the traps are, and what the right way to frame that conversation is, but if you are willing to do the ground work you can achieve amazing results.
One of the defining characteristics of organizations who get value from their data versus those that don’t is that the leaders who manage their data focus on the leading conversation, not on the stories they can tell after their analysis. This problem is only exasperated by egos and by the fact that so much of the material and talk in the industry is filled with justifications for those that do not want to address the real issues at hand. Much of the data marketplace, from managers to agencies, is filled with those that would come up with creative ways to tell people exactly what they want to hear and to come up with a story that shows impact, even if there is no factual basis for that claim. There are articles, speakers, and “experts” throughout out who have mastered the art of sounding intelligent without actually adding anything new or functional to the organizations of which they address. There are many groups who have their own biases in believing their value is presented, just like any other group, because they focus simply on the actions some takes or on their ability to make a recommendation. your key responsibility is to focus the same skills and control the message in the same way towards that which will actually drive value for the organization, not that which sounds good but is hollow. It is vital that from day one and onwards that leaders control and help shape the conversation instead of responding continuously to requests. Successful organizations define actions and successes, focus on discipline, and prepare for action before the data, not after.
There is no more true statement then: “Success and failure is determined before you act, not after.”
There is zero chance of you avoiding push-back if you fail to do the dirty work of setting the stage properly. If you create an environment where you don’t focus on the idea but instead on the discovery, on the outcome and not the input, and work with groups to add value to their ideas instead of facilitate their ideas, you will find amazing results achieved throughout the organization. Ultimately you need to be agnostic about what wins and loses, and instead focus on how people arrive at a decision and if it answers the correct business question. Shy away from this aspect of the job and you face the challenge of dealing with the backfire effect or finding ways to justify actions you rationally know are not valuable.
If you want to avoid painful confrontations, you will always have two options. Option one is the easier one, convince yourself that presenting data that supports people politically or that just getting someone to act is somehow providing value. In this option you will never be delivering news people don’t want to hear. The second option is to focus on the painful disciplines prior to actions and to deal with some discomfort before you get too far and stay away from anyone’s ego. In this you will have to deal with some discomfort, but you will be able to make a true and meaningful impact to your organization. Other departments, executives, and even your own management will never be able to make this decision for you, this decision is a personal one and one that you either choose to make, or one that is chosen for you.
Why we do what we do: What do you Really Know? Dunning – Kruger Effect
Most people are familiar with the famous Bertrand Russell Quote, “The whole problem with the world is that fools and fanatics are always so certain of themselves, and wiser people so full of doubts.” The challenge is how would you understand where you are in that paradigm? Are you the fool or are you the wise? How do you even know your real level of competence at any one moment? What is the difference between an expert and the average person?
One of the most famous psychological studies in the last few years is the Dunning-Kruger Effect. Its best description is:
The Dunning–Kruger effect is a cognitive bias in which unskilled people make poor decisions and reach erroneous conclusions, but their incompetence denies them the metacognitive ability to recognize their mistakes. The unskilled therefore suffer from illusory superiority, rating their ability as above average, much higher than it actually is, while the highly skilled underrate their own abilities, suffering from illusory inferiority. To put simply, when you don’t know what you don’t know, you have no ability to differentiate what is right or wrong.
Confidence is a tricky thing, as you need it to be able to stand up to challengers, but at the same time you need to be careful to not inflate it based on impression and not reality. Without confidence, we would never be able to convince others of any point we are trying to make. We have all dealt with people who obviously talked much more about their impact then could possibly be based on reality, but how do we know that we are not repeating the same mistake. Even worse, how do we know when others are playing on this psychological trick to take advantage of us, even if they do not consciously know they are doing it at the time?
The truth is that you will find far fewer real experts then those that claim to be. Statistically, an expert would be in the upper 5 or 10% of a certain field, yet we both have no way of measuring this and we are over run with experts claiming to be the best at what they do. Everyone thinks that what they are doing is the best way, otherwise they wouldn’t be doing it. Even worse, we get caught up in all sorts of promises, be it other customers claims, or their own skill set to evaluate that information. As the world becomes more complex, or as people from outside disciplines attempt to take their prior knowledge and apply it to a new field, they become even more susceptible to this problem. Even worse, like all biases, this impacts the more intelligent people more then the less intelligent. Dunning-Kruger is a double edged sword, as those that are most likely to be susceptible to the claims of experts are those that are least skilled in their own right:
“The skills needed to produce logically sound arguments, for instance, are the same skills that are necessary to recognize when a logically sound argument has been made. Thus, if people lack the skills to produce correct answers, they are also cursed with an inability to know when their answers, or anyone else’s, are right or wrong. They cannot recognize their responses as mistaken, or other people’s responses as superior to their own.”
This entire phenomenon is what causes the vicious cycles and what explains the over saturation in the analytics communities of people propagating the same tired actions by giving them new names and by finding others to make them feel good about their failed actions. Stories become the ultimate shortcut to show how amazing something is, without ever actually providing logical evidence to arrive at that conclusion. The truth is that people are rewarded for their ability to give people who already hold or held power ways to continue to run their empire, often with little relevance to getting results or to doing the right thing. This is certainly not unique to analytics, but it is important to this audience. We follow the greatest speakers, not the greatest thinkers. We worry about the best ways to make a presentation or to get reports out, not trying to stop entire conversations that are both negative to the organization and inefficient. We focus on how do we get others to see things our way, not on if we are seeing things the right way? Far more time is spent on how to convince others then it is on trying to analyze our own actions. One of the best quotes I have heard is, “There is no correlation between being a great speaker and having great ideas.”
So in the end, we are responsible to ourselves to look in the mirror and ask if we are suffering from a belief in others, or if we are discovering the best answers. Is convincing others and yourself that you are expert the most important action you can take? Just because you can make an analysis and use it to convince others, does that actually make it correct or valuable? Or is the discovery of the next right answer more important then getting credit for owning an action? It is up to everyone to decide what they really want to accomplish in their time, but if you are more interested in doing the right thing, then you must always be aware of Dunning-Kruger.
In order to do this, we must first set rules that help us hold ourselves and others accountable for what they do, in order to remove as much bias from evaluating success as possible.
Here are some simple steps to help make sure you are reaching the levels of success that you might believe that you are achieving:
1) Always ask, “In what ways can we challenge what we are doing?” or “How can I break this process”? No gain comes from doing things the exact same way you have been doing them.
2) Read, grow, look beyond your group. Know that you have never found the right answer, and the search is more important than the actual answer.
3) Define success up front. This is not just the goals your boss sets for you, but more importantly what it is that will define a successful program?
4) Make sure you are not measuring the outcome, but your influence on the outcome.
5) Seek out those that will challenge everything you believe. You do not need to agree, but only talking to like minded people is the fastest way to become the observed with Dunning – Kruger.
6) Assume that if you have not found a way to break a process in the last year or two, that you are not trying hard enough
7) Challenge everyone to take an idea to the next level. The first thing we come up with is comfortable. The next is growth.
8) Know that you will get an outcome from any action, so measuring just that does not tell you anything about the value you bring
9) If the words “I don’t know” are the end of the conversation for you, then you can be sure you are the sufferer of this bias.
10) Most importantly, change all the rules, and challenge all the rules, not to be difficult, but because you only get better by making others around you better.
These may seem like abstract general concepts and not directly related to your business or your day to day job, but the reality is that these are the actions that should define success there far more then the outside world. Growth is the goal, not the status quo, and as such we need to make change and going out of our comfort zones the priority, not re-wording past actions as new in order to convince others or yourself you have changed. Take others past their comfort zone and they will take you past your own. Keep getting better, and always know that you are never done and that you do not have the “correct” answer. Keep searching, and always question those around you, and you will always be vigilant against falling into the wrong end of Dunning – Kruger.
Testing can be the ultimate expression of this, you are free to test things far past your current comfort zone. You are free to not validate tired ideas but to explore and discover in a rational and predetermined way the actual value of things, not just the perceived value. In order to do this though, you must fundamentally want and prepare to discover these things. The greatest problem with most test programs is they never enable themselves to find out they are wrong, but instead focus on proving someone right.
The nice part is that just because you or someone you know suffers from Dunning – Kruger, it does not mean they always will. Every person you meet thinks they are doing the right thing, even when they are not. Change the conversation to the end goal, and talk about all the options that are in front of you, and you get past the egos that keep conversations from truly moving forward. Take the time to talk and to challenge people, and do not trust anyone that does not challenge you. You have many impartial tools that allow you to measure things and to work with others, but these tools only work when we use them in an unbiased manner, not to tell us what we want to hear.
Why we do what we do: You are Not so Special — The Forer Effect
While the quest for personalization may be newer to some members of the online marketing world, the reality is that is a concept that is old as sales. People have been trying to convince others that they alone were getting a special deal or that their message was meant just for them. One of the great practitioners of this concept was P.T. Barnum, who famously billed his circus as “we’ve got something for everyone.” On some level everyone understands the appeal of being special and of having someone take the time to tell me something that is unique to just me. The greatest salespeople though understood one of the great ironies of personalization, which is that general statements, when given in context, often are treated as deeply personal and are extremely powerful. This concept is known as the Forer effect, or more directly, the tendency of people to interpret statements as being accurate for them personally, even when they are not.
The Forer effect gets its name from B.R. Forer and came about from a series of experiments that he performed in 1948. His famous study involved giving a personality test to all of his students. He told them that they were all receiving unique personality analysis, and they were to rate that analysis on a scale of 0 to 5. All of the students actually received the exact same results, using such lines as, “While you have some personality weaknesses you are generally able to compensate for them.” and “You also pride yourself as an independent thinker; and do not accept others’ statements without satisfactory proof.“. Despite the exact same statements being made, his students average score for his “analysis” was 4.26.
The favorite trick of psychics, conference speakers, and astrologists, this psychological bias is important to understand, especially when thinking about the concept of personalization. General statements hidden to look like targeted messages have much greater impact than direct statements, and are far more likely to increase belief in the speaker. Personalization, as it turns out, is about not being personal, or at least not to the Nth degree. Personalization is about the match of the general and the pseudo specific, and it is about taking that message to the largest group possible, not just the ones that directly match the message. The more we can measure different types of messages, and the more we can find the largest groups that respond to them, the better our results, since functionally the more targeted the message, the less overall gain we get to improving total site performance.
So you might ask why is this so important in the quest for personalization? This bias tells us that overly thinking personalization and designing a large number of specific messages is both a waste of resources, but also far less likely to create a positive outcome. It also tells us that the message itself does not have to match the rules that dictate the outcome; general statements have an impact for a large variety of people, not just a specific targeted group.
As you start thinking about and tackling your personalization programs, it is important to understand the nature of why you are doing these actions. Barnum knew that he was there to sell his circus, and every action he did had only that outcome in mind. He was one of the most famous practitioners of a single success definition, and he knew that no matter what he did needed to drive more to spend more on his circus. The same is true of all online efforts. Your goal in the end is to make more money, and the key is not to focus on a specific message, or to over rely on experts or correlative information to tell you when and how to target. The key is to test out all sorts of possible content, and to see how you can best present them to people to allow you the efficiency of largest group of people possible.
This is why a message about a specific product may work best for Firefox users, why time of day may be the best match for your re-targeting content, and why smaller segments are so inefficient. It is also one of the main reasons why targeting content without the discovery process of the value is far more likely to lose you revenue than generate more. It turns out the more you try to narrow a message or assume an outcome, the worse your results will be. Somewhat specific messages work for far larger groups than you could ever imagine, and you only know the true power when you let go of your own ego and preconceived notions and explore.
Stop thinking of personalization as trying to build a one on one message with a customer, that does not work and is extremely inefficient. Instead explore the various ways that you can create different content, and then explore who the largest groups are that you can present that to. This means always going through a discovery process of figuring out what matters, and then figuring out for whom. You may want to target to only people who looked at brand X, or page Y, or who have done a come to your site 3 times without purchasing, but that in no way means that you should limit the message to just that group. The less control you exert on the specifics of a message, and the more you are open to new possibilities, the more likely you are to find larger and more meaningful outcomes.
Explore what the value is of different messages and of taking it to different groups. You have powerful tools at your disposal to do just that, to discover and take these more general statements to large groups. From simple A/B tests all the way to automated machine learning, the real key to value comes from how you think about the problems, not in your ability to just find a group and target to it. Not only that, but you have the ability to measure the efficiency of various discoveries and techniques against each other. You are not limited to creating these stories, or just targeting to a specific persona, you have so much more at your disposal if you just allow yourself and others the flexibility to learn and grow.
P.T. Barnum is also famous for how he could get people to pay for anything, with the most famous example being the egress. It wasn’t meant for anyone specific, but he could get just about anyone to fall prey to the mystery. He didn’t have to target that message to just one group, or to offer it for only people who were on their way out, he figured out how to take that message to everyone. He understood that just because a group might be inclined for something, that just limiting your message to that group was a waste of his time. He was the ultimate salesman, but he knew that the key was to make it look like you were walking a fine line and being extremely specific, while at the same time in no way going that far.
So the question comes down, as you explore personalization, or you selling the egress? Or are you the one on your way out that door?
TL;DR 