In my quest to tackle the major biases that plague our industry, the second one I want to tackle is Expectation Bias or “the tendency for experiments to believe, certify, and publish data that agree with their expectations for the outcome of an experiment, and to disbelieve, discard, or downgrade the corresponding weightings for the data that appear to conflict with those expectations“.

In statistics and math, you deal with the concept of expected outcome. You take the payout, factor in the percent chance of it happening, and then factor in the number of chances you are taking to reach there. This is the exact reason that the lottery is such a bad investment, because it has a massive outcome, but such a low likelihood of reaching that conclusion that your expected payout is never going to happen (Neglect of probability bias). Just because one person did win the lottery does not mean that most people win the lottery. The same is true for just reading the stories or copying others. There is a want to push out and tell people about how advanced you are, as well as the want to see immediate payoffs (hyperbolic discounting). There is also the need to distinguish yourself or your offering and to discount others if an action only works 1 out of 20 times, and even then you don’t know if it is the best outcome even for that group, what possible good is it going to do you?

Obviously, the problem then becomes knowing the likelihood and the relative value of the outcome…

Good luck getting that information if you don’t have direct interaction. Not only are people hesitant to share if they know, but people are inherently wired to not seek or know that information themselves (choice supportive bias). What does happen however is a giant group of ankle biters… people or groups (especially agencies or “experts”) who promise that they have all the answers, who tell you that if you listen to them, everything will be golden, and that all of their clients are super successful. It is not evil, it is not even that they know what they are acting selfishly, they are just wired to think in these terms, as any human is to want to believe them. This phenomenon and the play of it on the human psyche is a recurring theme and one that leads to very inefficient and under performing programs, where people are left either defending poor results or seeking yet another magic answer to all their problems.

Successful consulting and successful programs focus on the disciplines of success, not just the actions. They talk about both good and bad outcomes, and how to get more of one and less of the other. They focuses on what defines successful actions and alternatives, not just what you are doing or a single possible alternative. Challenge anyone or anything that only talks about “success stories” or who can’t talk about how often things work and how that compares to alternative processes. There exists a feedback loop where bad practices are shared and championed just because they are being acted on, and not because they are valuable. It allows them to tell stories that show that they got someone to do… something. It is up to you to stop your path down that mobius strip, and instead challenge yourself and others to think differently and to think in terms of finding a better way and a better answer. Programs have to be able to learn, grow, adapt and deal with real world problems, which means that any single “answer” is just playing to your hubris and leaving you open to future failure.

One of my favorite sayings is “Information is not knowledge”. Seeking out better information and challenging the stories and practices of others, challenging your self to learn and grow and change your biases and your want to be “right” is the only way for your program to truly move forward. Focus on the system, focus on learning, focus on fixing why you do what you do, and never stop, never let go, and never let yourself fall for these traps. That is how you succeed.