One of the great frustrations of an optimization group can be working with other groups to build out and optimize their current initiatives. While the natural outcome of this frustration is shy away and just to hand all control of testing to each group, result show that this is consistently the worst way to get value from your optimization efforts. To add to this some groups think the only way they can expand is to add resources, instead of improving how and when you do certain actions. In a lot of cases, the most important battles that you end up waging having nothing to do with getting a test live or with sharing results, but instead in helping grow a new discipline throughout an organization. So how then do you work within your organization to build out the correct disciplines while making sure that you are able to test as often and as much as possible?

While there is no single answer to solving this ageless riddle, there are a number of common factors that differentiate the groups that do get real value, from the ones that have to come up with stories to justify their existence. These can often be the hardest, slowest, and most political actions you take for your program, but in the end they are almost always the efforts that produce the greatest returns. The practice of optimization is about change, not just in elements in a user experience, but also in the quest to improve how the organization itself thinks about and tackles problems. How these specific disciplines play out will always take a unique aspect to match your current organization, but the core disciplines will always remain.

DO – Make education a primary goal of your entire program

From the start, be it single success metric or just why and how you test, you have to take control and help others understand how and why you are going to tackle problems. Stopping people from measuring the wrong things can be more important then what you do measure, but also requires you to stand up and have the conversation that the person asking doesn’t want to have prior to starting the test.

Testing seems easy, but the reality is when you are doing it right it can cause a lot of confusion and discomfort for a lot of different members of each team. You are directly measuring that outcome of things and adding accountability to opinions that makes almost everyone uncomfortable. You need to be cognizant of their worries and current focus while helping them understand why you need to do things. It is vital these conversations happen before you take any action, otherwise you will inevitably either sub optimize or even worse, get into a political battle between groups.

DON’T – Try to tackle the entire organization at one time

There is a famous saying, “How do you eat an elephant? One bite at a time”. The same holds true for growing your practice within the organization. Don’t try to convince everyone to do the right thing, instead focus on specific people or groups, and start with less high profile ones. No one is going to just change for the sake of change, especially since almost all change you are looking to create is against their own personal empire and agenda. Once you show success with those smaller groups, and you show how discipline played a big role in the outcomes, this will help you convince or deal with other groups and eventually you will find that the entire elephant has been eaten.

DO – Make sure that all testing conversations focus on a series, not on the execution of an idea

People always start testing thinking of ways to validate their single idea. They think X is better than Y, and want to use testing to prove it. This is without fail a massively flawed and inefficient manner to test, yet it is where a lot of groups stay due to the ease of just doing what others want. The key to any test is to make the learning, growth, and iterative improvement part of every conversation, starting at day one. Never let conversations just stand that are about which is better, but instead change them to the discovery of what is valuable and how do you exploit that information. It may cause a bit of discomfort at the start, but the reality is that if you do not make this a priority, you will inevitably start getting into the rut of single concept tests.

DON’T – Report everything that everyone wants

This is especially uncomfortable for analytics groups that try to add testing on as just a function. Part of optimization is the discipline to focus on what does matter, and to not pretend that you can answer the whole slew of questions that come. People bought more, but what did they click on? Where did they go? Why did they do what they did? The reality is that you have no clue even the correlation from a single data point, so that data is irrelevant. People spent more, but what product? Again, this will only cause conflict between specific product teams and do nothing to focus on the change that needs to happen.

The key is to focus on education and your single success metric to understand why you don’t do these things. It is ok to look at other metrics, so long as you do not draw ANY conclusions from them in a single test, and you look at ones that have a chance to provide value (this means not clicks). You will be able to spot patterns only after you collect a large number of data points and are disciplined in how you look at the larger picture of your optimization efforts.

DO – Make the analysis of optimization patterns far more important than a single test

Over time you will start to build out knowledge across tests. While it is easy to focus on only what you are dealing with in any given day, the reality is that the real lessons that you can learn don’t come from a single data point, but across large numbers of tests. These lessons, be it what types of changes or where you should focus, become the most valuable piece as they shape your entire future product road map. This means that you need to make this information available, but also champion these larger lessons and help others understand that specific tests probably won’t reveal as great of lessons.

DON’T – Just run an idea someone brings to you

Deconstruction, the ability to look at the assumptions that made someone come to a conclusion, becomes a vital skill in the hands of a great optimizer. Every time you tackle an assumption, you enable a greater possible outcome and also allow each test to possibly impact far more then intended. Want to test content to a specific user group on your front door? Does content matter most? How about which group? How about where in the user flow? You can easily start building tests to have enough variations and to tackle these larger questions, but only if you make it a priority. One of the great complaints of optimization is the concept of local maximum, but the reality is that this mostly comes from the limits of the imagination of the testers, and not from the specifics of the test itself.

I am sure that most people at multiple points here asked how realistic these suggestions are? There are hundreds of excuses why people do things, but the reality is that it is up to you to stand up and do the right thing, even if uncomfortable, at all times, otherwise you will find it nearly impossible to do the right thing when it is required. If you are proactive in your education, and make these a focus, you will find it is far easier to push back when and as needed.

There are hundreds of other possible things you can do to expand the optimization practices throughout your organization. If you simply nail these few items, you will avoid almost all of the major pitfalls that lead to programs becoming stagnant or pretending they are generating far more value than they really are. No step you take will be easy, but if you are consistent, up front, and make these actions a priority over the specific actions of running a test, you will be amazed at how far the entire practice has gone in just a few months.