If you think about the trinity of key elements to a successful optimization organization, the one that really determines the ability to drive the others are the people. It’s not enough to get people that are skilled or know the tool, those can be trained; what is most important are to find people that will and can challenge the status quo and help change your organization for the better. You need people whose mission in life is to find better ways to do things and who do not accept the status quo. If you keep doing what you were doing, how do you expect to get better?

Your people and the roles they fill, are what will allow you to change your organizations view of testing and organize in efficient ways to accomplish tasks. So what people do you need? What roles do you need to fill and how do you know that you are doing the right things in those roles? There are common key roles within each testing program. All these roles might be filled by one person or they may be filled by 5 people each. It is not the number of people, but how they think and act that is most important. What matters most is the ability to put the right hat on when needed and to think and act in a way to be successful. Success here is defined as making the company the most money in the most efficient way possible. If your goal is not to assist your company in making money over making yourself look good, then please ignore the success and failure part of this post.

If you want to succeed, you need each of these roles to be identified, leveraged correctly, and to work together. Having just one bad apple, or letting someone turn into a villain, in your group will guarantee that you are limiting your results.

Program Sponsor –

Role – This is the person whose neck is on the line to make things work. They are there to align teams and to get everyone working towards a common goal. In order to accomplish this, your sponsor needs both the willingness and power get change done in your organization. They do not need to be involved in every day to day activity, but when it comes time to make sure that other groups are not upset by challenging their imminent domain and to make sure that there is not infighting, these need to be the real leaders to get that done. They also help make sure the other roles are doing their job and making sure that the program is not being used to just to make one person or group look good. Their last duty is to make sure that things don’t just work right one time, but over the long haul, and that the program is assisting the entire organization to change how it views and accepts testing.

You will be successful if – You take the time to understand that disciplines of testing and how to make it work in your organization. You make sure people are held accountable for their actions and for delivering results, and make sure that there is constant efficient effort with all parts of the group. You stop people from just trying to make stories up about how “right” they are and instead actively try to get people to appreciate being wrong. You make it clear that you and the team are not there to be yes men to other groups, but that by working together that you can all work towards a common central goal.

You will fail if – You don’t get involved or get too involved. There is a fine line to setting the stage and allowing others to do what they need. You will also not succeed if you allow the program to cower to other powerful people’s whims or do not deal with the need for IT, design, analytics, and marketing to work together towards a single common site goal. You will also fail if you are more interested in building your empire then actually getting results. More often than not programs fail when people try to make things look better for them over actually achieving results and if you are not willing to stop that behavior, then you will not have a successful program. If you accept just getting an outcome over getting the best outcome, then your program will never be truly successful.

Testing Owner –

Role – This is the person who runs the day to day operation of the testing program. Unlike the sponsor who sets the higher level agenda and holds people accountable, your testing owner is the person who makes the magic happen and makes sure that the program is run day to day. Liaising with other groups, setting the daily agenda, making sure consistent resources are leveraged and making sure that the program does not get too caught up individual tests or ideas are your primary duties. In order to do that, you will spend a large majority of your time educating and working with other groups to change their perceptions of what makes a successful test, that you are measuring the value of every action. Creating the momentum and changing people’s perspectives to insure a consistent optimization process and not just a series of projects is your day to day directive.

You will be successful if – You can navigate the political waters and keep the many plates spinning. If you can do all that and make sure that every action is not wasted on non-strategic things, then you have a real chance to be a rock star. You must work proactively to educate people about the disciplines of testing and how it is both different and valuable to the other disciplines in your organization. Being proactive and not letting others set the conversation based on their agendas is the primary directive of this role.

The number one challenge is not the day to day activities, but keeping the conversation at the right level and not allowing any of the small details such as technical implementation, test ideas, resources, or questions from outsiders to derail the program. It’s easy to do once, but doing it every day for months and years is the real discipline.

You will fail if – You are more interested in playing nice or promoting yourself then getting the right answer. If you get caught up in the concepts from reading the latest blog or conference or thinking that just getting a single answer from a test somehow makes you successful, then you are on the path to failure. You have to stop yourself and your team from thinking that just the act of getting a result somehow makes you successful.

Most programs fail to see higher value because the people in this role are so desperate to get a single result that they try to make it like other groups, or they start rewarding other groups for just finding something better and not for finding the best use of resources. They get so frustrated by the politics that they start making excuses, to others and especially to themselves, that they have accomplished more then they really have. More then anything, it is the failure to stand up and to do the tough things like challenge people and not allow popular opinion to drive things, even if they might help you politically get ahead, that is the leading killer of programs.

Technical resource –

Role – This is the person who knows how everything gets done and can make magic happen with code. They own the creation of test code and the process by which is works with other code on the site. They should be part of every discussion working on the efficiency of efforts and talking about the best of the 50 ways to accomplish this task. They can also be helpful in liaising with other IT resources to make sure that everyone is on the same page.

You will be successful if – You are part of the conversation but understand that just because you can do something doesn’t mean you should. You will need to make sure that we are looking at things from how they fit together in a discipline based view as opposed to what can we do. In order to do that, you will be part of the conversation talking about efficiency of not just doing one thing, but making sure that all tests are deconstructed and that you are testing multiple feasible alternatives, not just what is being asked originally of you. They are also strongly working with and educating the IT group to make sure that other projects do not interfere with and are enhanced by your testing efforts, nor are testing efforts slowed down by getting stuck in IT processes designed for very different types of projects.

You will fail if –– You think that your job is just to do what others tell you and that process matters more than outcomes. You will also fail if you try to do testing like other IT projects, especially QA. Many people will want to reward you for making their ideas come to life, but fundamentally if you are not stopping bad ideas, if you think it is someone else’s problem, then you are promoting yourself and not adding value. If you cannot find a way to make things work pragmatically in a speedy manner, then you have no chance of succeeding.

Optimization Analysts –

Role – The optimization analysts is the person who owns the reports and the outcomes of the testing program. They may also be the one that sets things up in the tool and who helps project manage day to day activities. When it comes time to make calls on outcomes, they take lead, creating reports and helping educate others on what was learned from each and every action. They will need to know a little bit about many different things and should be able to hold people accountable to established rules of action, while at the same time talking to other groups on their level.

You will fail if – You set the way that things are done before a test, not after. If you don’t cower to every question that is asked or pretend that you know something that you don’t. If you are willing to remind people to focus on what matters and not just dive into every question because you could find some data. If you work tireless in finding better ways to do things.

Ultimately, you are successful here by what you convince others to not do far more then what you do.

You will not succeed if – You try to answer every question after the fact or if you allow others to change measurements to meet their personal agendas. If you do not understand the biases of people and how they are trying to use your and the data your provide for their own self-promotion. If you are not willing to step up and challenge people or to remind them of what matters, then you will never succeed. If you get caught up on the latest “buzz” or “expert” and forget to work towards the right goals, then you will also not succeed.

Designer –

Role – You work with the testing team to create the materials needed for each test. You make sure that you are testing not just the concept but also the execution of each concept by producing multiple ways for each item thrown your way. You own the relationship with your design teams and you work with the team to make things look good enough to go live on the site, while making sure that you are open to new things that don’t meet current standards. You will be the one making sure that you have a large amount of different feasible alternatives for every effort. You will also be the one helping communicate what is learned and how it changes or breaks current “best practices” or style guides for your group.

You will be successful if – You make sure that you are testing out everything that is feasible and that the alternatives are different enough from each other. You get excited by the chance to try new things and know that the worst thing you can do is try to get “approval” for ideas.

You will fail if – You get too caught up in what has been there before or style guides or “experience” or any of a hundred other excuses and fail to test out things that go outside of everyone’s comfort zone. You don’t get excited by the challenge of being creative and get too caught up on requesting or wanting feedback. You will also fail if you allow tests to get too caught up on content and fail to address the other much more efficient parts of the user experience such as real estate, function, and presentation.

There are of course many other roles and the nature of each of the ones listed is different for each organization. What does not change are the disciplines to succeed. Make sure that you are aware of who is doing what for your group, and even more importantly you are doing the things to succeed and avoiding the pitfalls that each person is bound to come across. You won’t be perfect day one, but that doesn’t allow anyone in any role to avoid the responsibility of getting better and making things work for you.