As personalization and testing continues to become more and more mainstream you are starting to see a whole slew of groups that are being introduced to testing, or who may believe they have more functional optimization knowledge then reality. So many groups would get be better off if they just avoided a number of common pitfalls that befall new programs. While earlier I put together my list of the top 7 deadly sins of testing (no single success metric, weak leadership, failure to focus on efficiency, testing only what you want, confusing rate and value, and falling for the graveyard of knowledge), I want to instead give you a very quick checklist to make sure that at least your first few efforts in testing are not more painful and far more fruitful then what normally happens.

It does not matter if you have tested before, or are new to testing. What matters instead is how you tackle today’s issues and how you set yourself up to succeed. Breaking down the components of a successful first test allows you see the action items, and allows you to move towards the moments that really make or break a program. Nothing makes everyone’s life harder than starting out testing on the wrong foot, sine everyone will think that is how things are going to happen from then on. With that in mind, if you do nothing but follow this simple checklist, you are going to be far better off.

1) Decide on what you are trying to accomplish – Pick one metric for all your tests, across your entire site or org that defines success. This might be easy or hard, but it is vital. This is especially difficult for people coming from an analytics background who are used to reporting on multiple things and finding stories.

2) Pick one or two pages to start – Do not try to eat the entire elephant in one sitting.

3) Do not focus on your test ideas – You are going to have test ideas in mind, and you are going to want to only talk about and focus resources on that one test. Without fail this is where most groups want to focus, but I cannot stress enough how not important your concept for improvement will be to a successful program.

4) Make sure your first test is very simple – Don’t start trying to do a full redesign of your entire site or completely change a user flow. Pick one page and one thing to start. If you are not sure, then pick a page and design your first test to measure the relative value of all the items on the page.

5) Decide on your initial segments – Make sure all tests have a segment list. Not being focus on a specific segment will make it far easier to find exploitable segments and will start the process of learning even if you do not intend to use them right away.

Here are some basic rules for segments to make your life easier:

• Must be at least 5% of your total population (10% of a smaller site). This is total, not identified traffic
• Must have a comparable measure (you can’t measure new users versus Google users, since there are new Google users).
• Have to be defined PRIOR to the start of any campaign
• Need to cover the 4 big user information types (the listed items are just examples):

Temporal –
Day parting
Day of week

Physical –
Geo location
Browser
Operating system (basically all the stuff you get from the user agent string)

Referrer –
How did the user get to the site
Key word types
Referrers
Channels

Behavior –
New/returning
Used internal search before
Purchaser/non purchaser

6) Start building a proper infrastructure – Build out a technical frameworkfor your entire site for testing. This may not be tied to the first test, thought that will be part of it. Get access so that you can run tests on 80% of your pages, using a combination of local and global set-up. A little pain up front will save you from lots of pain later on. It is always best to avoid death by a thousand cuts whenever possible, even if you don’t see that issue immediately.

7) Decide on rules of action – Make sure everyone is very clear on how you are going to call a test and act on winners before you launch your first test.

8) Making sure you are not going to QA tests like you do other parts of your site – So many testing programs are destroyed by letting IT decide on the work flow and on how you are going to QA your tests. You may have to work with your IT group, but it is vital that testing is not owned by your IT group but instead your marketing and product teams.

9) Point your resources towards feasible alternatives, not adding more metrics – Any additional time and additional resources need to be used on the following two things:

A) Adding as many alternatives to the test as possible, traffic permitting
B) Educating and working with groups to make sure they understand why you are testing and how you are going to act.

10) Remember that the most important time for your program is not the first test – Your first test is fun to focus on, but the period after your first test is where you can start to really focus on testing discipline. This is the time that defines whether you get good at just running tests, or if you build a rock star optimization program. So many groups fail because they miss how vital this time is.

There you go, 10 simple steps to make sure that your first moments in testing do not take your program astray. This is hardly the end of the road, but if you simply avoid setting yourself up for failure, then you can really start to look ahead to all the opportunity that is out there.