How to Test

There are a number of different ways that you can conduct a TMT cycle.  Before we look at this though - a very common and valid question is: how long do I need to run my test before I can trust the results?  Or how many visitors do I need to have to each different recipe before I can trust the result of the test?
This is very difficult question to answer as it depends on the noise and the difference in the results between the control and the recipe.   Good testing software will tell you the ‘Confidence Level’ that your test has achieved (i.e. 60%, 90%, 95%, 99% etc).  However if you don’t have sophisticated testing software, as a general rule of thumb, I like to run my tests on a minimum of 500-1000 high quality visitors (quality prospects – not random clickers etc) before trusting the results.

Let’s take a look at the different ways to conduct a TMT cycle.

1. Basic Testing
   Run the control – run the recipe.  This sort of testing is very susceptible to the effects of noise.  Be very careful and think through any outside noise that may impact the results of the test.  And make sure you run large sample sizes to keep the confidence level of your tests nice and high.
    
2. Split Testing
   Split testing allows you to run the control and the recipe(s) at the same time with the traffic split between the two.  This helps you minimise the impact of any noise.   Split testing requires specialised software, but as long as you have it the software is simple to use.  If you are new to testing, split testing is a good place to start because it is easier to get solid scientific data, gain some testing experience and avoid many of the common pitfalls.
    
3. Multivariate Testing
   Multivariate testing allows you to test many different variables at the same time using sophisticated statistical analysis.  Multivariate testing allows you to optimise a page as quick as possible but it requires powerful software, careful planning and it is complex to set up a successful TMT cycle.