How To Get Your Website A/B Testing Right?
First of all, pick a testing tool for your site. I mostly use Mocking Fish because of its user-friendly interface, and most of all- “Its free�. You can use others too, based on your knowledge and technical expertise. Though with Mocking Fish, I do not need any hardcore programming skill, just a bit of computer knowledge is enough here. In addition, I might suggest you to use Google Analytics, but it is a bit complex for the non-coders. Testing is not a joke, though a hardcore coding skill is not required by Mocking Fish, I recommend you to take a help of a developer when you perform an A/B test on your site. Testing requires an expertise if not coding then testing experience is a must required criteria. A/B testing produces some facts and figures as the results. So, you should also have some computational and statistical knowledge in order to get it right. If it is not your cup of tea, employ someone who can do it for you. Misreadings and misconceptions with the A/B testing results may cause serious aftereffects if you do not analyze it properly. You need to get a very confident testing result, the reason behind this need is that you might think the version A gives you most of the conversions while in reality, Version A itself might become the reason for most of the conversion failures on your site. So, it is imperative that you do not take it lightly. You might carry the A/B tests on your site for a year and still after a whole year, you might find yourself in the same place where you started. This is because, you did it wrong, you just wasted your time, money and effort.
You need to carry the test on a big enough sample sizeA/B testing produces more confident results when applied to the bigger sample sizes. A/B testing performs an analysis on the test results and presents a final suggestion based on the multiple statistical analysis. The results based on any statistical analysis are more confident when they are applied to the bigger sample sizes. The bigger sample sizes reduce the deviation of a sample from another sample. Thus, the average result is more precise. Therefore it is imperative to make sure that you have a big enough sample size on which statistical analysis could be done. When your Website A/B testing tool says there is a 95 % confidence level that your variation will beat the control, then it means that if the results are statistically significant, there is a chance that your change might give you the better conversion result than the original version. But, if the results are not statistically significant then, it might be caused due to any random factor on your site. Trusting a result without any statistical inference might not be a good idea here. That’s why I trust Mocking Fish tool, for a confident result based on the strong statistical inferences. Duration of the test is very important For any high traffic site, you can get the most needed size of the sample in a day or two. But the actual sample size is not only measured in quantities, it also considers the composition of your sample size. Let it be a thousand of visitors in a day or two, but the composition of your sample size is a combination of the sample size from the varied locations, times, seasons, night, day, your PR activities, your Advertisements etc. A complete sample for A/B testing includes a balanced combination of all the above factors. So, for a valid A/B testing quantity of traffic + composition of traffic including a complete cycle of business phases, should be met. So, considering all these factors while employing an A/B test on your site could help you get a more confident test result that can improve your conversion rates significantly. Include these factors in your testing activity and ensure that you do not fall a prey to the cases of False Positive or False Negative. You should imperatively give importance to the Statistical significance of your test results by carrying out the tests with proper iterations and proper sample size and composition.