Marie Despringhere, UK Country Manager, Optimizely discusses how failed tests can actually bring the most value and that it is how you learn from the tests that can have an impact.
The results of website testing are often surprising. Who knew that simple images can perform better than elaborately produced videos and that sometimes even headless models can attract more website visitors to the order page?
90 percent of all A/B tests do not achieve the expected results and as a consequence are frequently deemed as “failures” by the managers responsible.
However, just because a test shows negative or unclear, statistically insignificant results, it doesn’t mean that it failed. But what does “failed” actually mean, and are these tests really negative? Optimizely UK Country Manager, Marie Despringhere, discusses why failed tests can actually bring the most value and that it is what and how you learn from these tests that can have an impact.
Failures as the drivers of success
Even James Dyson, British entrepreneur and inventor, once said, “I love failures”. And it’s hard to argue with him – believe it or not, he built 5,216 prototypes before he invented the foundation of his success, the bag-less vacuum cleaner. The ingenious inventor Thomas Alva Edison, known for his many failed attempts before his invention of the light bulb, also had an opinion of failures – “I have not failed. I’ve just found 10,000 ways that won’t work.”
A/B testing is always an experiment, so it can sometimes seem difficult to know where to get started. But remember, tests with “negative” results or without clear winners are not insignificant.
On the contrary – they often provide the company with important insights about their visitors and planned product developments that they would not have otherwise considered. Additionally, a negative result proves that your test influences user behaviour, even if not as expected.
So-called “negative” results are a basis for targeted follow-up tests and pave the way to successful optimisation by drawing the right conclusions and incorporating these into subsequent tests.
If clear alternatives such as other images or colours are available, you have the opportunity to turn a negative result into a positive one. So, even if your colleagues and employees are sceptical, don’t be deterred if the new variant has not transformed into a winner compared to the original.
Remember the bigger picture
Always remember that in its very nature A/B testing is a process that is generally successful in several stages; approaching your goal through a series of tests is often the best way to make sure you don’t get too many surprises at once. This way, you can check which changes have a negative or positive effect and respond accordingly. In addition, smaller findings and positive feedback can be incorporated more simply.
Don’t get bogged down in one area of your website. Sometimes, the test object or area is already well optimised – for example, if many iterative tests have been carried out on the same test idea or in the same area. In this case, it is more worthwhile to concentrate on other areas of the website that have a clear potential for optimisation.
Getting the team on board
Finally, to make your testing campaigns and their results as impactful as possible, you should discuss all the tests that have taken place at regular meetings with the rest of the team. Check that there is a balance between simple and elaborate tests, tests to increase yield and to reduce costs, and tests initiated by different departments. Also make sure that the negative, positive or “insignificant” results of the tests already carried out are included in subsequent tests.
For the success of your testing program, it is also crucial to establish a testing culture that always focuses on the goals of your company. Keep a broad overview and never be discouraged by individual results – on the contrary, use them as an opportunity to find out more about your visitors. These insights into the expectations and preferences will ensure that your investment in testing is worthwhile.
By Marie Despringhere
UK Country Manager
Optimizely