26 August 2020 Luke Goldsworthy

The Value of A/B Testing

Many factors contribute to the success of an eCommerce store. Even beyond personnel, you have got to consider your sales conversion rate, average order value, cart abandonment rate, price… and so many more factors. Consistent use of A/B testing can help you optimise these metrics.

Put simply, A/B testing compares two or more variants of something by exposing them separately to different subjects selected at random. Through the statistical analysis, you can determine what response each variant received. These tests can be run over and again to refine the result against new variants.

For example, in eCommerce, you may want to test which image in an advertisement would generate the highest click-through rate. To do this, you select two different images and create separate ads for each of them. Run both ads for a period of time, then compare the click-through rate of each to determine which image was more effective. 

Each A/B test follows the same general steps, following the scientific method:

  • Ask a question (What image will generate the highest click-through rate?)
  • Do research (Which two images shall we test?)
  • Run the test (Ads with both different images are run at random)
  • Examine the results (Which image performed better?)
  • Implement change (Use higher-performing image) 

Please note that you must run all tests for a sufficient time so that there are enough responses to provide an accurate indication of it. The smaller the sample size, the less accurate the result is.

Here are two areas where you could consider running an A/B test:

Product Page

A product page has multiple elements that are ideally suited to A/B testing. Among the most important of these are page layout, product images, product description, and inclusion of expert tips and reviews.

Another is the Call to Action. Too many product pages may contain ‘boring’ calls to action despite their importance in the conversion process. You can look at improving your CTA by running a test between the existing button and a newly designed button. Sometimes small changes such as the wording on the CTA, its font and size, and even its colour, can have an impact.

Once you analyse the results and know which button was more successful, you should try to determine why, and then apply this knowledge into creating a more demanding A/B test to further hone in on the best outcome. For example, once you are confident that a style and colour is producing the best results, you may then want to consider adding elements such urgency to the button, telling customers the offer is only available for 24 hours.

Free Shipping

There are many options when it comes to shipping. In a post in June, we discussed the importance of providing multiple shipping options as part of the best checkout practice. Equally important is the cost of shipping. Unexpected shipping costs remain one of the top reasons for cart abandonment.

A first test to consider would be comparing the conversion rate of a product based on whether its base price included shipping costs, or whether the shipping was added as an additional cost during checkout. 

A second test could analyse the effect on the conversion rate of creating a threshold value for free shipping against no free shipping, and subsequently testing different threshold values if the test favoured that option. 

If you do decide to go with free shipping, make sure this is highlighted on your site.

We’ve just briefly looked at a couple of areas where you could consider running an A/B test, to give an indication. Still, there are many more valuable areas to test, including email marketing, page layout, pricing, CTA format and placement, and of course ad campaigns. Doing some simple A/B testing could be a major factor in retaining a customer through their interaction with your site or going elsewhere.

Remember, the use of A/B testing isn’t a one off.  By using some simple tools and processes, consistent testing leads to better results and a more efficient eCommerce site and team.

If you want help determining which A/B tests would be good for your eCommerce business to run, click below to set up a no-obligation free time with The Playhouse Group to discuss it now.

Get in touch with our team and we can talk to you about this or any eCommerce questions you may have.