When it comes to businesses, the internet has been a complete gamechanger. Almost every successful company has some form of online presence. Websites are an effective method of attracting more customers if done right. Getting it right is where A/B testing comes in. As it stands, the average conversion rate of website visitors to customers is just 2 per cent. That means that 98 per cent of visitors to a website are not converted to customers!
So how exactly will A/B testing help? A/B testing will allow businesses to develop and optimize their websites through simple testing and evaluation. In doing so, they can evaluate the effect each version has on users thereby identifying the best version. This will improve the effectiveness of a business’ marketing and user experience and will lead to substantially higher conversion rates than the average 2 per cent. The authors of this book are co-founders of the leading A/B testing platform in the world and who better to learn from than the experts?
In this summary readers will discover:
- What is A/B testing and how does it work?
- Why you need to be prepared for change
- Getting your website right
- The importance of failure
- Spreading the word about data-driven approaches
Key lesson one: What is A/B testing and how does it work?
To put it simply, A/B testing allows you to show varying forms of your website to different focus groups. Whilst this is done, data is collected regarding engagement and reactions to the different versions. This data will allow you to select the most effective version.
A working example of this was demonstrated by Siroker whilst he worked on Barack Obama’s 2008 campaign. The campaign website originally allowed users to make donations to the campaign or sign up for a newsletter. In order to do this, users of the website had to click on a ‘sign up’ button which was on a page with an image of Obama amongst his supporters. So the team in charge of the A/B testing tried out different combinations of photos and buttons. After testing the various combinations they found the best combination to be an image of the Obama family with a button that read ‘learn more’. This combination resulted in visitors being 40.6 per cent more likely to share their email address. In real numbers, this was 2.8 million more subscribers and $57 million in donations.
Impressive right? The positive impact that A/B testing can have on your business is massive. Even more appealing is that it is a lot more cheaper to do now than it was previously. This is due to the fact that it is now a well-proven method and there are designated platforms that make the implementation of A/B testing easy.
A clear plan of what you would like to test and the results desired are important points to remember before you implement A/B testing. This is crucial to establishing the type of data that will be collected and how the success of your website will be evaluated. These are referred to as quantifiable success metrics. For example, for an online magazine, clicks would not be a good metric for judging the content of the site. A better approach would be to look at comments, shares and repeat visits to know which content is received well. Likewise, an e-commerce site will probably judge success by a completed purchase as compared to just looking at the number of visits to the site.
So, ensure that you have your hypothesis and success metrics before you start. They will keep your tests focused and prevent you from randomly testing features.
Key lesson two: Why you need to be prepared for change
You could start off with the intention of just making minor changes to your website but A/B testing could lead you to other unexpected changes. A/B testing could provide data that could lead to changes in your site’s structure and design.
Take for example Disney’s experiment on their homepage for ABC family. The data they received from their search logs revealed that visitors to the site were looking for specific shows. Because of this, they revamped their entire homepage and created one which listed all the shows on the network which made it easier to navigate. Disney’s success metrics were to increase the number of clicks by 10 to 20 per cent but instead, their change led to a 600 per cent increase!
This was a major result derived from a major change. The effect that A/B testing can have on your site and business is huge because it will highlight details you might have missed before. It’s also worthwhile remembering that you can actually continue A/B testing whilst you are redesigning your site. Netflix is a good example to demonstrate this as it was exactly what they did when they redesigned its user interface in 2011. Originally, it suggested just four titles but the now popular rows of thumbnails and titles proved to be the best variation when A/B testing.
Key lesson three: Getting your website right
There are a few things about getting your website right that should be intuitive. No one wants to visit a cluttered webpage that has images and pop-ups fighting for their attention – it’s an immediate deterrent. The rule of thumb for website design is that less is definitely more. A/B testing further fortifies this rule by clearly demonstrating that the elimination of unnecessary fields has a major impact on engagement.
To evaluate and declutter your own site, along with the elimination of unnecessary fields, you can also break up long forms into multiple pages or make use of the hide function. The hide function will enable you to hide fields that are not currently needed. You can easily unhide them when they are. This makes things clearer on your site and free of deterrents to users.
Successful implementation of this was once again demonstrated by the author during Obama’s 2012 re-election campaign. When they could not cut back on any of the information given on the donation page, they instead made the submission form look shorter by breaking it down into two separate pages. The changes made during this optimization stage result in $190 million in donations!
Another equally important approach to optimizing your website is to ensure that you are using the right language in your text. The idiom may be that actions speak louder than words but in this case, you need the right words to drive the actions of your users. The use of verbs over nouns is recommended in order to achieve this. You have to tell users to do something, not ask them. This is where the term ‘call-to-action’ comes into play. LiveChat’s visual designer tested a simple switch-up of words to test if verbs could increase customer action. To do this, during a free trial campaign she changed the button which said ‘free trial’ with one that said ‘Try it Free’. The result? A 14.6 per cent improvement in click-through rate and all it took was a verb!
You also have to make the meaning of clicks clearer. Take for example the Clinton Bush Haiti Fund. Instead of a ‘submit’ button on the donation page, the A/B testers tried a ‘support Haiti’ button. The change in language led to this variation drawing in more donations per page view.
Key lesson four: The importance of failure
When it comes to A/B testing, failure is an important step in the process. When your variations fail, they still provide valuable data which, although not important now can be useful later on.
e-Commerce company Chrome learned this first hand when their attempt at using videos to promote their products had absolutely no effect on conversion rates after 3 months. As much as this was a failure in terms of their success metrics, they also learned that video use in future will not have any impact on conversion rates.
Failed A/B tests will also give you valuable information about your users. Take gaming website IGN for example. They did a simple test to see if moving the ‘Videos’ link in the navigation bar from the right to the left had any impact. It sure did. A 92.3 per cent difference in fact – but in the wrong direction – their clicks decreased by this number. This may have been a complete failure in terms of website optimization, but it enabled IGN to realize that most of the site’s users were return visitors. These return visitors couldn’t find the link in the usual spot and this resulted in the decreased traffic.
Failures in A/B tests will therefore always give you important information one way or another so don’t be discouraged by them.
Key lesson five: Spreading the word about data-Driven Approaches
A/B testing has many advantages but in order for it to be known, you have to spread the word. This is especially true if you want to use it in your organization and you want your colleagues to truly believe in it. The easiest way to do this is to present examples of A/B testing and its results – and to do this often. This continuous positive reinforcement will convince stakeholders that this data-driven method works and it works well. This will ensure that they will be more receptive to A/B testing and what it can achieve.
Lizzy Allen did this when she first joined IGN in 2010 as an analyst. Surprisingly, no one at the company had heard of A/B testing before she introduced it. She introduced the method to the company over a period of one month and initiated the A/B Master Challenge to keep them engaged in the concept. The challenge allowed IGN’s staff to vote on the results of the A/B tests and to predict which one would be most successful. Not many people made accurate predictions but what it did achieve was enlightening staff members of the benefits of data-driven approaches and what they can achieve. This led to the easy adoption of A/B testing by the company.
The key takeaway from A/B testing is:
A/B testing is an easy and affordable way to test different variations of your website to ensure that you have the most effective version. Changes are usually small but sometimes, the data you collect may result in a complete restructure of your website. Either way, the benefits of A/B testing cannot be denied. You can increase traffic to your site, optimize the content or increase revenue. You might be surprised about how much you learn about users and your site!
How can I implement the lessons learned in A/B testing:
Before you set out to begin your A/B tests, ensure that you have a hypothesis which you would like to test and your success metrics clearly defined. This will keep your tests and results focussed on one area. Analyse the data you collect to find out which version of your site works best and don’t be discouraged if none of them does. This will still provide you with valuable information which can be used later on. Also, remember that A/B tests could reveal faults in your site that were not noticeable before, so be prepared to redesign your site if this happens.
