A/B testing is a method that will help you when making a decision about the final form of the service you want to deploy. You will let your users choose, which version is more suitable for them.
You will needDesign A and Design B
Timeapproximately 24 hours
Peopleat least 50 users
Difficulty2 – depending on what tools you use
- choosing between two designs
- you want to find out if the new version works better than the original one
- you know what goals the designs should achieve
Do not use when...
- you have only one version (try User Testing instead)
- you have many versions to choose from
- you cannot say who is your user going to be
- Design two new forms of the service or compare the already existing one with the newly designed. You can also, for example, examine information leaflet or website of an institution.
- The testing should be conducted together with users who are going to actually use the product or service. Do not leave the selection and addressing of the tester for the last minute.
- Divide the users in two groups. One group will work with Design A, the other one with Design B.
- The power of A/B Testing lies in the quantity. When testing websites or apps, you should certainly take advantage of tools that will make the subsequent analysis easier.
- Ask both groups to fulfill the same task or determine the questions and hypotheses that should be tested, e.g. how many users will manage to order your service or product.
- In case they are assigned a task, observe the how they cope with fulfilling the task and select the version that was easier for the users.
- Evaluate your questions and hypotheses.
- Use the more successful version in the actual practice.
Keep in mind that...
- you can test more than two versions
- a more complicated testing of several versions can be also conducted with conjoint analysis
- it is worth to monitor not only the results, but also their statistical relevance