These are A/B tests and this is why you want to use them

A red button or a blue one, which delivers more conversion? And do you really raise performance with a different algorithm? Or do you wonder whether that web app would benefit from new navigation?

When you develop something new, or adjust an existing product, you want one thing: that the new version scores better. So you need to check what actually works better, and that is where you use an A/B testGlossary · In briefA/B testAn A/B test is an experiment in which two groups each see a different variant. Comparing their behaviour shows which variant performs better against a predefined goal.Read more⁠. It is that simple.

What are A/B tests?

You probably want to know which design choices really steer your end users in the desired direction, and an A/B test, also called a split test, is a fairly simple method for that. In practice you randomly split your visitors into two groups, group 'A' and group 'B'. Each group then sees its own variant of your digital product. By comparing behaviour side by side, where people click, the routes through the application, the time they spend, or pure load time, you quickly see which of your two versions scores better on the criteria you care about. Simply evidence-based development.

Why do you want to use A/B tests?

Suppose you want to place a contact button in a web application. The list of design questions then rolls out on its own: bottom right or centred on the screen, which colour works, do you put an icon next to the text, and does a subtle arrow on that button even deliver more clicks? An A/B test effectively forces you to back every guess with evidence you generated yourself, because you simply run a real-world trial. Especially in larger e-commerce environments that feeds straight into the margin. A conversion shift of just 0.1% can easily deliver a solid revenue increase for the company.

When are A/B tests relevant?

That is why we usually weave A/B tests in as a normal part of building new features or switching to a completely different site. We then find out reliably whether those changes actually contribute to better performance, and we finish a project with an end product that has simply passed the market test. That lets us steer at every moment toward what your business benefits from most. The annoying part is only that you do need a healthy visitor volume for it. Twenty testers. Then nothing but noise rolls out of your statistics and you get nothing useful from it in the end.

What about the risks?

With A/B tests you decide yourself how large or how small you make it. That is what makes it so flexible. You research a detail, such as a button colour and its effect on clicks, but you can also go all out and give group A a radically different interface from group B. Is that new frontendGlossary · In brieffrontendThe frontend is the part of a website or application that users see and operate, such as pages, buttons, forms and interactive screens.Read more⁠ a failure? Then you would rather not have everyone drop off. Precisely for that reason you set the size of both groups smartly in advance. You limit the risk by exposing a smaller share of your visitors to the test. During the run you follow the numbers live, adjust where needed, and you simply configure at which result you increase or decrease a group's traffic percentage.