Introducing A/B testing
Measuring the effects of updating your content
link{a}list.io
The Startup's Online Toolkit
January 31, 2019

Linkalist offers landing page functionality and in the best tradition of dog-fooding, we use this feature to present our own landing page and general product information pages. It recently came up that I wanted to try out a few changes to these landing page. So the obvious thing was to look into what was out there and see if we could put together a minimally viable version of an a/b test using the tools we have already.

So the first thing we need to do is set up some test groups so that we can arrange our landing page articles into groups. Each of these groups is given a weighting so that when a potential customer arrives on our site, they get allocated a test group according to a random selection based on this weighting. They will remain in this group for the duration of the session. So far, so good.

Next, we need to assign our landing articles to groups. In general, if we’re only changing a single article, the best thing to do is just put that article into the group and then copy the group. Then you’ll end up with a second version of the article that you can change as you see fit to make up the test.

If you’ve set up your application to use Google Analytics, you’ll need to go in to the Custom Definitions section of your Property and add in a new Custom Dimension called “Test Group”. Once you’ve done this, linkalist adds the correct Test Group to all page views and events so you should be able to construct reports based on the test group and so verify which test group is performing best for you.

There is also very simple reporting built into your registration list as that will record the page and test group associated with the registration so that you can see where it came from.

This all needs to come with the usual health warnings for A/B testing in that you need fairly substantial volumes of data coming in before you can make any judgements on the effectiveness of changes.

You can also add blog posts into test groups to conduct experiments on the relative effectiveness of various blog posts. For now, A/B testing can only be done on content but we’ll be adding functionality to allow you do the same with design changes on your landing page.