A/B Testing

A/B testing is one of the best ways to determine the impact of your advertising is having. This form of testing can be used to see the effect of different creative messaging or overall impact of advertising so is applicable to many situations. The basic approach is dividing your audience into two (or more) groups treating the groups differently and evaluating the differences in their performance. 

To do A/B testing with Thirty-One Circles, there are four simple steps:

  1. Create your base audience.
  2. Create two (or more) Hybrid audiences from your base with different A/B testing groups in each.
  3. Treat your A/B test audiences differently in your advertising. Expect this stage to take at least a month.
  4. Compare the results with the audience comparison tool.

WHAT ARE A/B TEST GROUPS?

In Thirty-One Circles A/B testing groups divide all registered users into one of four groups (A/B/C/D), each containing 25% of all your customers. Once a user is in a group, this never changes regardless of your targeting rules. Therefore, someone in Group A is in Group A across all your audience rules and will never move, making them perfect for A/B testing.

When you select audience groups in your targeting, you will notice the size is described as "approximately 25% / 50% / 75%", as depending on your specific targeting, there may not be an equal group of A/B/C/D customers. However, this variation is typically minimal and can be accounted for in the analysis.

TEST DESIGN

Before starting A/B testing, we advise making sure your audience setup will give you the clearest results. To do this, we recommend checking two things:

  • Is your audience big enough to be divided and still targeted effectively? This will depend on your platform and use case, but we recommend checking the audience size against those live to benchmark expected performance. 
  • Is membership closed (so the audience doesn't change across the test)? To do this, use a fixed date range in the past and don't target any extension attributes that might change.

To evaluate results after running your advertising. Use the audience comparison tool to compare the different results and relative to audience size.

We recommend starting the time range for the A/B test analysis at least a few weeks after you start treating the audiences differently. This gives a cleansing period for attribution windows, making the results cleaner and easier to interpret.

More questions?

Contact our support team at support@thirtyonecircles.com who are always ready to offer advice and assistance.