A/B Tests

Overview

A/B Testing your broadcast emails can drive significant incremental performance improvements. The article will walk you through test types and the process of setting up a new test step by step.

Test Types

Rejoiner offers two different types of A/B test types:

1. Split Tests: Standard split tests enable you to deliver a control creative and a variant creative to specified sample size. A common practice would be to send the control creative to 50% of your list and the variant creative to the other 50%. The important distinction here is that standard split tests will send your control and variant to the entire list, all at once.

2. Controlled Split Tests: Controlled split tests enable you to run a smaller test with a subset of your list, then deliver the winner to the remainder after a specific wait time. With this test type, you'll configure both the size of the initial test group and how long Rejoiner should wait until the winner is selected and sent to the remaining customers on your list.

Configuring A/B Tests

1. On the Creative Options page, select A/B Test Broadcast as your Send Type. If you’d like to reuse an existing test, select one from the dropdown menu.

2. To set up a new test, click Create New Test.

3. Give Your Test a Descriptive Name: Try using a standard format like, “2020/01/01 CTA Copy Test”.

4. Choose Test Type: There are two possible test types to choose from:

   • A/B Split Test: Select this option to test two variations against each other, split across your entire audience. For example, you could send Email A and Email B, split up 50/50, 80/20, or even 90/10 across your entire audience.

   • Control Group + Delayed Winner: Select this option to test two variations against each other, split across a small segment of your audience. The winning variation will then be sent to the remainder of your audience. For example, you could send Email A and Email B to 20% of your audience. The winning email would then be sent to the remaining 80% of your audience following a set period of time.

5. Set Test Completion Rules: Once a winner has been determined, you have a few test completion options. What you choose here mainly concerns whether or not the test report itself will continue to aggregate performance data once a test winner has been determined:

   • Automatically deliver winner only: Selecting this option will cause the test report to stop aggregating data once a winner has been determined.

   • Notify me and continue testing: Selecting this option will continue to aggregate data in perpetuity.

   • Notify me and pause testing: Selecting this option will cause the test report to stop aggregating data once a test winner has been determined.

6. Set Creative Details: Creative details can then be set for both your Control and Variant.