A/B Split Test Reports

In this article, we'll dive into the reporting that's available for your A/B & Hold Out Tests.

Overview

    You can access the data for an active A/B Test at any time in one of two ways:

    1
    From your Email Campaigns Manager, click See Test Data on the test you'd like to monitor
    2
    From your dashboard, click Test Results, then choose your test from the drop down
    Note: Once a test has been activated, all future campaign stats are aggregated under the  Test Results tab and are not available under the Email Analytics or Campaign Results tab. Those tabs will continue to provide reporting on campaigns that are not actively being tested.

    A/B Testing Reporting

    Rejoiner's A/B test reporting uses send, open, click, and conversion events to calculate the confidence intervals, conversion rates, and statistical significance. An example of aggregated reporting for a 50/50 Split Test is shown here:  Rejoiner also provides reporting on an email by email basis as shown below. This allows you to compare each email in your campaign: 

    Interpreting A/B Split Test Results

    1. Variation: this column reports the name of the variation for a particular row
    2. Sends: this column reports the number of emails sent 
    3. Clicks / Opens / Conversions: this column reports the number of events that occurred relative to your specific test goal
    4. Open Rate / Click-through Rate / Conversion Rate / Revenue per Customer: this column shows the percentage of events that occurred relative to the number of emails sent
    5. Lift: this column reports the percentage lift of the Variant compared to the Control
    6. Confidence: this column reports the significance, or how different the confidence interval for the conversion rate for the Variant is when compared to the Control (this must be at least 95% confident before being flagged as significant)

Still need help? Contact Us Contact Us