Hold Out Test Reports
In this article, we'll learn more about hold out testing and how test results are calculated.
Hold out testing is the practice of measuring the purchase behavior/value of a control group who does not receive marketing against a group of customers who do receive marketing. The goal of the test is to compare the two based on the value of an individual customer within each group. These tests answer the question, 'Am I creating a more valuable customer by marketing vs. not marketing?'
Hold out tests are comprised of three parts:
Hold Out Test Report
Interpreting Hold Test Results
Sends: The total of number of emails sent to the Active group. The Holdout group will always receive 0 emails.
Customers: The total number of customers included in each group
Revenue: The total revenue generated by each group, regardless of open or click behavior
Revenue per customer: The total revenue generated by each group during the test divided by the number of customers in each group
Lift: The percentage improvement / lack of improvement of the Holdout group compared to the Active group