Confidence Interval
A confidence interval gives an estimated range of values which is likely to include an unknown population parameter, the estimated range being calculated from a given set of sample data. Best practice is to use a 95% confidence level for direct marketing.
The 95% confidence interval can be interpreted like so:
Suppose we test two executions (A vs. B) and see that A delivers a response rate of 1.20% with a sample size of 60K and B delivers a response rate of .90% with a sample size of 60K. By using the confidence interval calculator provided, we know that the real response rate seen can range from a high of 1.29% to a low of 1.11%. Similarly, the confidence interval around B can range from a high of .98 to a low of .92. The two confidence intervals don’t overlap. We can conclude that Execution A outperformed Execution B as the results are statistically significant at the 95% level of confidence.
This is a fancy way of saying that if we repeated this test 100 times, pitting one execution against another, 95 times out of 100 we’d expect results to come out the same way, with Execution A winning against the Execution B.
Sharing is nice. Please share with a friend or colleague.Related posts:
Related posts:
