As with any other type of marketing, a tailored, customized email marketing campaign made specifically for your audience will be more successful than a broad marketing message. But, as most email marketers will tell you, figuring out what works for your particular set of subscribers can be a long, arduous process. There is no magic formula to creating an email marketing campaign – you can rely on common elements and a smart design, but even then your campaign may not be having the impact it could. A/B Testing is one way marketers try and fix this issue. When done properly, an A/B test will give you concrete metrics as to what did and did not work with your subscribers – data you can then use to build your marketing campaign. Novice marketers, however, may not fully understand A/B testing, and so to help them out, here are the answers to five commonly asked questions about A/B testing.
1. What, exactly, is an A/B test?
An A/B test is also known as a split test because it tests the impact of two, different versions of a particular element of your marketing campaign. Say, for example, you aren’t very confident in your subject lines and you want to try something different. An A/B test randomly splits your subscriber base into two groups, allowing you to send one version of an email with a standard subject to your control group, and another version with a modified subject. You can then use standard marketing metrics like how many times the email was opened to determine what version worked better.
2. How do I create an A/B Test?
Just like with any experiment, you first need to have a clearly defined hypothesis; if I change X, then Y will happen. The proving or disproving of that hypothesis is what your conclusions, and future recommendations, will be based on, so make sure your hypothesis looks at clearly defined, testable factors. You also want to make sure that all other elements of the experiment are “normal” – so avoid doing things like testing during the holidays, or during any other time your subscriber base might be behaving differently than usual. As for the actual campaign, most marketing email sites and applications give users the option to run an A/B test, and might even analyze the results for you, so click around your favorite app or site and see what features it has.
3. How often should I perform one?
Really, as often as you think you need to. A/B testing needs to be an ongoing process. Remember, though, that the longer you let the test run, the more reliable your results will be. Don’t rush through the testing process. Take your time, note any changes, and continue to experiment.
4. What part of my email marketing should I test out?
That is up to you, though like we said, it should be something that is testable against a control. Images vs no images, calls to action in the subject line vs no calls to action in the subject, putting the subscriber’s first name in the email vs putting in no name at all. After you do the broad tests, you can start looking at more specific variables, like the types and sizes of images, or different fonts. Just make sure you only test two things at a time – a changed version of the element being tested, and a usual version.
5. How do I know if my results are good?
The reliability of your results will depend on how many people were a part of the experiment and how long it ran. No matter what you do, there will always be problems or errors that need to be accounted for. However, the more people that take the test, the less those problems will actually impact your results. In statistics the probability that your data occurred by chance is called a significance level, and having a significance level of less than 5% means your results are statistically significant and you can be confident them. Most email marketing services and sites will spit out a significance level at the end of the campaign, but there are also calculators online that will tell you if your results are statistically significant.
Conclusion
A/B testing has become one of the most widely used tools in email marketing impact analysis and, if done properly, can really help increase your opens, click-through, conversions, and ROI. When designing your experiment, remember to clearly identify the specific element of your marketing that you want to test, and don’t be afraid to let the test go on for a couple of weeks. The more data you collect, the more confidence you can have in your results. Then, when you figure out what works, it’s just a matter of incorporating that into your normal marketing practices.
