{"id":11643,"date":"2016-08-18T09:01:07","date_gmt":"2016-08-18T14:01:07","guid":{"rendered":"https:\/\/www.hostgator.com\/blog\/?p=11643"},"modified":"2016-08-18T09:01:07","modified_gmt":"2016-08-18T14:01:07","slug":"what-we-learned-from-ab-testing-over-300-million-emails","status":"publish","type":"post","link":"https:\/\/www.hostgator.com\/blog\/what-we-learned-from-ab-testing-over-300-million-emails\/","title":{"rendered":"What We Learned From A\/B Testing Over 300 Million Emails"},"content":{"rendered":"

\"AB<\/p>\n

Take a look at the two emails below. Which one\u00a0do you think performed better, the one\u00a0on the left or the one on the right?<\/p>\n

\"Email<\/p>\n

If you guessed the one on the left, you were right!<\/strong><\/p>\n

We found the email with more text had a 11.83% lift in open rates and a 22.49% lift in click-thru rate. \u00a0The open rate is more of a red herring, as readers can’t tell until they open the email what the contents look like. But the nearly 23% lift in click-thru rate makes a strong argument for readers preferring the email with more textual context for the images.<\/p>\n

Having more text that explains the images could help in cases where the HTML doesn’t come through properly on an email reader, as it gives readers more information about the value they can hope to gain from the linked content.<\/p>\n

You can see that both emails have different formatting, as well, so that could also be contributing to the increase. We would need to test out more of the individual elements to narrow down how much of the increase was due to the heavier text.<\/p>\n

At HostGator<\/a> we send millions of emails in a year. With each email comes a new opportunity to test something. The example above illustrates multivariate testing<\/strong> (which involves testing multi<\/em>ple varia<\/em>bles), but today we’re going to focus on A\/B testing<\/strong>.<\/p>\n

What Is A\/B Testing?<\/h2>\n

The A\/B test is a simple tool that businesses of all sizes use to make their email marketing more effective. If you\u2019re not familiar with A\/B testing, we\u2019ve put together this guide to show you why A\/B tests are so useful and how to get started. <\/span><\/p>\n

A\/B email tests are real-world experiments you create to see how changing one element in an email such as the subject line, graphics, or timing affects the way recipients respond. Each test compares only one variable (A) to only one other variable (B) to keep the results clear and easy to interpret.<\/span><\/p>\n

For example, a caf\u00c3\u00a9 owner might create two different subject lines for the same email newsletter and then send version A to one part of her list and version B to another part. If subject line A gets noticeably more opens and click-throughs than B, the caf\u00c3\u00a9 owner knows subject line A is the best choice to send to the rest of her list and to serve as a template for future subject lines.<\/span><\/p>\n

Speaking of subject lines, it’s time for another test. Take a look at the subject lines below. Which one do you think resulted in higher open rates – the one on the left or the one on the right?<\/p>\n

\"Subject<\/p>\n

In this test we tested\u00a0subject lines that highlighted different aspects of our cloud hosting<\/a> product launch. We wondered, which is more important to users – the value add\/features of the product or the discount?<\/p>\n

We found that the features won!<\/strong> With 99% statistical significance, the subject line focused on features (“double your speed”) had a 7% lift in opens and a huge 46% lift in click-through rate.<\/p>\n

We’ve rerun similar subject line tests, and again and again, the subject lines that promote features outperform the discount.<\/p>\n

What’s the takeaway here? Consumers first need to be interested in what you’re selling, before they’ll care about getting a discount.<\/p>\n

We’ve learned a lot from testing subject lines over the years. Here’s what we found works for us. Let us know in the comments if something different works for your brand!<\/p>\n