My Cart
Filters
ab test

24 Ideas How to Conduct an Efficient Email A/B Testing

A well-thought-out email strategy helps businesses to notably increase conversion rates, especially during the holidays when the demand for particular products is considerably higher than usually.

24 Ideas How to Conduct Efficient Email A/B Testing

But how can you know what emails can produce the desired effect on customers and encourage them to make a purchase? Only by experimenting with different variants of letters! In this article, we will give you some tips on how to organize an effective email campaign through A/B testing (or split testing) of emails.

Email A/B Testing


If you are running a Magento 2 web store and need to test different versions of emails, a smart choice here is to use the recently updated Follow Up Email Extension 2.0. The solution provides the A/B Testing Mode, which can be enabled on the Edit Email page in the Magento 2 Admin Panel.



First, prepare the email (version A) and its alternative variant (version B) required for your A/B test. These emails should differ only by one parameter, in other words, you can create two variants (or more if you need to conduct several A/B tests) of the same email that will differ in, say, headings, images, or texts. Let’s check what Follow Up Email 2.0 is ready to offer.

The Letter Settings Panel

The Letter Settings Panel


Using the A/B Testing Mode of Follow Up Email 2.0, you can experiment with the next parameters: Sender Name, Sender Email, Subject, Content, Header and Footer. In addition, you can take advantage of the provided variables and preview mode. With the extension, you can create messages with various layouts, as the solution supports HTML and CSS.

Having created the letters, next, you need to get access to the statistics displaying the number of sent emails, their openings, clicks, open rates, and click through rates.

A/B Testing Statistics

A/B Testing Statistics


And, Follow Up Email 2.0 provides the statistics on all the above key points.

Duration of Tests


To find the answer to this question, view the statistics of previous email campaigns. If the majority of clicks occurred during the first 24 hours, it is possible to limit the duration of testing to this time period.

Note that the time for your email split tests should be extended in case if you have a large customer base and did not manage to collect all the desired results in 24 hours.

Testing Parameters


Emails contain many components that need to be tested. Let’s consider each of them individually.

1. Sender Names


Having received your letter, first, the recipient pays attention to the name of the person who sent it. So, test this parameter by changing the male and female names for different versions or by choosing the company’s name for this field.

2. Email Subjects


When composing the subject of the letter, pay attention to its wording and length. The wording of a subject reflects the style of the following text. By reading the subject, a subscriber makes the decision whether to read the letter or move it to trash.

To find out the subjects that work better, try to use different wordings making them either personalized or general, writing them in the form of a statement, a question, a call to action, or in some other forms. Also, pay attention to the number of symbols used to write the subject, as too long email subjects might seem boring to customers.

3. Preheader Texts


This is the text that appears after the subject of a letter. In the example below, the preheader is highlighted with the red block.

Email Preheader

Email Preheader


Pay a close attention to this parameter, as the text of the preheader specifies the subject of the email and urges the subscriber to open or trash it. The experiments with preheader styles are much similar to the ones with subjects.

4. Fonts


A/ B tests can help you define the fonts to be used for email texts. In fact, there are no universal rules for using fonts in emails, so here, you are limited only by your imagination. However, it’s not recommended to use more than three fonts in one email, as the letter with too many fonts will look messy. For example, you can use different fonts for the header, main text, and product description of your message.

With Follow Up Email 2.0, you can experiment with fonts as you like. For this purpose, just edit the content on the Edit Email page.

Email Content

Email Content


And here how it looks in the Preview Mode.

Email Preview Made by Follow Up Email 2.0

Email Preview Made by Follow Up Email 2.0


Let’s consider the fonts that are commonly used for product descriptions:

  • Arial, Helvetica – very simple and well-perceived visually. These fonts are displayed correctly by any email service. Apple Mail and Gmail use these fonts by default.

  • Verdana – favorably influence the ease of reading due to its inter-letter space.


Below, you can see the example of an email made by Follow Up Email 2.0 with the use of all the above-described fonts.

The Email Using the Arial, Helvetica, and Verdana Fonts

The Email Using the Arial, Helvetica, and Verdana Fonts


Here, we implemented various styles for each font. For the header, we chose the bold Arial font of the 3px size. For the main text, Helvetica was used in its default size, and for the last two sentences of the letter, Verdana in italics was chosen, still, we kept the original size of the font. Compared to the example of the same letter we brought earlier, this one might seem more appealing to customers.

5. Presentation of Information


Different text layouts work differently as well. You can send emails with plain texts, lists of facts and features, or texts written in two or even three columns with some pictures attached and background colors. Below, you can see three examples of the same email demonstrating different layouts.

Plain Text

Plain Text


List of facts


List of facts


Information Presented in Columns (Using Pictures and Background Colors)


Information Presented in Columns (Using Pictures and Background Colors)


In the latter two versions, the texts look well-structured compared to the first variant. The emails organized as lists or divided by columns are pleasant to read and easy to remember.

6. Number of References


What is the best option for your business: one main reference or a few links attached to the email? Try to test it and make your choice.

7. Link Types


What works better: text or graphic links? One more question to consider during your email A/B tests.

8. Tone of the Offer


To test this parameter, use official tone for the first variant of the email and more informal one for the second. Test both variants.

9. Teasers


Cryptic messages are often used to create a sense of intrigue and make customers interested. So, try to test various teaser options, add thrilling images if you like to see how this tactic might influence the purchasing decision.

10. Limited Offers


"Instant sale!", "Only for … hours ...", "Have the time to buy, there are only ... days left". It is believed that the sense of urgency better impacts sales. So, test various types of offers by creating, for example, a usual newsletter and an urgent newsletter to find out which one will be more effective in terms of sales.

11. Announcements


Which announcement looks and sounds better to you: "10% discount" or "Save $30"? Consider the variants of announcements to be tested.

12. Color Schemes


Each color produces a particular effect on customers. The yellow color, for example, attracts attention, the red color increases the heartbeat by creating the feeling of urgency. The blue color causes a sense of peace and comfort. The black color is the symbol of power and authority. In advertising, this color often symbolizes luxury. So, learn more about the psychology of colors to improve your mailings.

13. Navigation or Attention Marks


Additional items such as arrows, icons, etc. serve to highlight most important fragments of emails and attract attention of readers. Try to add them to your email to see what effect this move will produce compared to the letter with no additional items.

14. Price Visualization


Consider how to better visualize prices to attract more attention of the audience and increase the CTR. Try to highlight the prices of your products by using color blocks, bold fonts, and other visual effects to find out which variant seems to be more appealing to customers.

15. Location and Number of Calls to Action


Often, the CTA (Call to Action) buttons are placed at the bottom of the contents. Try to test the variant where these buttons appear at the top of the email. Carefully consider the number of CTAs to be included, as multiple CTAs can distract readers’ attention. Compose, for example, the version with two CTAs and the one with three CTAs and compare the results.

16. Images of Products


When testing this parameter, make a series of photos of your product that will differ in style or camera angle to see the photos customers like more.

17. Number of Product Images


Sometimes, it is very difficult to decide how many products to advertise in a single email. So, test the variant with one product image and the option containing the whole line of product images and then compare the results.

18. The Size of Product Images


The larger is the size of the product image, the more details subscribers will see on it and will likely better know it. So, try different picture sizes in your emails.

19. Video Content


If you are not sure whether to use video content in your mailings or not, test it and make sure. Using videos you can present your products, add video guides, or simply congratulate clients on their holidays.

Your video can be easily embedded in the HTML code of the email in the Content field on the Edit Email page of Follow Up Email 2.0.

20. GIF Animation


A GIF picture with dynamic animation attached to emails always catches the eye since many readers are already tired of static images. GIF animation is often used for promotions, as flashing frames evoke reader's desire to find out details of the proposal. You can test different versions of your GIF images or even combine them with your video clips.

21. Frequency of Emails


This parameter should be discussed in more detail.

When you see positive effect from your mailings, including subscribers interest and engagement, sales growth, etc., there will be a temptation to send emails more often. Most companies start with weekly mailing, but then, they increase the number of emails to 3-4 per week or even send them every day, including weekends.

This increase in sending usually produces only short-term positive effect, but eventually, you risk to scare off clients with your assertiveness.

The optimal frequency can be determined in practice, again, by conducting A/B email testing. First, select two random segments of the audience in your database. Then, choose one of the segments and send it a series of emails with a certain frequency.

For the second group, you should send the same emails with a different frequency. Then, compare the indicators of each email for each segment. The key indicators for this test are the number of openings, CTRs, target actions, and the number of “unsubscribes”.

Note that if any of the tracked indicators dropped by 10-20%, there is no need to worry about such an insignificant decrease. But, if you see a notable growth of the parameters, then you can keep sending emails with the optimal frequency without ceasing to closely monitor all the above-mentioned KPIs.

Also, try to adjust the frequency of emails depending on the stage of the life cycle of a subscriber. New subscribers are ready to receive letters more often, for example, several times a week, as they are still finding out about your brand and want to learn more. Those who are in your client base for a long time (at least, six months) would probably prefer to receive emails from you 1-4 times a month.

22. Day and Time of Sending


Any marketer will say that the best days for sending emails are Tuesday, Wednesday, and Thursday. The best days for useful and entertaining content are Friday and even weekends. Such mailings should be sent in the morning or in the afternoon when internet users are the most active. So, try to test various days and time periods for sending your emails.

23. Bonus Offers


In this case, you need to determine which offers are more preferred by your subscribers: gifts, bonuses, discounts, etc., then you can send emails with different variants of incentives and then choose the one that produced a better effect.

24. Trial Period


For many people, it is very important to try a product or a service before making a purchase to find out whether it is suitable for them or not. Therefore, you can create two A/B testing options: the first one, for example, will offer a free trial subscription for two weeks and the second - one month. Thus, you can determine the time your customers need to evaluate your products without rush.

Conclusion


A/B testing is a reliable tool for optimizing email campaigns.  Since there is no one sure way to create efficient emails, you need to compare different variations to find out the one that works better for your audience and therefore, for your business. And the Magento Follow Up Email 2.0 Extension by Aheadworks offers great opportunities for A/B email testing, thanks to which you can apply practically all the above-described ideas.