In Mailchimp, click Create, then Email, then A/B test, name it and choose your recipients. On the Variables step pick one thing to test (subject line, From name, content or send time) with up to 3 variations, set the share of recipients for the test and how the winner is chosen, then finish the content and send.
Decide what you're testing before you open Mailchimp
A good test answers one question. Mailchimp enforces that: each test covers one variable, with up to 3 variations. The four options are subject line, From name, content and send time. Pick the one you're least sure about. If you've never tested subject lines, start there, because it's quick to set up and easy to read.
A/B testing for email is included with the Essentials plan or higher. Multivariate tests, which mix several variables, need the Standard plan or higher.
Setting up the test
- Click the Create icon, then Email, then A/B test. Enter an email name and click Begin.
- On the Recipients step, choose the audience or segment, then click Next.
- On the Variables step, click the variable to test and use the plus or minus icons to set up to 3 variations.
- Use the slider to set how many recipients get the test, and choose how the winner is picked and how long the test runs.
- Complete the Setup and Content steps. If you're testing content, add content for each variation.
- On the Confirm step, review the pre-delivery checklist, then click Send or Schedule.
If you're testing subject lines, each can be up to 150 characters, though Mailchimp recommends 60 or fewer. Preview text can also be up to 150 characters.
Test size and how the winner is chosen
| Setting | Rule |
|---|---|
| Minimum test share | 10% of recipients; the slider won't go lower |
| Suggested sample | At least 5,000 subscribed email contacts per combination |
| Winner chosen by | Opens, clicks, total revenue, or manually |
| Send time tests | Always go to 100% of the audience |
| Wait before sending the winner | At least 4 hours, per Mailchimp |
| Tracking | Opens and clicks tracking can't be switched off |
Tracking stays on because it's how Mailchimp decides the winner. Winner notification emails go to account users with a user level of Manager or higher, so make sure the right person will see the result.
Content tests should be judged on clicks
Mailchimp Help's Create an A/B test article flags the trap. When you test content, choose the winner by clicks rather than opens, because an open happens before the reader has seen the content. Judging a content test on opens tells you nothing about the content itself. Opens are the right measure for subject line and From name tests, where the thing you changed is what people see before they open.
The same article notes that tests can't go below 10% of your recipients and that each one covers a single variable. If your list is small, Mailchimp's suggestion of 5,000 contacts per combination won't be possible, so treat results as a hint rather than proof, and repeat the test before you change your approach.
Three tests worth running first
- Subject line: a question against a plain statement, judged on opens.
- From name: your own name against your business name, judged on opens.
- Content: one main call to action against two, judged on clicks.
Run each one on two or three campaigns before you decide. One test on one send can easily be a fluke. For a wider look at the method, my roundup of the best A/B testing tools covers testing beyond email.
How HubSpot runs email tests
HubSpot's A/B test measures engagement for different versions of an email with a sample of recipients, then sends the best-performing version to the rest. If you're weighing email platforms, my comparison of MailerLite and Mailchimp is a good place to start.
Want AI doing the heavy lifting in your marketing?
I build the systems that handle the boring 80 percent, so you get your week back. Done properly, with the human kept in.
My advice before your first Mailchimp split
Test one thing, give it at least 4 hours, and match the measure to the variable: opens for subject lines and From names, clicks for content. Keep a simple note of each test and its result, so a year from now you've got your own evidence instead of guesses. If you'd like a sense of how Mailchimp compares with an e-commerce-focused tool, my Klaviyo versus Mailchimp guide covers that.
Want your tool on this list? Featured spots and reviews here are paid partnerships. Here's how to partner with me.
Frequently asked questions
What can I test in a Mailchimp A/B test?
Subject line, From name, content or send time.
How many variations can I run?
Up to 3 for one variable.
How big should the test group be?
At least 10% is required, and Mailchimp suggests at least 5,000 contacts per combination.
Which measure picks the winner?
Opens, clicks, total revenue, or a manual choice. Clicks are advised for content tests.
How long should I wait before sending the winner?
At least 4 hours, per Mailchimp.
