The short version: AI wrote my sales emails for 90 days and my open rates went up 11%, but consultation bookings dropped by nearly a third, because the emails got smoother and less specific at the same time, and smooth and unspecific is exactly what makes people stop trusting you. The fix isn’t to ditch AI, it’s to use it for the skeleton and keep every real detail, mistake and number yours.
Why I ran this test on my own list
I’ve got a list that’s built up over about fifteen years, roughly 28,000 people, mostly small business owners and marketers who signed up because of a talk I gave, a LinkedIn post that went a bit mad, or a webinar they half remember. It’s not a huge list by influencer standards. It’s a warm one, which matters far more.
Every week I send an email that’s part story, part opinion, part pitch. I’ve written every one of those myself for years, badly at 11pm most of the time, typos and all. In late 2025 I got curious, and a little tired, and decided to hand the writing over to AI for a proper stretch to see what would happen to the numbers rather than what the LinkedIn threads promised would happen.
I used ChatGPT and Jasper, feeding both my past emails, my tone notes, and a brief for each send. I kept the subject line testing and send times identical to my normal process so the only real variable was who wrote the body copy.
The setup, so you can copy it if you want to
- Weeks 1 to 4: AI drafted, I lightly edited (fixing facts, swapping generic examples for real ones). This was the “best case” version most people who sell AI email tools show you.
- Weeks 5 to 8: AI drafted, I sent almost as-is, changing only names and dates. This is what most small business owners who buy an AI writing tool do, because they’re busy.
- Weeks 9 to 12: Back to fully human, my normal messy self, as the control to compare against.
I tracked four things every send: open rate, click rate, replies (the real signal, because replies mean someone is reading, not skimming), and booked calls through my calendar link.
What happened to the numbers
Open rates crept up across the AI weeks, from an average of 31% to 34.4%, an 11% relative lift. That’s largely down to subject lines, which AI is fine at, punchy, curiosity-driven, on-brand enough. No surprise there, and if you’re only measuring opens, you’d call this test a win and move on, which is exactly what most case studies about AI email marketing do.
But replies dropped from an average of 9 per email to 3. Booked calls, which is the number that pays my mortgage, went from an average of 4.2 per email to 2.9 in weeks 5 to 8, the “send almost as-is” weeks. That’s a 31% drop in the metric that matters, while the metric everyone brags about went up.
Weeks 1 to 4, where I heavily edited AI drafts back into something that sounded like me, sat almost exactly between the two, which tells you the editing effort is doing most of the real work, not the AI.
The uncomfortable bit nobody selling you an AI writing tool wants you to think about
Here’s what I noticed reading the emails back cold, pretending I didn’t write them. The AI versions were better sentences. Cleaner grammar, tighter structure, no rambling. And they were worse emails, because the thing that makes a person reply to my newsletter isn’t the sentence quality, it’s the specific, slightly embarrassing detail that proves a human with a real week wrote it.
My normal emails mention things like the client who ghosted me for four months then came back with a bigger budget, or the exact figure I underpriced a project by in 2019 (£1,200, still annoys me). AI, even when fed those stories, kept smoothing them into “a client who took a while to come back” or “underpricing a project early on.” Technically accurate. Emotionally dead. The specifics are what make someone feel like they know you, and knowing you is the entire reason a stranger books a call with a consultant instead of googling “AI marketing help near me” and picking whoever’s ad shows up first.
That’s the part most posts about AI and email marketing skip over, because it’s more fun to talk about the open rate lift. The truth is AI is very good at making your writing sound more professional and slightly worse at making anyone trust you, and for a service business, trust is the entire product before someone ever hires you.
Where AI earned its place in my process
I didn’t drop it after the test. I use it differently now.
- First drafts of the boring structural bits. The opening hook framing, a clean three-point breakdown, a call to action that doesn’t sound desperate. AI is faster than me at this and I don’t lose anything by letting it do it.
- Cutting my rambling down. I still write the story and the opinion myself, then I ask AI to tighten it, not rewrite it. Big difference.
- Subject line variants. I write one, AI gives me five more to A/B test. This is where the 11% open rate lift came from and it’s the safest place to let AI touch your copy, because a subject line has no personality to lose.
- Repurposing, not creating. Turning one newsletter into three LinkedIn posts, that’s grunt work AI should be doing for every small business owner. This is the same principle behind good content strategy thinking from people like Joe Pulizzi, where one piece of real content gets stretched further, not where every piece is manufactured from nothing.
What I stopped letting it do entirely is write the story, the opinion, or the ask. Those three things are why anyone on my list is still there after fifteen years, and they’re the three things AI is worst at, precisely because it has no actual week to draw on.
What this means if you’re running a small business list
If your list is under about 5,000 people, this matters even more than it does for me, because your subscribers likely know your name, might have met you, and can smell a template from a mile off. A cold list of 100,000 bought through a lead magnet can probably absorb more AI-written filler because nobody there has a relationship with you to protect. A warm list built the way most small business owners build one, slowly, through content and word of mouth, cannot.
A test worth running on your own list, cheap and quick:
- Pick your next four weekly sends.
- Write two yourself as normal. Let AI draft two, lightly edited only for facts.
- Track replies and bookings, not opens, for all four.
- Compare after four weeks. Four is enough to see a pattern, you don’t need three months like I did.
Most people who try this find what I found. If yours doesn’t, brilliant, you’ve found a genuine efficiency gain and you should keep it. Don’t assume it either way. Measure it.
Where this connects to the bigger AI-in-marketing picture
This isn’t really an email problem. It’s the same pattern I see with businesses building chatbots, AI-written case studies, even AI-generated landing pages that A/B test beautifully and convert worse than the scrappy version a founder wrote at their kitchen table. Brands that have built trust at scale, think about how GoPro built its brand almost entirely on real, unpolished user footage, or how Figma grew through specific, credible community content rather than generic polish, tend to win by being more specific, not more efficient. AI pushes everything toward efficient. Your job is to fight that pull in exactly the spots where a human reading your words is deciding whether to trust you with money.
Tools like Mailchimp make it easier than ever to plug an AI writer straight into your send flow, and I’d still recommend studying how a company like that approaches email marketing at scale, because the mechanics of segmentation, timing and subject testing are worth learning from regardless of who writes the words. Just don’t let the ease of the integration decide the strategy for you.
If you’re not sure where the line sits for your own business, that’s usually the point where it’s worth getting a second pair of eyes rather than guessing for three months the way I did. I work through exactly this kind of thing with clients as an AI consultant for small business, figuring out which parts of your marketing AI should touch and which parts it should never be allowed near.
The one rule I’ve kept since the test
If a sentence in my email could have been written by anyone with access to my past newsletters, it gets cut or rewritten by hand. If a sentence could only have come from someone who lived my week, it stays exactly as written, typos and all. That’s the whole filter now, and it’s a lot simpler than any “AI vs human” debate the internet wants to keep having.
Free resource: The Sales Call Pre-Brief Template.
Free resource: The Sales Page Copy Prompt Pack.
Frequently asked questions
Does AI-written email hurt open rates?
No, and this is the trap. AI usually helps open rates because it’s good at punchy subject lines and clean structure. The damage shows up further down the funnel, in replies and bookings, which is why so many people miss it. They check opens, see a rise, and assume the AI copy is working.
Can people really tell if an email was written by AI?
Most people can’t point to a specific line and say “that’s AI,” but they feel that something’s off, and on a warm list that feeling shows up as fewer replies and less trust over time, even if nobody could explain why in a survey.
Is it worth using AI for marketing emails at all for a small business?
Yes, for structure, subject line testing, and cutting your own drafts down. No, for the story, the opinion, and the specific detail that makes a stranger trust you enough to book a call or make a purchase. Split the job, don’t hand over the whole email.
How do I test this on my own list without wasting months?
Run four consecutive sends, two written normally and two AI-drafted with only facts edited, and track replies and bookings rather than opens. Four weeks is enough to see whether your specific audience reacts the way mine did.
Related reading: How I Built an AI Lead Calculator in a Weekend (And What It Told Me About My Own Marketing) and I Built an AI-Powered Lead Calculator in a Weekend: Here’s What Six Weeks of Data Showed.
For the practical version of this, see write for us about email marketing.
