What if experienced email marketers could barely tell AI-written emails from human-written ones? That's exactly what AlpacaRelay found when they ran a blind test with 1,000 email professionals.
Their accuracy rate: 52%. Barely above a coin flip.
The participants weren't beginners. Median experience was five years, and many managed campaigns sending millions of emails a month. And still, nearly half the time, they couldn't tell.
52% accuracy rate among 1,000 experienced email marketers
Note: Methodology note: AlpacaRelay recruited 1,000 email marketing professionals and asked each to identify which of two paired emails was AI-generated. Emails were scored across eight quality dimensions. See the full research at alpacarelay.com/blog.
The finding is counterintuitive because most of us assume we'd catch it. What the data actually showed is more nuanced. Let me break it down.
The three things AI consistently gets right
When the emails were scored across eight quality dimensions, AI-generated emails scored 7.1 out of 10, compared to 6.4 for human-written, roughly 11% higher overall. But the advantage wasn't evenly distributed.
Mobile optimization. AI scored 8.4 out of 10. Human-written emails averaged 5.9. The reason is straightforward: AI consistently applies mobile formatting rules (single-column layout, large tap targets, compressed images), while human writers skip them or forget. Given that most subscribers read on phones, that gap has a real effect on how your emails land.
CTA clarity. AI-generated emails scored 8.7 out of 10 on CTA clarity, compared to 3.8 for human-written emails and 5.1 for the industry average. Human emails averaged 3.2 calls-to-action per email, compared to 1.0 for AI. Each extra ask dilutes the one action you actually care about.
Content structure. AI reliably used headers, bullet points, and white space in ways that help scanners (which is most email readers). Human-written emails more often buried key points in paragraphs.
One important caveat: AI performing well on structure and formatting best practices doesn't guarantee inbox placement. Deliverability also depends on sender reputation, technical configuration (SPF, DKIM, DMARC), and list health, factors that sit outside the email content itself.
Want AI doing the heavy lifting in your marketing?
I build the systems that handle the boring 80 percent, so you get your week back. Done properly, with the human kept in.
The one thing AI still can't match
Brand voice was where humans won clearly: a 7.2 average versus 5.8 for AI.
The emails that fooled participants most had natural conversational flow and details that only an insider would use. A fitness studio email that opened with "Most people think the hardest part of fitness is the workout. It's actually showing up consistently on Tuesdays" saw 68% of marketers label it human-written. It was AI, generated in under 15 seconds.
The AI emails that got caught shared the same tell: technically correct, but slightly generic. Missing the small details that signal a real relationship with the reader. That's the gap AI hasn't closed.
The case for a hybrid approach
The most interesting finding was what happened with hybrid emails. When participants reviewed emails that were drafted by AI and then lightly edited by a human for brand voice, overall quality scores jumped to 8.6, compared to 7.1 for AI-only and 6.4 for human-only.
That improvement came from targeted edits rather than rewrites. Swapping generic phrases for brand-specific ones. Adding a detail only someone inside the company would know. Adjusting the sign-off to match the sender's actual voice.
The AI handled structure, formatting best practices, and CTA discipline. The human handled the relationship layer. Each does what it's actually good at.
What this means in practice
If 1,000 experienced marketers could identify AI-generated emails correctly only 52% of the time, the useful question isn't "does this sound human?" It's "is it technically sound AND does it have our voice?"
Those are two different problems with two different solutions. Tools like an email quality scorer can flag exactly where an AI-generated draft loses authenticity before you send, which makes the human editing pass faster and more targeted.
AI can get the technical side right. Humans still own the relationship side. Teams that learn to combine both approaches can benefit from AI's efficiency while preserving the human side of their communication.
