The short version: Look at your last 200 support tickets before you look at any demo, check exactly where the AI hands off to a human and how messy that handoff is, and ask what happens to your customer data if you cancel. Everything else, the chatbot personality, the “94% resolution rate” on the sales deck, the pretty dashboard, matters far less than those three things.
Start with your tickets, not the vendor’s demo
I made this mistake myself in 2023 when I was helping a client, a small e-commerce brand selling skincare tools, pick a customer service platform. We sat through four demos. Every single one was slick. Every single one showed the bot handling a perfect “where’s my order” query and resolving it in eleven seconds. What none of them showed was what happened when a customer wrote in furious, using bad grammar, asking about a refund on a product that had been discontinued three months earlier. That’s the query that landed in their inbox forty times a week.
So before you book a single demo, pull your last 200 support tickets and sort them by type. Most small businesses find that 60 to 70% fall into five or six categories: order status, returns, sizing or spec questions, billing, and “I want to speak to a person because I’m annoyed.” Take that list into every sales call and ask the vendor to show you, live, how their AI handles the messiest three examples you’ve got. Not their example. Yours.
Where the AI needs to hand off to a human
This is the bit most buyers skip and it’s the bit that decides whether customers end up loving your support or leaving one-star reviews about talking to a robot. Every AI platform will resolve the easy stuff. The question is what it does with the hard 20%.
- Does the handoff happen with full context, or does the customer have to repeat everything they already typed to the bot?
- Is there a visible, obvious way for a customer to say “I want a human” at any point, not buried three menus deep?
- Can you set rules by ticket type, so a billing dispute over £200 always goes to a person, but a password reset never does?
I’ve written before about how AI chatbots handle customer service enquiries and where they fall down, and the pattern is consistent: the failures aren’t dramatic, they’re small and repeated. A bot that keeps offering the wrong article. A bot that closes a ticket it hasn’t solved. Ask any vendor to show you their false-close rate, not just their resolution rate. Most won’t have that number ready, and that tells you something too.
The uncomfortable bit vendors don’t lead with
Here’s the part that gets glossed over in most buying guides: an AI customer service platform will not cut your support headcount by half in year one, and if a salesperson promises that, be careful. What it does well, if you set it up right, is take the repetitive 60% off your team’s plate so the humans left are dealing with the interesting, high-value, relationship-building conversations, the ones that turn a complaint into a loyal customer. That’s a real and valuable outcome. It’s just not the outcome most people are being sold.
I’ve seen small teams try to use AI as a straight replacement for a support person and it goes badly within about six weeks, because the AI handles volume beautifully and handles nuance badly, and nuance is exactly what upset customers need. The businesses that get real value treat the AI as a triage layer, not a replacement layer. That distinction should shape what you buy.
Integration matters more than intelligence
A platform with a brilliant language model but no proper connection to your order system, your CRM, or your existing help desk is close to useless. If the AI can’t see that a customer’s order shipped yesterday, it will confidently give a wrong answer, which is worse than no answer at all.
Check specifically:
- Does it connect natively to your ecommerce platform (Shopify, WooCommerce, BigCommerce) or does it need a custom build?
- Does it pull live order and account data, or does someone need to manually feed it information?
- Can it sit across email, live chat, and social DMs from one dashboard, or will your team be juggling three logins?
If live chat is a big part of what you’re weighing up, it’s worth reading through the live chat services worth considering for a small business in 2026 before you commit, because some of the AI customer service platforms are really live chat tools with AI bolted on, and the base chat experience underneath matters just as much as the AI layer on top.
Where your data lives
Ask this directly in the sales call: if we cancel next year, do we get our full conversation history and can we export our custom training data, or does it stay locked in your system? A surprising number of platforms make this awkward on purpose, because switching costs are good for retention. I’d treat any vendor who hesitates on this question as a red flag, whatever the rest of the demo looked like.
Also ask where the model is trained, whether your customer conversations are used to improve the vendor’s general model (some platforms do this by default and you have to opt out), and whether that’s compliant with UK GDPR if you’re handling EU or UK customer data. This one gets skipped constantly because it’s boring, and it’s exactly the sort of thing that becomes a genuine problem eighteen months in.
Pricing models you need to understand before you sign
AI customer service pricing has three common structures and they behave very differently as you grow:
- Per seat, usually £25 to £90 per agent per month, straightforward but doesn’t account for AI-handled volume at all.
- Per resolution or per conversation, often £0.30 to £1.50 per AI-resolved ticket, which sounds cheap until volume spikes in December and your bill triples.
- Usage or API based, priced on messages or tokens, which is the model behind most of the newer platforms built on GPT or Claude APIs directly.
If you’re weighing up a platform built on API pricing, it’s worth understanding how that market works before you negotiate. I broke this down in how do you price an API based messaging service for business, and it applies just as much to customer service tools as it does to marketing messaging.
Whatever the model, ask for a worst-month estimate, not just an average one. Black Friday, a product recall, a viral complaint on X, these spike your ticket volume 3x to 10x overnight, and per-resolution pricing can turn a £200 month into a £1,800 month without warning.
Reporting that tells you something
Dashboards showing “tickets resolved” and “customer satisfaction score” are table stakes and mostly meaningless on their own. What you want to see:
- Resolution rate broken down by ticket category, not just an overall average
- How many “resolved” tickets get reopened within 48 hours (the real measure of whether the AI solved anything)
- Average handling time for the human agents once the AI has triaged, compared to before
Run a proper 30-day trial before signing anything longer than a quarter. Track those three numbers weekly. If reopened-ticket rate is climbing, the AI is closing tickets to look good on the dashboard, not solving customer problems, and that’s a pattern I’ve seen more than once with platforms that lean too heavily on “resolution rate” as their headline metric.
A simple seven-step way to evaluate any platform
- Pull your last 200 tickets and categorise them
- Bring your three messiest real examples into every demo
- Ask for the false-close and reopened-ticket rate, not just resolution rate
- Test the human handoff yourself, pretending to be an angry customer
- Confirm data export and model training policy in writing
- Model your worst-month pricing, not your average-month pricing
- Run a 30-day live trial with real tickets before any annual contract
Most vendors will happily walk you through steps one to four in a sales call. Very few will offer step seven without being asked directly. Ask anyway. If they push back on a trial, that’s a data point in itself.
Where an AI consultant earns their fee here
If you’re a small business owner without a dedicated ops person, this whole evaluation process eats a week you probably don’t have. This is one of the areas where bringing in outside help pays for itself, because a consultant who’s watched a dozen of these implementations go right and wrong will spot the vendor’s soft answers in a demo that you might miss on your first pass. If you want a clearer sense of what that kind of help costs, I’ve laid it out in detail on how much an AI consultant costs, and it’s usually far less than the cost of a wrong platform choice locking you into a bad contract for a year.
The time saved isn’t just in the evaluation either. Once it’s running well, an AI customer service platform is one of several tools that gives small business owners hours back each week, which I’ve covered more broadly in how AI powered productivity tools help small business owners save time. Customer service is usually the single biggest time sink on that list, so getting the platform choice right matters more here than almost anywhere else in the business.
Frequently asked questions
What’s the biggest mistake people make when choosing an AI customer service platform?
Judging the platform on the vendor’s demo instead of on how it handles your own messiest, real customer tickets, which usually produces a completely different result once you run them through.
How much should a small business expect to pay for an AI customer service platform?
Most small businesses land somewhere between £50 and £400 a month depending on volume and pricing model, but usage-based platforms can spike far higher in peak months, so always model a worst-month cost before signing an annual contract.
Will AI customer service replace my support team?
Not in year one and probably not ever entirely. It works best as a triage layer that handles repetitive queries so your human team can focus on the complaints and relationship-building conversations that keep customers loyal.
What should I ask about before signing any contract?
Ask exactly what happens to your customer data and conversation history if you cancel, whether your conversations train the vendor’s general model by default, and whether you can run a real 30-day trial using your own tickets before committing to a longer term.
