Straight answer: you can see a genuine amount of your competitors’ keyword data for free using Google’s own search operators, their page titles and sitemaps, the Wayback Machine, and a handful of free-tier tools, but none of it gives you the full picture a paid tool like Semrush or Ahrefs would. What you get free is enough to find real gaps and steal real ideas. What you don’t get is exact search volume and a complete list, and anyone who tells you otherwise is selling you something.
Why I still do this the manual way
I’ve had Semrush and Ahrefs subscriptions on and off for over a decade. But when a client asks me to look at a competitor before we’ve even signed a contract, or when I’m poking around a niche I’m not billing anyone for yet, I don’t fire up a paid tool. I do it the free way first, because it forces me to read the competitor’s site instead of just scanning a spreadsheet of numbers someone else’s crawler guessed at.
Last month I did this for a small accountancy firm in Surrey who wanted to know why a competitor two towns over kept beating them for “payroll services for small business.” Forty minutes of free digging, no login, no trial started, and I’d found 23 keyword variations they were ranking for that my client had never touched, including “payroll bureau vs in house payroll” and “auto enrolment pension staging date checker.” Neither of those came from a tool. They came from reading the competitor’s blog category page and their site’s internal search suggestions.
Start with the site: operator
This is the one people skip because it sounds too basic to work. It works.
Type site:competitorname.com into Google and you’ll get every page Google has indexed for them. Scroll through the titles. Titles are where people put their target keyword 90% of the time because that’s still basic on-page SEO. You’ll see patterns fast: “how to,” “cost of,” “vs,” “for beginners,” “checklist.” Those patterns tell you their content strategy without any tool at all.
Then narrow it: site:competitorname.com "payroll" shows you every page where they’ve used that word, which tells you how deep they’ve gone on a topic. If they’ve got 14 pages mentioning payroll and you’ve got two, that’s your gap, no subscription needed. I cover the wider version of this approach, building a full keyword list from scratch with nothing but a browser, in how to do keyword research for free without expensive tools.
Read their sitemap like it’s a menu
Most sites have a sitemap at competitorname.com/sitemap.xml. It’s built for search engines, not humans, but you can read it too. It lists every URL on the site in one place. Open it, and instead of clicking through pages one by one, you can see their entire content structure at a glance: how many blog posts, how many service pages, how many location pages if they’re a local business.
I’ve had clients in local services (plumbers, dentists, accountants) who didn’t realise their competitor had built 40 separate location pages, one for every town within a 20 mile radius, each targeting “[service] in [town].” That’s not a keyword tool insight. That’s just opening a text file.
Steal from People Also Ask and autocomplete
Search your competitor’s brand name plus a topic, e.g. “Xero payroll setup,” and look at the People Also Ask box. Click a few of those, more appear underneath. This is Google showing you real questions real people type, pulled from actual search data, for free, every single time.
Do the same with autocomplete: start typing “[competitor’s main service] ” into Google and see what it suggests. Then try “why is [competitor’s service]” and “[competitor’s service] vs” and “[competitor’s service] alternative to.” These are keywords with genuine intent behind them, and they cost nothing.
AlsoAsked.com has a free tier that maps these questions into a visual tree, which saves you the clicking. AnswerThePublic also has a limited free search per day. Neither replaces a paid rank tracker, but both are useful for finding the shape of a topic before you write anything.
Check their page titles and meta descriptions directly
View the source of a competitor’s key pages (right click, view page source, or just look at the browser tab title) and read the title tag and meta description word for word. These are written by a human trying to rank for something specific. If their homepage title is “Payroll Services London | Auto Enrolment & PAYE Setup,” you now know their three target phrases without a single tool. This sounds obvious. Almost nobody does it because scrolling a spreadsheet feels more like “research.”
Use the Wayback Machine to see how they got there
Go to web.archive.org and paste in a competitor’s URL. You can see snapshots of their site going back years. This shows you which keywords they added later, meaning which topics they tested and kept, versus what’s been there since day one and never changed. If a page’s title changed from “Payroll Software” in 2022 to “Payroll Services for Small Business Owners” in 2024, that’s a real signal they moved from a broad, competitive term to a more specific one that converts better. That’s competitive intelligence you cannot buy, because no paid tool tracks historical title changes as standard.
Use your own Search Console for near-misses
This one surprises people. Google Search Console is free for your own site, and it shows you every query you’re already appearing for, even ones you’re not actively targeting, ranked at position 40, 50, wherever. Filter by impressions, sort ascending by position, and look at everything sitting between position 11 and 30. Those are keywords Google already thinks you’re relevant for. Chances are your competitor ranks on page one for several of them. You’ve just found your fastest wins without looking at their site at all.
The free trial route (and the bit nobody says out loud)
Semrush and Ahrefs both offer short free trials, and Ubersuggest has a free tier with limited daily searches. You can absolutely use these to pull a competitor’s top keywords in one sitting. Plenty of guides tell you to “just use the free trial” as if it’s a clean loophole.
Here’s the bit those guides don’t say plainly: these trials require a card upfront on most plans, and the entire business model depends on you forgetting to cancel. That’s not a conspiracy theory, that’s how subscription businesses forecast revenue; churn from forgotten trials is baked into their numbers. If you’re going to use a trial, put a calendar reminder in for the day before it renews, screenshot everything you need in one session, and cancel immediately after. Treat it as a one hour job, not an ongoing free tool.
And here’s the second uncomfortable truth: even the paid tools are showing you modeled data, not a live feed of Google’s actual rankings. Search volume, keyword difficulty, “estimated traffic,” all of it is built from sampled click-through data and algorithms guessing at the rest. A keyword tool telling you a competitor ranks number 3 for something with 2,400 searches a month is an estimate dressed up as a fact. Free methods just make that estimation more obvious, because you’re the one doing the guessing instead of paying someone else to hide it in a dashboard.
Check what AI answer engines already say about them
This is newer ground and most keyword guides haven’t caught up to it yet. Ask ChatGPT or Perplexity something like “what companies offer payroll services for small businesses in Surrey” and read what comes back. If your competitor gets named and you don’t, that tells you they’re already associated with certain topics in ways that traditional keyword tools won’t show you, because this is about being mentioned, not ranked. I wrote a full breakdown of what influences this in how to get your small business mentioned by ChatGPT, and it’s worth doing alongside your keyword digging, not instead of it.
Turn what you find into something worth publishing
Finding the keyword gap is the easy 20%. Most businesses stop there, save a spreadsheet, and never write the content. If you’ve found that your competitor has 14 pages on a topic and you have two, don’t panic and try to publish 12 pages in a week with AI and no editing. I’ve seen that go wrong more than once, thin pages that Google quietly stops indexing within a month. There’s a right way to bring AI into this process without losing quality, covered in how to use AI for SEO the right way without wrecking your rankings, and if you’re building this out across a whole content calendar, building an AI content team to outperform competitors walks through how to structure that without it turning into a mess of unedited drafts.
If writing isn’t your thing or your time, and you’d rather brief someone than do it yourself, how to hire a freelance writer who delivers covers what to ask for so you don’t waste your first three briefs finding out the hard way.
Don’t ignore social signals while you’re at it
Keywords aren’t the only thing worth checking on a competitor. While you’re on their site, look at where their traffic seems to be coming from, LinkedIn posts driving comments, a YouTube channel with actual views, a newsletter mentioned in the footer. Rankings and social visibility feed each other more than most SEO guides admit, and I’ve laid out the actual mechanics of that in how Google’s SEO rankings are affected by social media.
What to do this week
- Pick your two biggest competitors, run site: searches on both, and list every content pattern you see in their titles
- Pull up their sitemap.xml and count how many pages they have in categories you don’t cover at all
- Search their brand plus your main service term and mine the People Also Ask box for at least 10 questions
- Check your own Search Console for anything ranking positions 11 to 30 and treat those as quick wins first
- If you use a free trial tool, do it in one sitting, screenshot what you need, cancel before renewal
None of this takes more than a few hours, and none of it costs anything beyond your time. The honest limitation is that you won’t get exact search volumes or a tidy exportable list the way a paid tool gives you. What you will get is a working list of real gaps, written in the competitor’s own words, which is frankly more useful for writing something than a spreadsheet of numbers you’d have to interpret anyway.
Frequently asked questions
Can I really find competitor keywords without any paid tool at all?
Yes, using Google’s site: operator, their sitemap.xml, People Also Ask, autocomplete, and the Wayback Machine you can build a real list of the topics and phrases a competitor targets. You won’t get exact search volume numbers, but you’ll get the actual keywords and phrasing, which is often more useful for writing content than a volume estimate.
Are free tools like Ubersuggest accurate for checking competitor keywords?
They’re directionally useful but the numbers are modeled estimates, not live rankings pulled straight from Google. Use them to spot patterns and rough scale, not as an exact source of truth, and treat any tool’s “estimated traffic” figure as a rough guide rather than a fact.
How long does it take to research a competitor’s keywords manually?
Expect 30 to 60 minutes per competitor for a solid first pass: 10 minutes on site: searches, 10 minutes on their sitemap, 15 minutes on People Also Ask and autocomplete, and the rest reading their actual page titles and top content.
Is it worth using a free trial of Semrush or Ahrefs just for this?
Yes, if you use it deliberately: gather everything you need in one focused session and cancel before it renews, since these trials are built to convert forgotten cancellations into paying customers. Treat it as a one-off data pull, not an ongoing free resource.
Related reading: How an SEO Rank Tracker Helps You Monitor Keyword Positions (And What It Won’t Tell You) and What Is Content Led SEO and How to Build a Strategy Around It.
I take guest contributions on this topic, so you can write for us about SEO.
