The short version: llms.txt is a small markdown file you put in your website’s root folder that lists your most important pages for AI systems to read, but no major AI crawler has confirmed it uses the file, so treat it as a cheap, harmless extra rather than something that will fix your visibility in ChatGPT or Perplexity.
If you decide you want one, the free llms.txt generator builds the file from your site name, description and key pages in about two minutes.
What llms.txt is, in plain terms
llms.txt is a text file, written in markdown, that sits at yourdomain.com/llms.txt. It’s meant to work a bit like robots.txt or sitemap.xml, except instead of telling search engines what to crawl, it’s supposed to tell large language models what your site is about and which pages matter most.
The idea came from Jeremy Howard, co-founder of the AI company fast.ai, who published the concept in September 2024. His pitch was simple: language models have limited context windows and struggle to read a full website the way a human does, so a short, curated file pointing to your best pages, written in clean markdown rather than messy HTML, should help an AI understand your business faster and more accurately.
A typical llms.txt file looks something like this:
- An H1 with your business or site name
- A one or two line summary of what you do
- A section listing your key pages with short descriptions, formatted as markdown links
- Sometimes an optional section for less important pages
That’s it. No script, no schema, no complicated markup. It’s meant to be readable by a human in about ten seconds and by a model in even less time.
Where the confusion comes from
Because llms.txt sounds official and sits next to robots.txt, a lot of business owners assume it’s a recognised web standard that AI companies check the way Google checks a sitemap. It isn’t, not yet. There is no governing body enforcing it, no W3C standard, and no confirmation from OpenAI, Anthropic, Google, or Perplexity that their crawlers read the file at all.
I’ve had three separate clients ask me in the past two months whether their web developer had “done the llms.txt thing” and whether that alone would get them recommended by ChatGPT. It won’t, and anyone telling you it will is selling you something. I wrote a longer breakdown of the mechanics in what llms.txt is and whether your business needs one, but the short version repeated everywhere online: it’s a proposal, not a protocol.
The uncomfortable bit nobody likes to say out loud
Here’s the part most posts on this topic skip past. I checked server logs for two client sites that added llms.txt files back in March 2026. Over 90 days, we saw zero confirmed requests for that file from GPTBot, ClaudeBot, or PerplexityBot, the user agents those companies use to crawl the open web. Plenty of requests for robots.txt, plenty for the sitemap, none for llms.txt.
That doesn’t mean it’s pointless. It means the file is currently aspirational: some tools and AI-powered browser plugins do read it, and it costs almost nothing to add, but if you’re hoping it will single-handedly change whether your business shows up when someone asks ChatGPT for a recommendation, you’ll be disappointed. What moves that needle is the same unglamorous stuff that always has: clear, structured content, consistent mentions of your business across the web, and pages that directly answer the question a buyer would type. If you want to know whether any of this is working, the more useful exercise is running the test in how to check whether ChatGPT is recommending your business before you spend an afternoon on a file that might currently be read by nobody.
Does your website need one anyway?
I still add llms.txt to client sites, and here’s my honest reasoning for doing it despite the log data above.
- It costs about fifteen minutes to write and upload, so the downside risk is close to zero.
- Adoption could shift quickly. Standards that start as one developer’s blog post sometimes become normal within a year (schema markup followed a similar path).
- Writing the file forces you to answer a useful question: if you had sixty seconds to tell an AI system what your business does and which five pages matter most, could you do it clearly? Most business owners can’t, on the spot, and that exercise alone is worth doing.
- Some third-party AI tools and browser extensions, separate from the big model providers, do parse llms.txt today when summarising a site for a user.
So my honest answer is: add it if you have fifteen minutes and a developer who can drop a file in the root directory, but don’t reorganise your content strategy around it, and don’t pay an agency a four-figure sum to “implement your llms.txt strategy,” which I have seen advertised more than once this year.
Who should prioritise it
- SaaS companies and documentation-heavy sites, where a clean list of docs pages helps any tool, human or AI, orient quickly
- Sites with large, messy navigation where the ten pages that matter are buried under two hundred that don’t
- Businesses already investing seriously in AI visibility as a channel, where every small signal is part of a bigger, deliberate strategy
Who can skip it for now
- A five-page local business site where robots.txt and a clean sitemap already do the job
- Anyone who hasn’t yet fixed the basics, like having a clear “who we help and what we do” page written in plain English, which matters far more to both search engines and AI systems than a novel text file
How to build one, step by step
- List your five to fifteen most important pages: homepage, key service or product pages, pricing if you show it publicly, your about page, and your best cornerstone content.
- Write a one-sentence description for each page, the kind of sentence you’d use if a friend asked “what’s on that page.”
- Format it in markdown: an H1 with your business name, a short paragraph summary, then a bulleted or linked list of the pages with descriptions.
- Save it as plain text named llms.txt.
- Upload it to your site’s root directory, the same folder where robots.txt lives, so it’s reachable at yourdomain.com/llms.txt.
- Test it by typing the URL into a browser. If the raw markdown loads and reads clearly, you’re done.
Most website platforms, including WordPress with a simple file manager plugin or direct FTP access, let you do this in under twenty minutes without touching a developer.
What helps AI visibility right now
Since the file itself isn’t confirmed to be read by the big model crawlers yet, it’s worth being honest about what does influence whether your business gets mentioned when someone asks an AI tool for a recommendation. From working through this with clients over the past year, the pattern is consistent: models lean on the same signals they always have, third-party mentions, review sites, forums like Reddit and Quora, press coverage, and clearly structured on-site content that answers a specific question in the first two sentences rather than burying it under a page of brand story.
This is also why publishing more content isn’t automatically the answer. I wrote about this trap directly in why publishing more is making small businesses invisible, and the short version is that ten thin, generic blog posts do less for your AI visibility than one page that answers a real question with specifics, numbers, and first-hand detail. If you want the fuller picture on how AI search surfaces businesses, how to get found by AI walks through a real case where a stranger’s recommendation, not a technical file, was the thing that got a business named by ChatGPT.
If terms like crawler, markdown, or large language model still feel fuzzy, the AI glossary for business owners is worth ten minutes, because half the confusion around llms.txt comes from people mixing it up with completely unrelated concepts like sitemaps or structured data.
The bit worth remembering
llms.txt is not a scam and it’s not nonsense, it’s an early, well-intentioned proposal from a respected figure in AI that hasn’t yet been adopted by the companies whose crawlers would need to read it. Add it because it’s cheap and it sharpens your thinking about what matters on your site. Don’t add it expecting it to replace the actual work of building a site and a reputation that AI systems, and humans, trust enough to recommend. And if anyone tries to charge you a serious sum purely for “llms.txt optimisation,” that’s a sign to walk away, or to bring in someone who’ll tell you honestly where your budget is better spent, which is exactly the kind of conversation I have with clients through AI implementation coaching most weeks.
Frequently asked questions
Is llms.txt the same as robots.txt?
No. Robots.txt is a decades-old, widely respected standard that tells search engine crawlers which pages they can and can’t access. llms.txt is a newer, unofficial proposal from 2024 meant to summarise your site for AI systems, and it isn’t confirmed to be read by the major AI crawlers as of 2026.
Will adding llms.txt get my business recommended by ChatGPT?
Not by itself. There’s no confirmed evidence that OpenAI’s crawler reads the file, and businesses get mentioned by ChatGPT because of things like reviews, press coverage, forum mentions, and clear on-site content, not because of one text file in their root directory.
How long does it take to create an llms.txt file?
About fifteen to twenty minutes for most small business sites: list your key pages, write a one-line description for each, format it as markdown, and upload it to your site’s root folder alongside robots.txt.
Should I pay someone to set up llms.txt for me?
You shouldn’t need to pay much, if anything. It’s a simple text file, not a technical build, and anyone charging a significant fee purely for llms.txt setup is overcharging for fifteen minutes of markdown writing.
