§ Yoast SEO

llms.txt with Yoast SEO

Whether Yoast generates an llms.txt file, how it interacts with the one you wrote by hand, and how to serve your own without fighting the plugin.

Yoast is a WordPress plugin, so the file is served by WordPress. What Yoast adds is content generation, not a new serving mechanism.

generate one from your site →

§ how to serve it

Serving /llms.txt from Yoast SEO.

step 1

Check what your Yoast version actually ships

llms.txt support has arrived across SEO plugins at different times and under different names, and what a given release generates varies. Before assuming it is handled, fetch the file and read it. If the plugin is generating one, it will be there; if not, the URL 404s or falls through to your 404 template. Do not take a settings toggle as evidence of the output.

bash

curl -s https://example.com/llms.txt | head -20

step 2

Decide which file wins before you write a second one

A plugin-generated file and a hand-written file at the same URL are a conflict, and which one is served depends on whether your file is a real file on disk or another rewrite rule. A real file in the web root wins, because the web server never asks WordPress. Two rewrite rules is a coin flip decided by priority - avoid it.

step 3

If you want Yoast to generate it, curate the input

A generated file is only as good as what you point it at. Yoast already knows which content you have marked as noindex, which posts are cornerstone content, and what each page's meta description says. Cornerstone marking is the closest thing WordPress has to "this is what a model should read first" - set it deliberately rather than leaving the defaults.

step 4

If you want your own, take the URL properly

Upload a real llms.txt to the web root. Because the web server serves an existing file before WordPress runs, this takes precedence over any plugin rewrite without needing to disable anything - and it keeps working if the plugin changes its behaviour in a future release.

bash

# Beside wp-config.php, not in wp-content/
public_html/llms.txt

§ what goes wrong here

What Yoast SEO gets wrong.

The failures below are specific to Yoast SEO. A generic llms.txt guide will not mention any of them.

Assuming an SEO plugin handles it because it handles sitemaps

The Yoast-specific misconception, and it is an understandable one: Yoast does generate and serve XML sitemaps, so people assume llms.txt is the same feature with a different extension. It is not. A sitemap is a complete machine-readable list of URLs; llms.txt is a short curated index with prose notes. A plugin that generates the second from the first has built a sitemap with markdown syntax.

A generated file that is just your sitemap in markdown

The failure mode to check for. If the generated file lists every post on your site with the meta description as the note, it will pass a validator and fail at its actual job. The value of llms.txt is curation and the quality of the notes. If the plugin cannot give you that, hand-write the file.

Yoast's noindex settings do not carry over

A page marked noindex in Yoast is excluded from your sitemap. Whether it is excluded from a generated llms.txt depends entirely on the implementation. Check: a staging page, a thank-you page, or an internal doc appearing in a public index is a leak, not a formatting issue.

The plugin and your robots.txt disagreeing

Yoast also edits robots.txt. If your robots.txt blocks GPTBot, ClaudeBot or the rest while an llms.txt invites them, the two files contradict each other. Both are public, both are read by the same clients, and only one of them can be what you meant.

§ related

Next to read.

WordPressllms.txt on WordPressWordPress routes every unknown URL through index.php, which is why an uploaded llms.txt can return your 404 template with a 200 status. Three ways to serve it, and how to tell which one your host allows.diagnosticllms.txt does not follow recommendationsA tool told you your llms.txt does not follow recommendations, without saying which one. Here is every rule the format actually defines, which ones are the spec and which are a tool's opinion, and how to tell them apart.Shopifyllms.txt on ShopifyShopify does not let you put arbitrary files at the domain root, so /llms.txt is not directly writable. What you can actually do, and which workarounds are worth it.toolllms.txt CheckerFetch your /llms.txt, validate it against the spec, and report what is missing.

→ llms.txt vs robots.txt: One Controls Access, One Is a Reading List

§ faq

Questions, answered.

Does Yoast SEO generate an llms.txt file?

SEO plugins including Yoast have added llms.txt features, and what each version generates differs. Rather than relying on a changelog, fetch https://yoursite.com/llms.txt and read what comes back - that is the only reliable answer for your specific install, and it takes ten seconds.

Should I use Yoast's llms.txt or write my own?

Write your own if you care about the file working. A generated index inherits your meta descriptions, which are written to earn clicks from humans in search results, not to tell a model what a page contains. Generation is a reasonable starting point; shipping it unedited is how you end up with a sitemap in markdown syntax.

How do I override a plugin-generated llms.txt?

Upload a real llms.txt file to your web root, beside wp-config.php. Web servers serve existing files before handing the request to WordPress, so your file wins without disabling anything in the plugin - and it stays winning if the plugin changes behaviour later.

Is llms.txt the same as an XML sitemap?

No, and the difference is the whole point. A sitemap is exhaustive, machine-readable, and written for crawlers that will fetch everything anyway. An llms.txt is short, curated, and annotated in prose, written for a client with a limited context budget that has to choose what to read. Generating one from the other loses exactly the part that makes it useful.

§ the part that matters

An llms.txt will not fix a site AI cannot read.

Before writing an index for AI clients, check that they can fetch your pages at all. LintPage runs 60 checks against a URL in about 30 seconds - free, no signup.

run a full scan →