~/search-console/excluded-by-noindex-tag

§ page indexing report

blocking

"Excluded by 'noindex' tag" in Search Console

Google found this page, crawled it, and was told not to index it. Here is where that instruction comes from, the four places it hides, and how to confirm it is gone.

This URL is not indexed and it should be. Something is in the way.

check a url →

§ check your url

Is it still happening?

Paste the URL Search Console flagged. We fetch it, read the HTML and the response headers, and report every robots directive we find and which of them actually blocks indexing.

what this proves

Whether a noindex directive is on this URL right now, and exactly which source it comes from.

what it cannot prove

We read the HTML as served, so a noindex injected by client-side JavaScript after the page loads will not appear here. Google renders JavaScript before it decides, so if this check is clean and Search Console still disagrees, use URL Inspection and look at the rendered HTML.

§ what google is telling you

What this error actually means.

Google reached the page without any trouble. Nothing is blocking the crawl, the server returned the page, and the HTML parsed. Then it read an instruction saying "do not put this in the index" and obeyed it.

This is the one indexing error that is never ambiguous. Google is not making a judgement call about quality or duplication. It is following an explicit directive that exists somewhere in your response. Your job is to find where.

It is also the error most likely to be a deployment accident rather than a decision. A noindex that was correct on staging is still correct-looking in production, and nothing about the page renders differently, so it survives review.

§ causes

5 things that produce this error.

Ordered by how often they turn out to be the cause, not by how obvious they look.

A robots meta tag in the HTML

The obvious one, and the first place to look. It sits in the <head> and is invisible in the rendered page. Note that "none" is shorthand for "noindex, nofollow" and blocks indexing just as completely.

html

<meta name="robots" content="noindex" />
<meta name="robots" content="none" />
<meta name="googlebot" content="noindex" />

An X-Robots-Tag response header

The same directive delivered over HTTP instead of HTML. It never appears in "view source", which is why it survives an entire debugging session. CDNs, reverse proxies, and hosting platforms all add it, and a rule scoped to a staging hostname gets copied to production more often than anyone admits.

http

HTTP/2 200
content-type: text/html; charset=utf-8
x-robots-tag: noindex, nofollow

A framework or CMS setting that writes the tag for you

Next.js metadata with robots.index set to false, the WordPress "Discourage search engines" checkbox under Settings > Reading, a Yoast or Rank Math per-post toggle, Shopify theme conditionals, a Webflow page setting. In every case the tag is generated, so grepping your source for "noindex" finds nothing.

// app/some-page/page.tsx
export const metadata = {
  robots: { index: false },  // renders <meta name="robots" content="noindex">
};

An environment variable that never got flipped

The pattern that causes the most damage: a conditional that emits noindex unless an environment flag says production. It works perfectly in every environment where the flag is set, and fails silently in the one where a deploy config was copied without it.

const noindex = process.env.VERCEL_ENV !== 'production';
// One missing env var on one deploy and the whole site goes dark.

A canonical pointing at a page that is itself noindexed

Less common and harder to spot. If this URL canonicalises to another URL, Google evaluates the target's directives. A clean page canonicalising to a noindexed one inherits the exclusion.

§ the fix

How to clear it.

step 1

Find every source before you change anything

There are only four places the directive can come from: the HTML meta tag, the X-Robots-Tag header, the canonical target, and your CMS or framework config. Check all four. Removing one and shipping is how this error comes back a week later, because the second source was doing the work all along.

bash

# Headers, including X-Robots-Tag
curl -sI https://example.com/page | grep -i x-robots-tag

# The meta tag in the raw HTML, before any JS runs
curl -s https://example.com/page | grep -i 'name="robots"'

step 2

Remove the directive, do not invert it

Delete the tag rather than replacing it with content="index, follow". Indexing is the default, so the positive directive adds nothing, and leaving a robots tag in place keeps the mechanism alive for the next person who edits it by hand.

step 3

Make the staging block impossible to leak

If you noindex non-production environments, key the condition on something that cannot silently default to the wrong value. Assert on the hostname you actually serve rather than trusting an env var to be present, and fail loudly when it is missing.

const host = headers().get('host') ?? '';
const isProduction = host === 'example.com';
// Absent config now means indexable, not invisible.

step 4

Then ask Search Console to look again

Open the Page indexing report, click into the "Excluded by 'noindex' tag" row, and use Validate Fix. Without it you are waiting on the natural recrawl, which for a low-traffic URL can be weeks. URL Inspection > Request Indexing does the same job for a single important page.

§ related

Next to check.

blockingBlocked by robots.txtGoogle was told not to fetch this URL at all. Here is which line of your robots.txt is doing it, why the page can still appear in search anyway, and how to fix it correctly.judgement callCrawled - currently not indexedThe most common and least actionable row in the report. Google fetched the page, read it, and decided not to index it. No tool can verify a fix for this one, including ours.blockingDuplicate without user-selected canonicalGoogle found several URLs with the same content, you did not say which one counts, so it picked for you. Here is how to take the decision back.toolMeta Tag CheckerCheck your page title, meta description, viewport, charset, and robots tags.

→ A Single Line of HTML Cost Us 47 Days of Organic Traffic

LintPage rules covering this: robots-noindex, x-robots-tag, canonical-url

§ faq

Questions, answered.

How long does it take for a page to be indexed after removing noindex?

There is no fixed interval. After you remove the directive and click Validate Fix, Google requeues the URL; established sites often see the status change within a few days, while a new or rarely-crawled site can wait several weeks. The status in Search Console reflects the last crawl, not the current state of your page, so a page can be perfectly indexable for a week and still show the old error.

Is 'Excluded by noindex tag' always a problem?

No. Plenty of URLs should carry noindex: internal search results, thank-you and checkout confirmation pages, paginated duplicates, filtered category views, and admin routes. The error is only a problem when it appears on a URL you want ranking. Read the affected URL list before you start fixing anything, because the row often contains exactly the pages you deliberately excluded.

Why does the noindex not appear when I view source?

Because it is probably not in the HTML. The X-Robots-Tag HTTP header carries the same instruction and never shows up in view-source. Check it with curl -I, or in your browser devtools under Network > the document request > Response Headers. This is the single most common reason people spend hours searching a codebase for a string that was never there.

Does robots.txt cause this error?

No, and the distinction matters. robots.txt controls crawling; noindex controls indexing. They produce different Search Console errors, and they interact badly: if you block a URL in robots.txt, Google cannot fetch the page and therefore cannot see the noindex on it. To remove a page from the index you must let Google crawl it and find the noindex. Blocking it in robots.txt instead can leave the URL indexed indefinitely with no description.

What is the difference between noindex and nofollow?

noindex keeps the page out of search results. nofollow tells Google not to pass ranking signals through the links on that page. They are independent, and content="none" applies both at once. If your goal is simply to keep a page out of the index, use noindex on its own and leave the links followable.

§ before the next deploy

Catch this one before Search Console does.

Search Console tells you weeks after the fact. LintPage runs 60 checks against a URL in about 30 seconds. Free, no signup.

run a full scan →