Seoxpert.io
highTechnical SEO

Pages with Noindex Directives

Pages carry noindex directives that prevent them from appearing in search results. Verify every one is intentional.

By Seoxpert Editorial · Published · Updated

Why it matters

noindex is a one-way door: pages carrying it are pulled from Google's index. When applied accidentally — a staging rule pushed to production, a template-level directive, a CMS toggle left on — it silently removes content from search with no warning in Search Console for days.

Impact

An accidentally-noindexed high-traffic page loses all organic traffic within weeks of Google re-crawling it. Recovery is slow: removing the directive only restarts indexing; ranking takes additional time to re-establish.

How it's detected

Scanner reads each page's <meta name="robots"> and googlebot directives and the X-Robots-Tag HTTP response header. Any occurrence of "noindex" or "none" in either place flags the page.

Common causes

  • Developer added noindex during staging and the rule was pushed to production
  • WordPress "Discourage search engines from indexing this site" checkbox left on after launch
  • Template-level rule applied to every page of a content type
  • X-Robots-Tag set globally at the server or CDN layer and not overridden

How to fix it

Export the full list of noindexed URLs. For each, decide: (1) keep noindex because the page is genuinely not meant for search (internal search results, thank-you pages, staging previews), or (2) remove the directive so the page can be indexed. Do not block noindexed pages in robots.txt — Google needs to crawl them to see the directive at all.

Code examples

Meta robots noindex (in the HTML head)

<!-- noindex: not shown in search results -->
<meta name="robots" content="noindex" />

<!-- target only Googlebot -->
<meta name="googlebot" content="noindex" />

<!-- remove: delete the tag entirely (not content="index") -->

X-Robots-Tag HTTP header (applies to any resource type)

HTTP/1.1 200 OK
Content-Type: application/pdf
X-Robots-Tag: noindex

# Useful for non-HTML resources where meta tags are impossible (PDFs, images).

FAQ

Does noindex block crawling?

No. It instructs Google not to include the page in search results, but Google must still crawl the page to see the directive. Do not also disallow the page in robots.txt, because that prevents Googlebot from ever seeing the noindex — the page may stay indexed from before.

How long until a noindexed page drops out of search?

Typically one to a few weeks, depending on how often Google re-crawls the URL. You can speed this up by requesting recrawl in Search Console, or by using the URL removal tool for urgent cases.

What about noindex,nofollow?

noindex,nofollow tells Google not to index the page and not to follow any links on it. Useful for private thank-you pages or internal tools that link to content that should also stay out of the index.

Found this issue on your site?

Run a scan to see if Pages with Noindex Directives affects your pages.

Scan my website →