Seoxpert.io
Get started
Glossary/noindex
Glossary

What Is noindex?

noindex is a directive that tells search engines not to include the page in search results. Two supported ways to set it: a meta robots tag in the page <head>, or the X-Robots-TagHTTP response header. The robots.txt “Noindex:” directive was deprecated by Google in 2019 and no longer works.

How to set noindex

Meta robots tag (HTML)

<head>
  <meta name="robots" content="noindex">
</head>

Place inside <head>. The directive applies to the URL it's on. Works for any HTML page.

X-Robots-Tag (HTTP header)

X-Robots-Tag: noindex

Set as an HTTP response header. The only way to noindex non-HTML resources (PDFs, images, raw JSON endpoints). Target specific bots: X-Robots-Tag: googlebot: noindex.

noindex vs robots.txt Disallow

The most-misunderstood distinction in technical SEO.

  • — noindex blocks the page from appearing in search results.
  • — robots.txt Disallow blocks crawlers from fetching the page. The URL can still appear in search results (without a description) if other sites link to it, because Google knows the URL exists even without crawling it.
  • — Critical: if you noindex a page AND block it in robots.txt, Googlebot never sees the noindex directive and the page stays indexed indefinitely. Always keep noindex pages crawlable.

When to use noindex

  • — Internal search-result pages. /search?q=... URLs are infinite-cardinality duplicates of the homepage.
  • — Filter / facet URLs. Filtered product or category pages that don't add unique value beyond the canonical category.
  • — Paginated archives beyond page 5. Deep pagination usually offers no unique value.
  • — Admin / utility URLs. Login, account settings, cart, checkout — pages that exist for users but shouldn't rank.
  • — Thank-you / confirmation pages. Post-form-submission destinations that don't serve search intent.
  • — Test / staging environments. Use noindex on staging hosts to prevent accidental indexing.

When NOT to use noindex

  • — Thin content you actually care about. Fix or delete the page; don't hide it.
  • — Duplicate content. Use a canonical tag — it consolidates ranking signals into the canonical URL. noindex loses those signals entirely.
  • — Pages with valuable inbound links. noindex doesn't inherit link equity to other pages; a canonical does.

Related terms

  • — robots.txt — the access-control file (different from noindex).
  • — Canonical tag — for duplicate-content consolidation (preferred over noindex).
  • — Meta description — another meta tag in the page head.

How Seoxpert checks noindex

Every scan reports pages carrying noindex (meta or HTTP header) and flags the dangerous combinations: noindex on canonical URLs, noindex pages also blocked in robots.txt, conflicting meta-robots directives. Run a free website audit or browse the technical SEO issue library.

Frequently asked questions

What is the difference between noindex and robots.txt Disallow?

noindex blocks indexing — the URL stays out of search results entirely. robots.txt Disallow blocks crawling — but the URL can still appear in search results (without a description) if other sites link to it. If you want a page out of the index, use noindex AND keep the page crawlable so Google can see the directive.

Can I use noindex in robots.txt?

No. Google deprecated the unofficial "Noindex:" directive in robots.txt in 2019. The only supported ways to set noindex are the meta robots tag (<meta name="robots" content="noindex">) in the page <head> or the X-Robots-Tag HTTP response header.

How long until a noindex page drops from Google?

Google must re-crawl the page to see the noindex directive. After re-crawl, the page typically drops from the index within days. If you also block the page in robots.txt, Googlebot never sees the noindex and the page stays indexed indefinitely — a common mistake.

What is the difference between noindex and nofollow?

noindex tells search engines not to include the URL in the index. nofollow tells them not to follow the links on the page (or to specific links if applied per-link). You can combine them: <meta name="robots" content="noindex, nofollow"> blocks indexing and link-following on the same page.

Should I noindex thin or low-quality pages?

For genuinely low-value pages that exist for navigation only (filter URLs, search-result pages, paginated archives beyond page 5), noindex is the right answer. For thin content pages on a topic you care about, deleting or merging them is usually better than noindex — the noindex hides the symptom without fixing the underlying content problem.

Audit noindex usage across your site — free.