noindex is a rule you serve on a page to tell search engines not to show it in their results, while the page stays live for anyone who visits it directly. It works in exactly one situation: when the crawler can reach the page and read the rule. It is written for the founder, growth lead or developer staring at pages in Google that should not be there, or missing pages that should be. It covers the two ways to deliver the directive, all twelve robots rules and how they combine, what noindex cannot do, how to confirm one has landed, and the failure modes that quietly remove more than you meant or nothing at all.
The short version
- A
noindexthat Google cannot crawl does nothing. Block the same URL inrobots.txtand the rule can never be read at all.noindexdoes not save crawl budget. It costs one, because the page has to be fetched before the rule can be seen.- The robots meta tag carries twelve rules, not one. Five of them shape how your result looks rather than whether it appears at all.
- When two robots rules conflict, Google applies the more restrictive one.
- Removal is not instant. Google's guidance is that a revisit can take months, and it publishes no typical figure.
What does noindex actually do?
It keeps a page that Google has already fetched out of search results. The page still loads, still returns 200, still works for every visitor and every link. The only thing that changes is whether Google is willing to show it.
That distinction matters because two very different states look identical from the outside. A page Google never crawled is absent because nothing was ever read. A page carrying noindex is absent because something was read and obeyed. The first is a discovery problem, the second is a working directive, and the fix for one makes the other worse.
Everything downstream in this guide follows from a single mechanical fact: the rule lives inside the response. Google has to request the page and receive it before the instruction exists as far as the crawler is concerned. A directive nobody fetches is a directive nobody follows.
What are the two ways to serve a noindex, and when does only the header work?
Two, and they are equivalent in effect. You can put a robots meta tag in the page's HTML, or you can return an X-Robots-Tag HTTP response header. Google treats both as the same instruction.
The meta tag goes in the <head>:
<meta name="robots" content="noindex">The header is set by the server or the CDN and travels with the response:
X-Robots-Tag: noindexMost teams reach for the meta tag by reflex, which is fine until they hit a route where there is no HTML to edit. Google's robots meta tag and X-Robots-Tag specification, last updated 2026-03-24, names the case plainly: "You can use the X-Robots-Tag for non-HTML files like image files where the usage of robots meta tags in HTML is not possible." The same page notes that "the X-Robots-Tag may optionally specify a user agent before the rules."
Three situations point to the header, and in the first two there is no alternative.
- Non-HTML files: a PDF, an image, a CSV export or a video file has no
<head>to put a tag in. If you want a generated report out of search, the header is the only delivery method available. - Routes whose markup you do not control: assets served straight from object storage, files emitted by a build step, or endpoints handled by a third party. You can usually add a response header at the CDN or reverse proxy even when the body is untouchable.
- Per-crawler targeting: the optional user-agent prefix lets you address one crawler and leave the others alone. The meta tag can do this too, using a crawler-specific
nameattribute, but at the header level you can apply the decision across a whole route pattern rather than page by page.
| Question | Robots meta tag | X-Robots-Tag header |
|---|---|---|
| Where does it live? | In the page's <head> | In the HTTP response headers |
| Works on non-HTML files? | No | Yes |
| Who normally ships it? | Whoever owns the template or CMS field | Whoever owns the server, CDN or edge config |
| Visible in "view source"? | Yes | No, you have to read the headers |
| Applies to a whole route pattern? | Only if the template does | Yes, by path rule |
A practical consequence of the last two rows: a page can carry a noindex that nobody on the team can see. That is a common reason a mystery deindexing investigation stalls before it starts.
Which robots rules exist, and what does each one control?
Twelve. Google's current specification lists twelve valid rules, and noindex is one of them. The other eleven are easy to go years without meeting, which is why snippet and preview behavior gets blamed on mysterious algorithm changes when the cause is a rule somebody set on purpose.
| Rule | What it does | What it controls |
|---|---|---|
all | No restrictions on indexing or serving. The default. | Both |
noindex | Do not show this page, media or resource in search results. | Whether it appears |
nofollow | Do not follow the links on this page. | Crawl of outbound links |
none | Equivalent to noindex, nofollow. | Both |
nosnippet | Do not show a text snippet or video preview for this page. | How the result looks |
indexifembedded | Allows indexing of content embedded in another page through iframes or similar tags, in spite of a noindex rule. | Whether it appears |
max-snippet:[number] | Cap the textual snippet at a set number of characters. | How the result looks |
max-image-preview:[setting] | Set the maximum size of an image preview. | How the result looks |
max-video-preview:[number] | Cap a video snippet at a set number of seconds. | How the result looks |
notranslate | Do not offer translation of this page in search results. | How the result looks |
noimageindex | Do not index images on this page. | Whether images appear |
unavailable_after:[date/time] | Stop showing this page in results after a given date and time. | Whether it appears |
Google lists all twelve under a single heading, "Valid indexing and serving rules," and does not label them one by one. The grouping in the third column is ours, and it is the split that matters when you are debugging: a rule either decides whether the page can appear at all, or it shapes how the result looks once it already does. Confusing the two sends people hunting for a ranking problem when what they have is a snippet setting.
indexifembedded is the one worth knowing about even if you never use it. It exists for content that should not rank on its own URL but should still count when it is embedded elsewhere, which is a real pattern for video players and widget endpoints.
The same specification keeps a short list of rules Google no longer uses. noarchive, nocache and nositelinkssearchbox are all ignored, and in two of those cases Google says the feature the rule controlled no longer exists. Finding one in an old template tells you how old the template is and nothing about how the page is treated today.
What happens when two robots rules conflict?
The stricter one wins. Google states it directly: "In the case of conflicting robots rules, the more restrictive rule applies." That single sentence resolves most of the confusion around combined directives, and you want it in your head before writing a multi-rule instruction.
You are allowed to combine them. Per the same specification, "you can create a multi-rule instruction by combining robots meta tag rules with commas or by using multiple meta tags." Both of these say the same thing:
<meta name="robots" content="noindex, nofollow">
<meta name="robots" content="noindex">
<meta name="robots" content="nofollow">The combination worth understanding is noindex, follow. It means keep this page out of results but keep crawling the links on it, and it suits a paginated series or a filtered listing whose own URL adds nothing but whose links lead somewhere that matters.
Read alongside the restrictiveness rule, it also explains a failure that catches teams out. If one template sets noindex, follow and a plugin adds nofollow further down the same <head>, the page ends up with both indexing and link following suppressed. Nobody wrote none, but that is effectively what was served. The habit that avoids it: never assume you are the only thing writing a robots rule onto the page, and check the rendered output rather than the template you edited.
What does noindex not do?
Four things it is regularly expected to do and does not. Each one has a proper tool, and reaching for noindex instead is how sites end up with a directive that solves nothing.
- It does not save crawl budget: the page must be fetched for the rule to be read, so the request happens either way. Our guide to what noindex costs your crawl budget works through why the directive adds a fetch rather than removing one.
- It does not consolidate ranking signals: links pointing at a noindexed page are not redistributed to a page you would rather rank. If the goal is to merge duplicates onto one URL, the right instrument is a canonical rather than a removal, and the two contradict each other when stacked on the same page.
- It does not work on a URL you have blocked from crawling: Google's specification is unambiguous that "if a page is disallowed from crawling through the robots.txt file, then any information about indexing or serving rules will not be found and will therefore be ignored." Our robots.txt guide covers why a block is not a removal and why blocked URLs can still surface.
- It does not make the page private: anyone with the link still loads it, and so does any crawler that ignores the rule. Keeping something genuinely confidential is a job for authentication, not for a hint to well-behaved bots.
The pattern behind all four: noindex governs one narrow thing, which is whether a fetched page is eligible to appear. Every problem outside that boundary needs a different control.
How long does a noindexed page take to drop out of Google?
Google does not publish a number for this, and it is worth saying so plainly rather than repeating one someone invented. What Google does say, in its documentation on blocking indexing, last updated 2025-12-10, is this: "Depending on the importance of the page on the internet, it may take months for Googlebot to revisit a page."
That framing is the useful part. Removal is not on a timer, it is on a crawl schedule, and the schedule is driven by how much Google cares about the URL. A page linked from your homepage gets revisited quickly. An orphaned PDF from three years ago can outlast your patience after you set the header, and nothing is broken when it does.
Two things follow for planning. If a handful of URLs genuinely need to go, request indexing on each through URL Inspection so the recrawl is prompted rather than waited for. If thousands need to go, accept that the tail runs on Google's schedule, confirm the directive is being served correctly, and measure progress by whether the count is falling rather than by whether any single URL has cleared.
How do you check that a noindex actually landed?
Check the response, not the template, and check it in two places. Google's documentation on blocking indexing, linked above, names both: "to test if your noindex implementation is correct, use the URL Inspection tool to see the HTML that Googlebot received while crawling the page. You can also use the Page Indexing report in Search Console to monitor the pages on your site from which Googlebot extracted a noindex rule."
URL Inspection answers the question that matters, which is what Googlebot received rather than what your CMS believes it sent. The page indexing report is the bulk view, and it is where you watch a planned removal actually progress instead of assuming it did.
Neither of those closes the header gap on its own. From my experience, the X-Robots-Tag is what gets missed in these investigations, because the HTML looks clean, view-source shows nothing, and nobody thinks to read the response. One command settles it:
curl -I https://example.com/your-pageIf an X-Robots-Tag line comes back, that is your answer. Run the same command against a PDF or an image route when you are auditing a directive you set at the CDN, because that is exactly where a path rule tends to be broader than whoever wrote it intended.
Which pages should carry a noindex?
Fewer pages than you would expect. The honest test is whether the page would ever be a satisfying answer to a real search, and whether it has any independent reason to exist in an index at all.
Working out which page types pass that test is a decision with its own branches, and we have already mapped it: our landing page guide sets out the index-or-noindex decision tree and how to put each branch into effect. This section is here to point you there rather than to answer the same question twice.
What goes wrong most often?
Four failures are the ones worth memorizing, and all four are quiet. Nothing errors, nothing alerts, and the symptom arrives weeks later as traffic that never came.
Blocking and noindexing the same URL: this one fails by construction. The block stops the fetch, the fetch was the only way to read the rule, and the URL survives in the index indefinitely. If a page needs to leave the index, keep it crawlable until it has actually gone.
Putting the rule in the wrong file: a related version of the same mistake is writing a noindex line into robots.txt. Google's robots.txt specification, last updated 2026-08-31, lists four supported fields: user-agent, allow, disallow and sitemap. noindex is not among them, and unsupported fields are ignored. The line reads like a working instruction in a code review and does nothing at all.
A staging directive reaching production: staging environments are usually noindexed by default, correctly so, and the rule travels with the build when nobody strips it. Of the four, this is the version our technical audits turn up most often, and it is invisible from the front end because the site looks perfectly normal to every human who checks it. Our guide to redesigning a site without losing rankings covers where in a launch sequence this gets caught.
Noindexing a page that has earned links: removing a page from the index does not pass its links anywhere. If a URL has genuine external links and you simply do not want it competing, a canonical or a redirect keeps that value in play where a removal throws it away.
Every one of these is cheap to catch and expensive to miss, which is the argument for checking directives on a schedule rather than on suspicion. If you would rather have indexing control audited properly across a whole site, that is part of our technical SEO work.
One habit is worth building out of all this: treat "is this page indexable" as a deploy check, not an SEO check. Directives are shipped by templates, plugins, build steps and edge rules, which means they change when code changes and not when anyone happens to be thinking about search. A rule that was right in staging is a bug in production, and the reliable moment to catch it is the moment the code moves.
Before you close this tab, pick two URLs: the one you would be most alarmed to find in Google, and the one you would be most alarmed to find missing. Check both properly, view source for the meta tag and curl -I for the header. If either answer surprises you, you have just found the first thing to fix.
Wrong Pages in Google, Right Pages Missing?
GrowthHasten audits indexing control so search engines show the pages you want found and leave the rest alone.
Talk to an SEO ExpertFrequently Asked Questions
What does noindex mean?
Noindex is a rule telling search engines to keep a page out of their results even though the page itself stays live and reachable for anyone who visits. Two delivery methods carry it: a robots meta tag inside the HTML, or an X-Robots-Tag header returned alongside the HTTP response by your server or CDN. Both mean the same thing. The rule takes effect only if the crawler can fetch the page and read it.
Does Google crawl noindex pages?
Yes, and it has to. The noindex rule lives inside the page response, either in the HTML or in an HTTP header, so Google can only obey a rule it has fetched and read. This is why noindex does not reduce crawling: the request happens either way. It also explains the most common failure with the directive, which is blocking the same URL in robots.txt so the crawler never reaches the rule at all.
What is the difference between noindex and nofollow?
They control different things. Noindex governs whether a page can appear in search results. Nofollow, used as a robots rule, tells search engines not to follow the links on that page. The rule none is equivalent to both together. One point of confusion is worth naming: nofollow in a robots meta tag applies to every link on the page, which is a different thing from a rel=nofollow attribute set on one individual link.
How do you check if a page has a noindex?
Check three places. Open the page source and search the head for a robots meta tag. Then read the HTTP response headers, using a command such as curl -I, because an X-Robots-Tag will never appear in view-source and is the case teams miss most often. Finally, inspect the URL in Search Console, which reports what Googlebot genuinely received instead of what your CMS thinks it served.
How long does it take for a noindexed page to drop out of Google?
Google publishes no typical figure, so treat any specific number you see quoted with suspicion. Its guidance says only that a revisit may take months, depending on how important the page is on the internet. Removal therefore follows the crawl schedule rather than a clock: a well-linked page usually clears quickly, while a neglected one can linger. For a handful of urgent URLs, request indexing in URL Inspection to prompt a recrawl.
How do I remove a noindex my CMS added by itself?
Find which layer is adding it first, because the fix differs by layer. Check your CMS or SEO plugin's visibility settings, because a single site-wide search-visibility toggle can apply noindex to every page at once and is easy to leave switched on after a launch. If the tag is not coming from there, read the response headers for an X-Robots-Tag set at the server or CDN. Each platform documents its own setting, and the vendor's documentation is the right place to look.
Tags
Anshuman Sinha
AI SEO Specialist, GrowthHasten
Anshuman Sinha is an AI SEO Specialist and Computer Science Engineer with over three years of experience in SEO and five years in web development. He specializes in Technical SEO, AI Search Optimization (AEO and GEO), SaaS SEO, and building high-performance websites with modern technologies.
View profile
