Your Site Has a Hidden Noindex Tag. Here Is How to Find It

You launched three months ago. You've been posting on X, sharing on Indie Hackers, getting some traffic. But when you Google your own site, nothing shows up. Not your homepage. Not any page. It's like you don't exist.

This is one of the most common SEO disasters for indie builders, and it's almost always caused by the same thing: a leftover noindex tag from development. Here's how to find it.

The quick test

Open Google. Search site:yourdomain.com. If you see zero results after your site has been live for more than a week, something is blocking Google from indexing it.

If you see some results but not all your pages, the problem might be partial (some pages have noindex, others don't).

Where noindex can live

There are three places a noindex can come from. You have to check all three.

1. Meta robots tag in the HTML

The most common. Somewhere in your HTML, you have a tag like this:

<meta name="robots" content="noindex" />
<meta name="robots" content="noindex, nofollow" />
<meta name="googlebot" content="noindex" />

This tells search engines not to index the page. It usually gets added during development so staging doesn't get indexed, then forgotten when the site goes live.

2. X-Robots-Tag HTTP header

A server-level version of the meta tag. You can't see it by viewing source. It looks like this in the HTTP response:

X-Robots-Tag: noindex

Nginx, Apache, Cloudflare, and some hosting providers can add this header at the server level. If it's there, the meta tag doesn't even matter - Google respects the header.

3. Robots.txt disallow rule

A different thing, but has the same effect for your SEO. If your robots.txt has:

User-agent: *
Disallow: /

Google is blocked from crawling your entire site.

How to check each one

Check the meta tag

Right-click your homepage, click View Page Source (not Inspect Element). Search the page for noindex. If you find anything, that's your problem.

Important: Inspect Element shows the post-JavaScript DOM. If your noindex tag is added server-side but removed by JavaScript, Inspect Element will lie to you. Always use View Page Source.

Check the HTTP header

Open Chrome DevTools (F12). Go to the Network tab. Reload the page. Click the first request (the HTML document itself). Look at Response Headers. Search for X-Robots-Tag.

Or, from the command line:

curl -I https://yoursite.com | grep -i robots

If anything comes back with noindex in it, that's your problem.

Check robots.txt

Open https://yoursite.com/robots.txt in your browser. Look for:

Disallow: /

If you see that on a line by itself under User-agent: *, you're blocking all crawlers. Fix it by removing the line or changing it to Allow: /.

Use Google Search Console

The most authoritative check. Go to Google Search Console. Click URL Inspection at the top. Paste your homepage URL. Google will tell you exactly whether the page is indexable and, if not, why.

Common messages:

• "URL is not on Google" with reason "Excluded by noindex tag" - found it
• "Crawled - currently not indexed" - Google found it but hasn't decided to index yet (different problem, usually content quality or thin content)
• "Discovered - currently not indexed" - Google saw the URL but hasn't crawled yet (just need to wait)
• "Blocked by robots.txt" - robots.txt is blocking the page

Where the noindex tag usually comes from

If you find a leftover noindex tag, it probably came from one of these:

• Environment-based config. Your NODE_ENV was set to "development" in production, and your code only adds noindex in dev, so you accidentally shipped with the dev env var.
• CMS setting. WordPress, Wix, Squarespace, and Ghost all have a "discourage search engines" setting in Settings → Reading or similar. Toggle it off.
• Framework default. Some static site generators include a noindex tag in the default template for the root layout. Check your base layout file.
• Cloudflare or hosting rule. Some providers add X-Robots-Tag to protect a staging environment. Check your deployment config.
• Password-protected preview. Vercel, Netlify, and others add noindex to preview deployments by default. If you're accidentally pointing your domain at a preview URL, that's why.

How to fix it

The fix depends on where it's coming from. Once you know the source, remove the tag, redeploy, and verify with Google Search Console.

For a meta tag in your HTML:

1. Find the tag in your template or layout file
2. Remove it (or change noindex to index)
3. Redeploy
4. View Page Source on the live site to confirm it's gone

For an X-Robots-Tag header:

1. Check your server config (nginx, Apache, next.config.js headers)
2. Check your hosting provider's dashboard
3. Check any CDN rules (Cloudflare, Fastly)
4. Remove the rule, clear the cache, test with curl

For robots.txt:

1. Edit the file
2. Change Disallow: / to Allow: / or remove the line
3. Redeploy
4. Verify by visiting /robots.txt

After you fix it

Removing the block doesn't instantly restore your rankings. Google has to re-crawl and re-index the pages, which takes days to weeks. Speed it up:

1. Go to Google Search Console
2. URL Inspection → your homepage
3. Click "Request Indexing"
4. Repeat for your most important pages
5. Make sure your sitemap.xml is submitted in Search Console

Expect initial indexing within 1-7 days after requesting. Full re-indexing of all pages can take 2-6 weeks.

How to never do this again

The root cause is always the same: development-only config that leaked into production. The fix is to check for noindex on every deploy, or to use a tool that does it for you.

CalmLaunch's scan checks for noindex tags, X-Robots-Tag headers, and robots.txt disallow rules automatically. It's part of the Discoverability category in the eight launch categories. If you run a scan before launch, you catch this in the first 30 seconds.

Common questions

How do I know Google has finished re-indexing?

Search site:yourdomain.com in Google. The result count should match (roughly) the number of pages you have. If you had 50 pages before and you're seeing 5 in search, Google hasn't finished.

Does Bing use the same noindex rules?

Yes. Bing, DuckDuckGo, and most search engines respect the same noindex meta tag and X-Robots-Tag. Fixing for Google fixes them all.

Does Perplexity or ChatGPT respect noindex?

Mostly yes for training data crawlers, but AI answer engines that crawl live pages (Perplexity, for example) behave differently. A noindex tag blocks your page from appearing in web search but may not block it from being cited in AI answers. To block AI crawlers specifically, you need separate directives like User-agent: PerplexityBot in robots.txt.

What if my homepage is indexed but my blog posts aren't?

Check each blog post's View Page Source for noindex. Sometimes a template used for one section of the site has a noindex tag that the homepage doesn't.