NoindexWatch

X-Robots-Tag checker

Enter a URL to see the X-Robots-Tag header on the final response and on every redirect before it, what each directive does, any canonical sent in the Link header, and the full response headers.

Where X-Robots-Tag comes from

The header is added outside the HTML: an nginx add_header or Apache Header set line, a CDN transform rule, a hosting platform's preview-domain setting, or an SEO plugin. Staging environments often send X-Robots-Tag: noindex on every response, and the rule can travel to production with the server config.

Header or meta tag

Google treats X-Robots-Tag and <meta name="robots"> the same way and applies the most restrictive of the two. For PDFs, images and other non-HTML files the header is the only option. Either way, Googlebot must be allowed to crawl the URL to see it: a robots.txt disallow hides the noindex. Test that with the robots.txt tester.

Crawler-specific values

X-Robots-Tag: googlebot: noindex applies to Google only; a value without a name applies to all crawlers. Send one header per crawler. This checker reads each header separately.

We fetch the URL once with GET, as NoindexWatchBot. Some servers send different headers to Googlebot. Results are not stored; cookie values are not shown.