Skip to content
SEO Madmanby Adam Hafez
Free tool

Robots meta tag validator with combined Googlebot result

Robots meta tag validator: check meta robots and X-Robots-Tag rules against Google's list, and catch bad values and conflicts. Free, runs in your browser.

Runs entirely in your browser. Nothing you enter is uploaded, logged or stored.

Nothing you paste here is sent anywhere; it is parsed in your browser only.

    Key takeaways

    • Google documents the robots meta tag names robots, googlebot and googlebot-news, and accepts the same rules in an X-Robots-Tag header.
    • When robots rules conflict, Google applies the more restrictive one, so nosnippet wins over max-snippet.
    • The none rule is equivalent to noindex combined with nofollow, so listing either alongside it is redundant.
    • The max-image-preview rule accepts only none, standard or large, and max-snippet and max-video-preview take a whole number of at least minus one.
    • The noarchive rule no longer controls anything in Google Search, because the cached link feature it governed has been removed.

    What it takes

    • Robots meta tags, pasted as HTML
    • X-Robots-Tag header lines

    More tools

    All of them free, all of them browser-only.

    Browse the tools

    How to use it

    1. 1Paste your tags and header linesOne per line. Both a full meta tag and an X-Robots-Tag line work, and you can mix them to see how a page's HTML and its HTTP headers combine.
    2. 2Read the combined Googlebot resultThe box at the top merges every entry that applies to Googlebot, meaning tags for all crawlers or for googlebot, and applies the more restrictive rule where they conflict.
    3. 3Fix anything marked as an errorErrors are rules Google would not parse, such as max-image-preview with a value it does not accept, or unavailable_after with no readable date.
    4. 4Check the warningsA warning marks a rule outside Google's documented list, a repeated rule, or an unavailable_after date that has already passed.
    5. 5Put user agent names in the header, not the ruleA header such as X-Robots-Tag with a googlebot prefix targets that crawler only, while a header without a prefix applies to all of them.

    A robots meta tag and an X-Robots-Tag header do the same job in two places, and a page can carry both. This validator checks them against the rule list Google publishes and shows what Googlebot ends up with once they are combined. It runs in your browser, and nothing you paste is sent anywhere.

    What the validator checks

    Each line you paste is read as either a meta tag or an X-Robots-Tag header, and each rule in it is checked against Google’s specification:

    • Tag name. A meta tag must be named robots, googlebot or googlebot-news, the names Google documents. Any other name is a warning.
    • Plain rules such as noindex, nofollow, none and nosnippet take no value. A value after one of them is an error.
    • max-snippet and max-video-preview need a whole number: 0 for none, -1 for no limit, or a positive limit. Google says max-snippet:0 is equivalent to nosnippet.
    • max-image-preview accepts none, standard or large, and nothing else.
    • unavailable_after needs a readable date. Google accepts widely adopted formats including RFC 822, RFC 850 and ISO 8601. A date already in the past is a warning, because the page is then treated as expired.
    • Unknown rules are flagged rather than guessed at, which catches typos and rules from other search engines.

    It also checks the rules against each other: a rule repeated in one line, none alongside noindex or nofollow, nosnippet alongside max-snippet, and indexifembedded without a noindex to pair with.

    How to read the results

    The box at the top is the combined Googlebot result. It merges every entry addressed to all crawlers or to googlebot, and reports whether the page is indexed, whether links are followed, and whether text snippets are blocked. Entries for googlebot-news or another named crawler are checked but left out of that merge.

    Below it, each line is listed with the crawler it applies to. Red is an error Google would not parse, amber is a warning, and grey is a note about redundancy or a resolved conflict. A line with no findings is marked valid and lists its rules.

    Worked example

    The prefilled input has four lines with problems left in on purpose:

    • <meta name="robots" content="noindex, max-snippet:50, nosnippet"> applies to all crawlers. The note says nosnippet and max-snippet conflict and the more restrictive rule, nosnippet, applies.
    • <meta name="googlebot" content="max-image-preview:huge"> is an error: max-image-preview accepts none, standard or large.
    • X-Robots-Tag: googlebot: unavailable_after: 25 Jun 2010 15:00:00 PST targets googlebot, and its date is in the past, so the page counts as expired.
    • X-Robots-Tag: noarchive, nofollow gets a warning for noarchive, which Google no longer uses.

    The combined result reads: Not indexed (noindex) · Links not followed (nofollow) · No text snippet.

    Common mistakes it catches

    • Invented values, such as max-image-preview:huge or max-snippet:short.
    • Retired rules kept from old templates, above all noarchive.
    • Contradictions between the header and the HTML. A header noindex and a meta tag that says nothing combine to noindex, because the more restrictive rule applies.
    • Redundant rules, such as none written next to noindex, which make templates harder to read.

    One mistake it cannot see from the tags alone: a Disallow in robots.txt. Google states that if a page is disallowed from crawling, its indexing and serving rules are never found. RFC 9309 governs that file, and the robots.txt tester checks it.

    Sources

    1. 1.Robots meta tag, data-nosnippet, and X-Robots-Tag specifications - Google Search CentralPrimary
    2. 2.RFC 9309: Robots Exclusion Protocol - IETF

    Frequently asked questions

    Which robots meta tag values does Google support?

    The validator accepts the rules on Google's specification page: all, noindex, nofollow, none, nosnippet, indexifembedded, notranslate and noimageindex, which take no value, plus max-snippet, max-image-preview, max-video-preview and unavailable_after, which do. Anything else is flagged as a rule Google does not document.

    What happens when robots rules conflict?

    Google applies the more restrictive rule. Its own example is a page with both max-snippet:50 and nosnippet, where nosnippet applies. The combined result in this tool follows the same logic for noindex, nofollow and nosnippet.

    Should I use a meta tag or an X-Robots-Tag header?

    Either works for HTML pages. For files with no HTML, such as PDFs or images, the X-Robots-Tag header is the only option, because there is nowhere to put a meta tag.

    Why is Google ignoring my noindex?

    The most common cause is a robots.txt Disallow on the same URL. Google says that if a page is disallowed from crawling, its indexing rules will not be found and are ignored. Remove the Disallow so Googlebot can crawl the page and read the noindex.

    Is noarchive still supported?

    Google says noarchive is no longer used to control whether a cached link is shown, because that feature is gone. It does no harm, but it does nothing in Google Search, so the validator marks it with a warning.

    Tools

    Hreflang XML sitemap generator

    Hreflang sitemap generator: turn language codes and URLs into sitemap XML with xhtml:link alternates, and check codes and URLs. Free, runs in your browser.