--- title: "IETF's AI Preferences group is still drafting, not shipping" url: https://seomadman.com/news/ietf-aipref-robots-txt-successor section: news published: 2026-06-18T00:00:00.000Z modified: 2026-06-18T00:00:00.000Z author: Adam Hafez topics: ["Technical SEO", "AI search"] --- # IETF's AI Preferences group is still drafting, not shipping ## The short answer The IETF chartered an AI Preferences (AIPREF) working group on 7 January 2025 to standardize how sites express preferences about AI training and use. As of September 2026 it has two Internet-Drafts, not published RFCs, covering a vocabulary of preference terms and how to attach them through robots.txt or an HTTP header. Nothing here changes today's robots.txt configuration. ## Key takeaways - The IETF's AI Preferences working group was chartered on 7 January 2025, following an IAB workshop on AI-Control and a side meeting at IETF 121. - The charter names the Robots Exclusion Protocol, RFC 9309, and HTTP response headers as candidate delivery mechanisms rather than proposing to replace either. - As of September 2026 the group has produced two Internet-Drafts, not published RFCs, a vocabulary draft and an attachment draft, both on the Proposed Standard track. - The vocabulary draft currently defines two preference categories, train-ai and search, each expressed as a simple allow or disallow value. - The attachment draft proposes a Content-Usage directive in robots.txt alongside a matching Content-Usage HTTP response header, not a replacement file. The IETF has a working group standardizing how a site tells AI systems what they may do with its content, and it has been at it for a year and a half without publishing a single RFC. The AI Preferences (AIPREF) working group was chartered on 7 January 2025, growing out of an IAB workshop on AI and content control and a side meeting at IETF 121 the previous November. As of September 2026 it has produced two Internet-Drafts. Neither is a finished standard. ## What the charter actually asks for AIPREF's charter describes the goal as standardizing "building blocks that allow for the expression of preferences about how content is collected and processed for Artificial Intelligence (AI) model development, deployment, and use." It names Well-Known URIs under RFC 8615, specifically the Robots Exclusion Protocol of RFC 9309, and HTTP response header fields as candidate mechanisms for attaching those preferences to content. The charter frames these as delivery options the working group can build on, not as a problem statement about robots.txt itself, and the datatracker record does not contain language calling existing AI-vendor signals confusing or non-standard. That framing shows up in commentary around the group, not in the charter text. The practical background is not controversial even without a quotable charter line: several AI companies already publish their own robots.txt user-agent tokens, GPTBot and Google-Extended among them, and site owners have had no single vocabulary for saying yes to search use while saying no to model training. AIPREF's job is to give that distinction a standard shape. This is a separate effort from llms.txt, the informal file-at-the-root convention some blog posts have wrongly attributed to a W3C standards track; AIPREF never touches llms.txt and has no relationship to it. ## The two drafts, and what they cover **draft-ietf-aipref-vocab** defines the preference vocabulary. As of version 07, dated 18 August 2026, it specifies two usage categories: train-ai, for using content to modify an AI model's learned parameters, and search, for retrieval applications whose primary purpose is directing users back to the original source. Each category takes a simple allow or disallow value; categories the vocabulary does not yet cover remain unspecified rather than defaulting either way. **draft-ietf-aipref-attach**, version 05, also dated 18 August 2026, defines how a preference gets attached to content: a new Content-Usage directive inside robots.txt, and a matching Content-Usage HTTP response header for sites that would rather signal at the response level than the file level. Both mechanisms are meant to coexist with RFC 9309 rather than supersede it. Both drafts carry the standard Internet-Draft boilerplate stating they do not reflect working group consensus in whole or in part, and both are dated to expire in February 2027 if not revised or advanced. That is normal for an active draft, not a sign of stalling, but it does mean the vocabulary and the syntax can still change before either document reaches working group last call. ## Why we care Nothing here changes what belongs in a robots.txt file today. The working group has not published an RFC, the vocabulary covers exactly two preference categories, and the attachment syntax is still draft text that IESG review could still alter. What's worth tracking is the direction: if AIPREF finishes, sites get one IETF-sanctioned way to say "index and cite me, don't train on me" instead of the patchwork of vendor-specific tokens crawlers currently interpret however they choose. Technical SEOs should watch draft-ietf-aipref-attach advance past working group last call before treating a Content-Usage directive as something to deploy, not before. ## Frequently asked questions ### Does this replace robots.txt? No. The attachment draft adds a new Content-Usage directive that would sit inside an existing robots.txt file, alongside a matching HTTP response header for sites that prefer that route. RFC 9309 stays the base protocol either way. ### Can I use AIPREF today? Not as a ratified standard. Both documents are Internet-Drafts, meaning they can still change before working group last call, IESG review and publication as RFCs. Nothing in this effort is a shipped feature any crawler is contractually bound to honor yet. ### Who is actually building this? The IETF AI Preferences working group, chaired by Suresh Krishnan and Mark Nottingham, with liaison relationships to the IPTC, the PLUS Coalition, and WHATWG or W3C for related content-format work. ## Sources 1. [AI Preferences (aipref) - Working Group](https://datatracker.ietf.org/wg/aipref/about/) - IETF Datatracker (primary) 2. [Charter for Working Group: AI Preferences (aipref)](https://datatracker.ietf.org/doc/charter-ietf-aipref/) - IETF Datatracker (primary) 3. [A Vocabulary For Expressing AI Usage Preferences](https://datatracker.ietf.org/doc/draft-ietf-aipref-vocab/) - IETF Datatracker 4. [Associating AI Usage Preferences with Content in HTTP](https://datatracker.ietf.org/doc/draft-ietf-aipref-attach/) - IETF Datatracker 5. [RFC 9309: Robots Exclusion Protocol](https://www.rfc-editor.org/rfc/rfc9309) - RFC Editor