IETF's AI Preferences group is still drafting, not shipping

The IETF has a working group standardizing how a site tells AI systems what they may do with its content, and it has been at it for a year and a half without publishing a single RFC. The AI Preferences (AIPREF) working group was chartered on 7 January 2025, growing out of an IAB workshop on AI and content control and a side meeting at IETF 121 the previous November. As of September 2026 it has produced two Internet-Drafts. Neither is a finished standard.

What the charter actually asks for#

AIPREF’s charter describes the goal as standardizing “building blocks that allow for the expression of preferences about how content is collected and processed for Artificial Intelligence (AI) model development, deployment, and use.” It names Well-Known URIs under RFC 8615, specifically the Robots Exclusion Protocol of RFC 9309, and HTTP response header fields as candidate mechanisms for attaching those preferences to content. The charter frames these as delivery options the working group can build on, not as a problem statement about robots.txt itself, and the datatracker record does not contain language calling existing AI-vendor signals confusing or non-standard. That framing shows up in commentary around the group, not in the charter text.

The practical background is not controversial even without a quotable charter line: several AI companies already publish their own robots.txt user-agent tokens, GPTBot and Google-Extended among them, and site owners have had no single vocabulary for saying yes to search use while saying no to model training. AIPREF’s job is to give that distinction a standard shape. This is a separate effort from llms.txt, the informal file-at-the-root convention some blog posts have wrongly attributed to a W3C standards track; AIPREF never touches llms.txt and has no relationship to it.

The two drafts, and what they cover#

draft-ietf-aipref-vocab defines the preference vocabulary. As of version 07, dated 18 August 2026, it specifies two usage categories: train-ai, for using content to modify an AI model’s learned parameters, and search, for retrieval applications whose primary purpose is directing users back to the original source. Each category takes a simple allow or disallow value; categories the vocabulary does not yet cover remain unspecified rather than defaulting either way.

draft-ietf-aipref-attach, version 05, also dated 18 August 2026, defines how a preference gets attached to content: a new Content-Usage directive inside robots.txt, and a matching Content-Usage HTTP response header for sites that would rather signal at the response level than the file level. Both mechanisms are meant to coexist with RFC 9309 rather than supersede it.

Both drafts carry the standard Internet-Draft boilerplate stating they do not reflect working group consensus in whole or in part, and both are dated to expire in February 2027 if not revised or advanced. That is normal for an active draft, not a sign of stalling, but it does mean the vocabulary and the syntax can still change before either document reaches working group last call.

Why we care#

Nothing here changes what belongs in a robots.txt file today. The working group has not published an RFC, the vocabulary covers exactly two preference categories, and the attachment syntax is still draft text that IESG review could still alter. What’s worth tracking is the direction: if AIPREF finishes, sites get one IETF-sanctioned way to say “index and cite me, don’t train on me” instead of the patchwork of vendor-specific tokens crawlers currently interpret however they choose. Technical SEOs should watch draft-ietf-aipref-attach advance past working group last call before treating a Content-Usage directive as something to deploy, not before.