AI citation hallucination rates: what three real studies found

Every few months a new number claims to answer “how often does AI make up citations.” Depending on which blog you read, that number is 1%, 7%, 40%, or 82%. All four figures trace back to real research, but they are not measuring the same question, and treating them as interchangeable is the actual problem. Three studies here can be verified at the source with a stated sample size and a stated method. The rest, including several widely repeated ranges, could not be traced past a secondary aggregator restating another aggregator, so they are not used.

Three studies, three different questions#

GPTZero, the plagiarism and AI-detection company, published a report on January 21, 2026 checking whether citations inside 4,841 papers accepted to NeurIPS 2025, a top peer-reviewed AI conference, actually exist and match what they claim to cite. That is a question about human-authored academic papers that happen to contain AI-generated reference lists, not about a chatbot answering a live question.

The Tow Center for Digital Journalism, publishing through Columbia Journalism Review, ran a different test in March 2025: it asked eight AI search tools, ChatGPT Search, Gemini, Perplexity, Perplexity Pro, DeepSeek Search, Microsoft Copilot, and two versions of Grok, to find and cite 200 real news articles from 20 publishers, 1,600 queries in total. That is a question about whether a live AI search answer correctly identifies and links a specific, real source.

A Cureus study published July 30, 2026 asked a third question: when ChatGPT-5 answers a clinical question and cites medical guidelines, how often is each individual bibliographic field, author, title, journal, page numbers, actually correct. Researchers checked all 2,736 references the model generated across 350 queries built from American Academy of Orthopaedic Surgeons guidelines.

What each one actually found#

GPTZero’s Hallucination Check tool flagged citations that could not be verified through academic databases and DOI or URL lookups, then had a human expert confirm each flagged case before counting it. That process confirmed at least 100 hallucinated citations spanning 53 distinct papers, about 1.1% of the papers checked, with teams from Google, Meta, Harvard and Cambridge among those affected. It is the most tightly defined and independently verifiable figure of the three: a fixed population of finished papers, a named detection tool, and a human review step before publication.

The Tow Center study found more than 60% of the 1,600 responses contained a citation error, defined as an incorrect article identification, wrong publisher attribution, or a missing or broken URL. Accuracy varied enormously by tool: Perplexity’s free tier had the fewest errors at 37%, while Grok 3 reached 94%. More than half of Gemini’s and Grok 3’s responses cited fabricated or broken links that led to error pages.

The Cureus study found ChatGPT-5 fabricated 7.13% of its 2,736 generated references outright, a figure close to what search snippets described as “7 to 8% for ChatGPT-5 with web search,” and this is the one place in circulation where that specific range does trace to a real, dated study. But fabrication was not the main source of error: only 49.34% of citations were fully correct across every bibliographic field, and all citations in a single query were completely accurate in just 8% of cases. PMID and title were the fields most often wrong.

Why the numbers don’t add up to one figure#

A companion piece of research, “Source or It Didn’t Happen,” a 2026 arXiv paper proposing a multi-agent detection system called CiteTracer, illustrates why these figures resist averaging even further. Its own benchmark reports 97.1% detection accuracy, but that number describes how well its detector classifies citations in a dataset it built itself, 2,450 synthetic mutated citations plus 957 known-fabricated real-world ones, not the underlying rate at which any AI system hallucinates citations in ordinary use. It is evidence that citation hallucination is being taken seriously as a detection problem, not a fourth data point for a headline percentage.

Line the three real figures up and the range runs from about 1% to over 60% for the same broad topic. That is not inconsistency in the research, it is three different definitions of “hallucination” applied to three different tasks: a citation existing at all, a citation pointing at the right article, and a citation’s individual fields being correct. A single AI answer could score well on one of those tests and badly on another at the same time. Any figure quoted without naming which of these three questions it answers, or a fourth like it, should be treated as unverifiable.

Why we care#

For a publisher, the Tow Center number is the one that matters day to day: when a licensing deal or a partnership gets an AI search tool to name your outlet, that citation still has close to even odds of misattributing the quote, linking a syndicated copy instead of the original, or breaking outright. A byline or a URL appearing correctly inside a chatbot’s answer is not proof the system is citing reliably elsewhere, and a partnership does not fix that on its own.

For an SEO evaluating an AI-citation-tracking tool, the GPTZero and Cureus studies are the more useful reference points, because they show that “verified” and “hallucinated” only mean something once the checker’s method is stated. A tool that reports a single hallucination percentage without saying whether it is checking existence, attribution, or field-level accuracy is doing what the aggregator blogs did: collapsing three different measurements into one number that sounds more precise than it is. Ask which of the three questions above a tool actually answers before trusting its score.