--- title: "AI citation hallucination rates: what three real studies found" url: https://seomadman.com/research/ai-citation-hallucination-rates section: research published: 2026-07-27T00:00:00.000Z modified: 2026-07-27T00:00:00.000Z author: Adam Hafez topics: ["AI search"] --- # AI citation hallucination rates: what three real studies found ## The short answer AI systems fabricate or mangle citations at rates that vary wildly by study: GPTZero's January 2026 audit found hallucinated citations in about 1% of NeurIPS 2025 papers, Tow Center found citation errors in over 60% of AI search engine answers about news, and a Cureus study found ChatGPT-5 fabricated 7% of clinical references outright. No single rate is correct. ## Key takeaways - GPTZero's January 2026 audit of 4,841 NeurIPS 2025 papers confirmed at least 100 hallucinated citations across 53 papers, about 1% of the total. - The Tow Center for Digital Journalism found more than 60% of 1,600 AI search engine answers about news articles contained a citation error. - A Cureus study found ChatGPT-5 fabricated 7.13% of the 2,736 clinical references it generated, and got every field right in only 8% of queries. - The three studies test different systems and different definitions of hallucination, so their numbers cannot be averaged into one rate. - Publishers and SEOs should treat any single quoted AI hallucination rate as one study's number, not a settled industry fact. Every few months a new number claims to answer "how often does AI make up citations." Depending on which blog you read, that number is 1%, 7%, 40%, or 82%. All four figures trace back to real research, but they are not measuring the same question, and treating them as interchangeable is the actual problem. Three studies here can be verified at the source with a stated sample size and a stated method. The rest, including several widely repeated ranges, could not be traced past a secondary aggregator restating another aggregator, so they are not used. ## Three studies, three different questions GPTZero, the plagiarism and AI-detection company, published a report on January 21, 2026 checking whether citations inside 4,841 papers accepted to NeurIPS 2025, a top peer-reviewed AI conference, actually exist and match what they claim to cite. That is a question about human-authored academic papers that happen to contain AI-generated reference lists, not about a chatbot answering a live question. The Tow Center for Digital Journalism, publishing through Columbia Journalism Review, ran a different test in March 2025: it asked eight AI search tools, ChatGPT Search, Gemini, Perplexity, Perplexity Pro, DeepSeek Search, Microsoft Copilot, and two versions of Grok, to find and cite 200 real news articles from 20 publishers, 1,600 queries in total. That is a question about whether a live AI search answer correctly identifies and links a specific, real source. A Cureus study published July 30, 2026 asked a third question: when ChatGPT-5 answers a clinical question and cites medical guidelines, how often is each individual bibliographic field, author, title, journal, page numbers, actually correct. Researchers checked all 2,736 references the model generated across 350 queries built from American Academy of Orthopaedic Surgeons guidelines. ## What each one actually found GPTZero's Hallucination Check tool flagged citations that could not be verified through academic databases and DOI or URL lookups, then had a human expert confirm each flagged case before counting it. That process confirmed at least 100 hallucinated citations spanning 53 distinct papers, about 1.1% of the papers checked, with teams from Google, Meta, Harvard and Cambridge among those affected. It is the most tightly defined and independently verifiable figure of the three: a fixed population of finished papers, a named detection tool, and a human review step before publication. The Tow Center study found more than 60% of the 1,600 responses contained a citation error, defined as an incorrect article identification, wrong publisher attribution, or a missing or broken URL. Accuracy varied enormously by tool: Perplexity's free tier had the fewest errors at 37%, while Grok 3 reached 94%. More than half of Gemini's and Grok 3's responses cited fabricated or broken links that led to error pages. The Cureus study found ChatGPT-5 fabricated 7.13% of its 2,736 generated references outright, a figure close to what search snippets described as "7 to 8% for ChatGPT-5 with web search," and this is the one place in circulation where that specific range does trace to a real, dated study. But fabrication was not the main source of error: only 49.34% of citations were fully correct across every bibliographic field, and all citations in a single query were completely accurate in just 8% of cases. PMID and title were the fields most often wrong. ## Why the numbers don't add up to one figure A companion piece of research, "Source or It Didn't Happen," a 2026 arXiv paper proposing a multi-agent detection system called CiteTracer, illustrates why these figures resist averaging even further. Its own benchmark reports 97.1% detection accuracy, but that number describes how well its detector classifies citations in a dataset it built itself, 2,450 synthetic mutated citations plus 957 known-fabricated real-world ones, not the underlying rate at which any AI system hallucinates citations in ordinary use. It is evidence that citation hallucination is being taken seriously as a detection problem, not a fourth data point for a headline percentage. Line the three real figures up and the range runs from about 1% to over 60% for the same broad topic. That is not inconsistency in the research, it is three different definitions of "hallucination" applied to three different tasks: a citation existing at all, a citation pointing at the right article, and a citation's individual fields being correct. A single AI answer could score well on one of those tests and badly on another at the same time. Any figure quoted without naming which of these three questions it answers, or a fourth like it, should be treated as unverifiable. ## Why we care For a publisher, the Tow Center number is the one that matters day to day: when a licensing deal or a partnership gets an AI search tool to name your outlet, that citation still has close to even odds of misattributing the quote, linking a syndicated copy instead of the original, or breaking outright. A byline or a URL appearing correctly inside a chatbot's answer is not proof the system is citing reliably elsewhere, and a partnership does not fix that on its own. For an SEO evaluating an AI-citation-tracking tool, the GPTZero and Cureus studies are the more useful reference points, because they show that "verified" and "hallucinated" only mean something once the checker's method is stated. A tool that reports a single hallucination percentage without saying whether it is checking existence, attribution, or field-level accuracy is doing what the aggregator blogs did: collapsing three different measurements into one number that sounds more precise than it is. Ask which of the three questions above a tool actually answers before trusting its score. ## Sources 1. [GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers](https://gptzero.me/news/neurips/) - GPTZero (primary) 2. [AI Search Has a Citation Problem](https://www.cjr.org/tow_center/we-compared-eight-ai-search-engines-theyre-all-bad-at-citing-news.php) - Columbia Journalism Review, Tow Center for Digital Journalism (primary) 3. [Evaluating the Citation Accuracy of ChatGPT-5 Using the American Academy of Orthopaedic Surgeons Clinical Practice Guidelines](https://pmc.ncbi.nlm.nih.gov/articles/PMC13525731/) - Cureus (primary) 4. [Source or It Didn't Happen: A Multi-Agent Framework for Citation Hallucination Detection](https://arxiv.org/abs/2605.08583) - arXiv