--- title: "A randomized field experiment on AI Overviews and organic clicks" url: https://seomadman.com/research/ai-overview-ctr-field-experiment section: research published: 2026-09-16T00:00:00.000Z modified: 2026-09-16T00:00:00.000Z author: Adam Hafez topics: ["AI search"] --- # A randomized field experiment on AI Overviews and organic clicks ## The short answer Agarwal and Sen's field experiment randomly assigned 1,065 Prolific participants to see or not see Google AI Overviews across 68,089 real searches in January and February 2026. Outbound organic clicks fell 39.8% when an AI Overview appeared, zero-click searches rose 34.5%, and a companion survey found no measurable gain in perceived search quality. ## Key takeaways - The first randomized (not observational) field experiment on AI Overviews found outbound organic clicks fell 39.8% when an AI Overview appeared on the results page. - Zero-click searches rose 34.5% in the AI Overview condition, while sponsored ad clicks and overall search frequency stayed statistically unchanged. - Effect size was not uniform: roughly 88% of the click loss was concentrated in cases where the AI Overview occupied the very top of the page. - A companion satisfaction survey found no significant difference in perceived information quality between the AI Overview group and the group with it hidden. - The study used a custom Chrome extension to randomize exposure per search, a design that avoids the before/after confound of comparing traffic across a rollout date. Most of what the industry knows about AI Overviews and click-through rate comes from watching traffic before and after a rollout, or comparing accounts that happen to have AI Overview exposure against accounts that do not. Both designs leave room for a confound: something else, a seasonal shift, an unrelated algorithm update, a change in query mix, could be doing some of the work attributed to the AI Overview. A field experiment posted to SSRN in April 2026 by Saharsh Agarwal (Indian School of Business) and Ananya Sen (Carnegie Mellon University) took a different approach: randomize whether the same searcher sees an AI Overview at all, and measure what changes. ## What the experiment actually did The authors built a Chrome extension and recruited 1,065 U.S. desktop Chrome users through Prolific, the same panel platform used across a lot of academic behavioral research. Over a two-week window per participant, spanning January 7 to February 10, 2026 in total, every search was randomly routed into one of three arms: a control group that saw Google Search exactly as Google serves it, a "Hide AIO" group where the extension detected and suppressed the AI Overview in real time, and a third arm redirected to Google's separate AI Mode interface. The extension logged which URLs appeared, which ones got clicked, how far participants scrolled, and the structure of each results page, across 68,089 total searches. **Randomizing within the same population is the point.** Because the same pool of searchers is split into groups that either do or do not see an AI Overview on a given query, differences between the groups are much harder to explain away as unrelated to the AI Overview itself. That is a different, stronger design than comparing one time period to another, which is what most publicly circulated AI Overview traffic figures rely on. ## What it found Outbound organic clicks fell 39.8% in the condition where an AI Overview appeared, relative to the same kind of search with the AI Overview hidden. Zero-click searches, meaning the participant never left the results page at all, rose 34.5% in that same condition. Sponsored ad clicks and the raw number of searches each participant ran held steady across groups, which points at organic results specifically rather than a general drop in search engagement. **The effect was not spread evenly across the page.** Roughly 88% of the total click-loss effect traced back to cases where the AI Overview sat at the very top of the results page, which Search Engine Journal's reporting on the paper says happened in about 85% of the AI Overview appearances in the sample. Reporting on the same figures split slightly on the exact clicks-per-search baseline (Search Engine Journal cites 0.38 rising to 0.61; PPC Land cites 0.37 rising to 0.62), a small enough discrepancy that it likely reflects rounding in each outlet's own write-up rather than a disagreement about the underlying 39.8% headline figure, which both report identically. **A satisfaction survey found no offsetting quality gain.** Participants in both the control and Hide AIO groups rated their overall satisfaction, the information quality they found and how easy it was to find what they wanted, on a 5-point scale after their sessions. The authors report no statistically significant difference between the groups on any of those measures. That detail matters for how the click-loss number gets read: it is not paired, in this study, with searchers reporting they got what they needed faster. ## Why we care If the 39.8% figure holds up under further replication, it is a materially different kind of evidence than the observational, before/after comparisons that populate most of the current AI Overview traffic-impact conversation, including in [SEO Madman's own comparison of seven quoted AI Overview traffic-decline figures](/reports/ai-overview-traffic-decline-estimates-compared). A randomized design does not, on its own, settle every open question. It describes one five-week window, one recruitment platform, one browser and one country. What it adds is a click-loss figure that is harder to attribute to something other than the AI Overview itself, which is the piece most observational studies on this topic cannot supply. ## Frequently asked questions ### How is this different from the other AI Overview traffic studies SEO Madman has covered? Most published figures on AI Overview impact, including several this site has already written up, come from observational data - comparing a site's or an industry's traffic before and after AI Overviews rolled out. This study instead randomly hides the AI Overview for some real searches and shows it for others in the same time window, which removes the risk that an unrelated seasonal or algorithmic shift gets mistaken for the AI Overview's effect. ### Does a 39.8% click drop mean AI Overviews cut all publisher traffic by 39.8%? No. The figure applies specifically to searches where an AI Overview actually appeared, which the study says was roughly 41 to 42% of queries in its sample, and it measures outbound organic clicks in a randomized Prolific panel, not aggregate traffic for any specific site or industry. ### Why does this piece rely on secondary reporting instead of just quoting the SSRN paper directly? SSRN's paper page served an automated bot-detection challenge that this piece's research process could not clear. Rather than guess at the paper's exact wording, the figures here are cross-checked against two independent trade outlets that reported on the paper directly and agree on the core numbers; the SSRN DOI is linked as the primary source for readers who want the paper itself. ## Sources 1. [The Impact of Google AI Overviews on Publisher Traffic and User Experience: Evidence from a Field Experiment](https://doi.org/10.2139/ssrn.6513059) - SSRN (Saharsh Agarwal, Indian School of Business; Ananya Sen, Carnegie Mellon University) (primary) 2. [Study Confirms Google AI Overviews Cut Organic Clicks 38%](https://www.searchenginejournal.com/ai-overviews-cut-organic-clicks-38-field-study-finds/573145/) - Search Engine Journal 3. [Researchers find Google AI Overviews cut publisher clicks 39.8%](https://ppc.land/researchers-find-google-ai-overviews-cut-publisher-clicks-39-8/) - PPC Land