A randomized field experiment on AI Overviews and organic clicks

Most of what the industry knows about AI Overviews and click-through rate comes from watching traffic before and after a rollout, or comparing accounts that happen to have AI Overview exposure against accounts that do not. Both designs leave room for a confound: something else, a seasonal shift, an unrelated algorithm update, a change in query mix, could be doing some of the work attributed to the AI Overview. A field experiment posted to SSRN in April 2026 by Saharsh Agarwal (Indian School of Business) and Ananya Sen (Carnegie Mellon University) took a different approach: randomize whether the same searcher sees an AI Overview at all, and measure what changes.

What the experiment actually did#

The authors built a Chrome extension and recruited 1,065 U.S. desktop Chrome users through Prolific, the same panel platform used across a lot of academic behavioral research. Over a two-week window per participant, spanning January 7 to February 10, 2026 in total, every search was randomly routed into one of three arms: a control group that saw Google Search exactly as Google serves it, a “Hide AIO” group where the extension detected and suppressed the AI Overview in real time, and a third arm redirected to Google’s separate AI Mode interface. The extension logged which URLs appeared, which ones got clicked, how far participants scrolled, and the structure of each results page, across 68,089 total searches.

Randomizing within the same population is the point. Because the same pool of searchers is split into groups that either do or do not see an AI Overview on a given query, differences between the groups are much harder to explain away as unrelated to the AI Overview itself. That is a different, stronger design than comparing one time period to another, which is what most publicly circulated AI Overview traffic figures rely on.

What it found#

Outbound organic clicks fell 39.8% in the condition where an AI Overview appeared, relative to the same kind of search with the AI Overview hidden. Zero-click searches, meaning the participant never left the results page at all, rose 34.5% in that same condition. Sponsored ad clicks and the raw number of searches each participant ran held steady across groups, which points at organic results specifically rather than a general drop in search engagement.

The effect was not spread evenly across the page. Roughly 88% of the total click-loss effect traced back to cases where the AI Overview sat at the very top of the results page, which Search Engine Journal’s reporting on the paper says happened in about 85% of the AI Overview appearances in the sample. Reporting on the same figures split slightly on the exact clicks-per-search baseline (Search Engine Journal cites 0.38 rising to 0.61; PPC Land cites 0.37 rising to 0.62), a small enough discrepancy that it likely reflects rounding in each outlet’s own write-up rather than a disagreement about the underlying 39.8% headline figure, which both report identically.

A satisfaction survey found no offsetting quality gain. Participants in both the control and Hide AIO groups rated their overall satisfaction, the information quality they found and how easy it was to find what they wanted, on a 5-point scale after their sessions. The authors report no statistically significant difference between the groups on any of those measures. That detail matters for how the click-loss number gets read: it is not paired, in this study, with searchers reporting they got what they needed faster.

Why we care#

If the 39.8% figure holds up under further replication, it is a materially different kind of evidence than the observational, before/after comparisons that populate most of the current AI Overview traffic-impact conversation, including in SEO Madman’s own comparison of seven quoted AI Overview traffic-decline figures. A randomized design does not, on its own, settle every open question. It describes one five-week window, one recruitment platform, one browser and one country. What it adds is a click-loss figure that is harder to attribute to something other than the AI Overview itself, which is the piece most observational studies on this topic cannot supply.