How you chunk a page changes whether AI retrieves it at all

Every AI search engine and every retrieval-augmented chatbot has to do the same thing before it can answer a question from a web page: cut that page into pieces small enough to embed and compare against the query. How those pieces get cut is rarely discussed outside RAG engineering circles, but a March 2026 academic benchmark measured it directly, and the gap between doing it well and doing it the default way is large.

What the study actually tested#

Muhammad Arslan Shaukat, Muntasir Adnan and Carlos C. N. Kuhn built what they describe as the first large-scale, cross-domain evaluation of document chunking strategies for dense retrieval. Instead of proposing one new method and comparing it to a couple of baselines, they benchmarked 36 chunking strategies against each other, grouped into fixed-size, semantic, structure-aware, hierarchical, adaptive and LLM-assisted families. Every strategy ran against the same UltraDomain dataset, spanning biology, physics, health, legal, maths and agriculture content, and every resulting set of chunks was embedded with five different embedding models so that chunking quality could be separated from embedding-model quality. Retrieval accuracy was scored with Normalised DCG at 5, the primary metric, backed by Hit@5 and MRR, and relevance was judged by a large language model grader rather than exact keyword overlap.

The gap between good and naive chunking#

The headline number is stark. Paragraph Group Chunking, which groups whole paragraphs into retrieval units rather than cutting at a fixed character count, reached a mean nDCG@5 of about 0.459 and a Hit@5 near 59%, meaning the correct chunk showed up somewhere in the top five results close to six times out of ten. Naive fixed-size character chunking, the default behavior of many scraping and indexing tools that simply cut text every N characters with no regard for sentence or paragraph boundaries, scored below 0.244 on the same metric, and its Precision@1, the chance the single best chunk was ranked first, sat at only 2 to 3%. That is not a marginal difference; it is close to an order of magnitude on the metric that matters most for a system that only surfaces one or two citations.

The study also found that no single method wins everywhere. Dynamic token sizing, which adjusts chunk length based on content density, performed best in biology, physics and health content, while paragraph grouping performed best in legal and mathematics content. The authors attribute this to how differently those domains are written: technical scientific prose and dense legal argument break into coherent units at different natural lengths.

Bigger models do not fix bad chunking#

One finding worth sitting with: larger embedding models produced higher absolute retrieval scores across every chunking strategy tested, but they remained sensitive to poor chunking. A better embedding model did not close the gap between a good chunking strategy and a bad one; it moved both up in parallel. The authors describe chunking quality and embedding size as complementary rather than substitutable levers, which means a site relying on a state-of-the-art embedding model downstream still loses retrieval accuracy if its own content is split badly upstream, whether that splitting happens in a scraper’s pipeline or in how the page itself is structured into sections.

Why we care#

This study measures a retrieval pipeline the researchers built, not Google’s, OpenAI’s or Perplexity’s live indexing systems, and its numbers should not be read as “add paragraph breaks, get cited twice as often.” What it does establish, with a controlled 36-method comparison rather than a single case study, is that chunking is a real and measurable variable in whether the right passage gets surfaced for a query at all, not a cosmetic detail. For a page meant to be read by a retrieval system as much as by a person, that argues for content organized into self-contained paragraphs and sections that make sense pulled out on their own, since every chunking method the paper tested that respected natural content boundaries beat the ones that ignored them.

What is not yet known#

The paper does not test HTML heading tags, <article> or <section> elements, or any other markup-level signal directly; its structure-aware and hierarchical methods work on paragraph and sentence boundaries in the underlying text, not on document markup. Whether marking up a page with proper h2 and h3 hierarchy produces the same retrieval gains this study measured for well-formed paragraphs remains an open question that would need its own study, and nothing here should be read as an answer to it.