Generative AI

RAG chunking strategies, compared

Fixed-size, recursive, semantic and structure-aware chunking — what each one does to retrieval quality, and how to pick.

Marcus Reid·Principal Engineer·July 8, 2026·7 min read

Chunking is the highest-leverage decision in a retrieval system and the one most often left at its default. Get it wrong and no amount of prompt tuning downstream will save the answers.

Fixed-size: fast, and lossy in the worst places

Splitting every N tokens is trivial to implement and cuts straight through tables, clauses and definitions. It works acceptably on uniform prose and badly on anything with structure — which is most business content.

Recursive: a reasonable default

Splitting on progressively finer separators — sections, paragraphs, sentences — respects natural boundaries and degrades gracefully. If you need one default, this is it.

Structure-aware: what we reach for on documents that have structure

Contracts, policies, manuals and specifications carry their meaning in their hierarchy. Chunking along headings and clauses, and attaching the parent path as metadata, means a retrieved chunk arrives knowing where it came from — which makes both filtering and citation dramatically better.

Semantic: powerful, and worth the cost less often than you think

Splitting where embedding similarity drops finds topic boundaries the formatting does not mark. It costs an embedding pass over everything, and on well-structured documents it frequently loses to simply respecting the headings that are already there.

Chunking strategies at a glance

Fixed-size

Good for: Uniform prose, fastest to build
Falls over on: Tables, clauses, anything structured

Recursive

Good for: A sensible default for mixed content
Falls over on: Very long unbroken sections

Structure-aware

Good for: Contracts, policies, manuals
Falls over on: Documents with no real hierarchy

Semantic

Good for: Unstructured text with topic shifts
Falls over on: Cost, and well-formatted documents

	Good for	Falls over on
Fixed-size	Uniform prose, fastest to build	Tables, clauses, anything structured
Recursive	A sensible default for mixed content	Very long unbroken sections
Structure-aware	Contracts, policies, manuals	Documents with no real hierarchy
Semantic	Unstructured text with topic shifts	Cost, and well-formatted documents

The parts that matter more than the strategy

Whichever you choose, these move retrieval quality more than the split itself:

Overlap between adjacent chunks, so answers spanning a boundary survive
Metadata on every chunk — source, section path, date, access scope
Keeping tables intact rather than shredding them across chunks
Measuring retrieval separately from generation, so you know which half is failing

What moves retrieval quality most

Illustrative weighting. The split strategy matters less than what you attach to each chunk.