chunking-strategies

What this skill does

# Chunking Strategies

## Strategy Tradeoff Matrix

| Strategy | Preserves Semantics | Cost | Best For | Chunk Unit |
|---|---|---|---|---|
| Fixed-size | No | Free | Homogeneous prose | Chars |
| Recursive character | Partial | Free | General text | Chars with hierarchy |
| Token-based | No | Free | Exact token budgeting | Tokens |
| Document-aware (Markdown/HTML) | Yes | Free | Technical docs, wikis | Headers/sections |
| Code-aware | Yes | Free | Source code | Functions/classes |
| Semantic (embedding breakpoints) | Yes | $$ | Long narrative, research papers | Meaning shifts |
| Proposition-based | Yes | $$$ | High-precision Q&A, legal | Atomic facts |
| Parent-child | Yes | Free | Need small match + big context | Hierarchy |
| Sliding window | Partial | Free | Dialogues, timelines | Overlap + stride |
| Contextual retrieval (Anthropic) | Yes | $$ | Production RAG > 5k chunks | Chunk + LLM context |

## Fixed-Size

```python
def fixed_chunks(text: str, size: int = 800, overlap: int = 200) -> list[str]:
    return [text[i:i + size] for i in range(0, len(text), size - overlap)]
```

Use only as a baseline. Breaks mid-sentence and mid-token.

## Recursive Character (default for most projects)

```python
from langchain_text_splitters import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=800,
    chunk_overlap=200,
    separators=["\n\n", "\n", ". ", "? ", "! ", " ", ""],
    length_function=len,
    is_separator_regex=False,
)
chunks = splitter.split_documents(docs)
```

Tries separators in order so paragraph boundaries are preferred over arbitrary cuts.

## Token-Based (exact LLM budgeting)

```python
from langchain_text_splitters import TokenTextSplitter

splitter = TokenTextSplitter(
    encoding_name="cl100k_base",  # GPT-4, text-embedding-3
    chunk_size=512,
    chunk_overlap=64,
)
chunks = splitter.split_text(text)
```

For Claude, use `anthropic.Anthropic().messages.count_tokens` to measure; approximate ratio is ~3.5 chars per token for English.

## Document-Aware: Markdown

```python
from langchain_text_splitters import MarkdownHeaderTextSplitter, RecursiveCharacterTextSplitter

headers = [("#", "h1"), ("##", "h2"), ("###", "h3")]
md_splitter = MarkdownHeaderTextSplitter(headers_to_split_on=headers, strip_headers=False)
header_chunks = md_splitter.split_text(markdown_text)

# Secondary splitter for oversized sections
char_splitter = RecursiveCharacterTextSplitter(chunk_size=800, chunk_overlap=150)
chunks = char_splitter.split_documents(header_chunks)  # metadata carries h1/h2/h3
```

Header metadata enables section-level filtering at query time.

## Document-Aware: Code

```python
from langchain_text_splitters import RecursiveCharacterTextSplitter, Language

py_splitter = RecursiveCharacterTextSplitter.from_language(
    language=Language.PYTHON, chunk_size=1500, chunk_overlap=200
)
ts_splitter = RecursiveCharacterTextSplitter.from_language(
    language=Language.TS, chunk_size=1500, chunk_overlap=200
)
```

Splits on `class`, `def`, `function` boundaries. Larger chunks because code lines are shorter than prose.

## Semantic Chunking (embedding breakpoints)

Splits where adjacent sentences diverge in meaning. Expensive (embeds every sentence) but yields coherent chunks on long-form content.

```python
from langchain_experimental.text_splitter import SemanticChunker
from langchain_openai import OpenAIEmbeddings

splitter = SemanticChunker(
    OpenAIEmbeddings(model="text-embedding-3-small"),
    breakpoint_threshold_type="percentile",   # or "standard_deviation", "interquartile"
    breakpoint_threshold_amount=95,
    buffer_size=1,
)
chunks = splitter.create_documents([long_text])
```

Manual variant for fine control (break where cosine distance between adjacent sentence embeddings exceeds the 95th percentile):

```python
import numpy as np
from sentence_transformers import SentenceTransformer

def semantic_chunks(text: str, pct: float = 95) -> list[str]:
    sents = [s.strip() for s in text.split(". ") if s.strip()]
    embs = SentenceTransformer("all-MiniLM-L6-v2").encode(sents)
    sims = [np.dot(embs[i], embs[i+1]) / (np.linalg.norm(embs[i]) * np.linalg.norm(embs[i+1]))
            for i in range(len(sents) - 1)]
    dists = 1 - np.array(sims)
    breaks = [i + 1 for i, d in enumerate(dists) if d > np.percentile(dists, pct)]
    out, start = [], 0
    for b in breaks + [len(sents)]:
        out.append(". ".join(sents[start:b])); start = b
    return out
```

## Proposition-Based Chunking

Decompose text into atomic factual propositions using an LLM. Each proposition becomes one chunk.

```python
from anthropic import Anthropic
import json

client = Anthropic()

PROMPT = """Decompose the passage into atomic propositions. Each proposition:
- Expresses exactly one fact
- Is self-contained (no pronouns without antecedents)
- Resolves coreferences inline

Return JSON array of strings. Passage:
{passage}"""

def propositionize(passage: str) -> list[str]:
    msg = client.messages.create(
        model="claude-sonnet-4-5-20250929",
        max_tokens=2048,
        messages=[{"role": "user", "content": PROMPT.format(passage=passage)}],
    )
    return json.loads(msg.content[0].text)
```

Highest retrieval precision; highest ingestion cost. Use for legal, medical, compliance.

## Parent-Child Chunking

Index small chunks (for precise retrieval) but return parent chunks (for context).

```python
from langchain.retrievers import ParentDocumentRetriever
from langchain.storage import InMemoryStore
from langchain_chroma import Chroma
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings

parent_splitter = RecursiveCharacterTextSplitter(chunk_size=2000)
child_splitter = RecursiveCharacterTextSplitter(chunk_size=400)
vectorstore = Chroma(collection_name="children", embedding_function=OpenAIEmbeddings())
docstore = InMemoryStore()

retriever = ParentDocumentRetriever(
    vectorstore=vectorstore,
    docstore=docstore,
    child_splitter=child_splitter,
    parent_splitter=parent_splitter,
)
retriever.add_documents(docs)
```

See also `advanced-retrieval` for multi-vector and sentence-window patterns that generalize this idea.

## Sliding Window with Stride

```python
def sliding_window(tokens: list[str], window: int = 512, stride: int = 256) -> list[list[str]]:
    return [tokens[i:i + window] for i in range(0, len(tokens), stride) if i + window <= len(tokens)]
```

Use for dialogue, legal contracts, timelines where context before and after each point matters.

## Anthropic Contextual Retrieval

Prepend LLM-generated context to each chunk before embedding. Reduces retrieval failure rate by ~35% on Anthropic's benchmark.

```python
from anthropic import Anthropic

client = Anthropic()

CONTEXT_PROMPT = """<document>
{whole_document}
</document>

Here is the chunk we want to situate within the whole document:
<chunk>
{chunk}
</chunk>

Give a short (1-2 sentence) context to situate this chunk within the overall
document for the purposes of improving search retrieval of the chunk.
Answer only with the succinct context and nothing else."""

def contextualize(whole_doc: str, chunk: str) -> str:
    msg = client.messages.create(
        model="claude-haiku-4-5-20250929",
        max_tokens=200,
        messages=[{"role": "user", "content": CONTEXT_PROMPT.format(
            whole_document=whole_doc, chunk=chunk)}],
        extra_headers={"anthropic-beta": "prompt-caching-2024-07-31"},
    )
    return msg.content[0].text

def contextual_chunks(doc: str, base_chunks: list[str]) -> list[str]:
    return [f"{contextualize(doc, c)}\n\n{c}" for c in base_chunks]
```

Use prompt caching on `whole_document` to drop cost by ~10x. Combine with BM25 + dense for best results.

## LlamaIndex Equivalents

```python
from llama_index.core.node_parser import (
    SentenceSplitter, SemanticSplitterNodeParser, MarkdownNodeParser,
    HierarchicalNodeParser,
)
from llama_i