rag-agent-builder

What this skill does

# RAG Agent Builder

Build powerful Retrieval-Augmented Generation (RAG) applications that enhance LLM capabilities with external knowledge sources, enabling accurate, contextualized AI responses.

## Quick Start

Get started with RAG implementations in the examples and utilities:

- **Examples**: See [`examples/`](examples/) directory for complete implementations:
  - [`basic_rag.py`](examples/basic_rag.py) - Simple chunk-embed-retrieve-generate pipeline
  - [`retrieval_strategies.py`](examples/retrieval_strategies.py) - Hybrid search, reranking, and filtering
  - [`agentic_rag.py`](examples/agentic_rag.py) - Agent-controlled retrieval with iterative refinement

- **Utilities**: See [`scripts/`](scripts/) directory for helper modules:
  - [`embedding_management.py`](scripts/embedding_management.py) - Embedding generation, normalization, and caching
  - [`vector_db_manager.py`](scripts/vector_db_manager.py) - Vector database abstraction and factory
  - [`rag_evaluation.py`](scripts/rag_evaluation.py) - Retrieval and answer quality metrics

## Overview

RAG systems combine three key components:
1. **Document Retrieval** - Find relevant information from knowledge bases
2. **Context Integration** - Pass retrieved context to the LLM
3. **Response Generation** - Generate answers grounded in the retrieved information

This skill covers building production-ready RAG applications with various frameworks and approaches.

## Core Concepts

### What is RAG?

RAG augments LLM knowledge with external data:
- **Without RAG**: LLM relies on training data (may be outdated or limited)
- **With RAG**: LLM uses real-time, custom knowledge + training knowledge

### When to Use RAG

- **Document Q&A**: Answer questions about PDFs, books, reports
- **Knowledge Base Search**: Query internal documentation, wikis
- **Enterprise Search**: Search proprietary company data
- **Context-Specific Assistants**: Customer support, HR assistants
- **Fact-Heavy Applications**: Legal docs, medical records, financial data

### When RAG Might Not Be Needed

- General knowledge questions (ChatGPT-like)
- Real-time data that changes constantly (use tools instead)
- Very simple lookup tasks (use database queries)

## Architecture Patterns

### Basic RAG Pipeline

```
Documents → Chunks → Embeddings → Vector DB
                                        ↓
User Question → Embedding → Retrieval → LLM → Answer
                              ↑         ↓
                         Vector DB    Context
```

### Advanced RAG Patterns

#### 1. Agentic RAG
- Agent decides what to retrieve and when
- Can refine queries iteratively
- Better for complex reasoning

#### 2. Hierarchical RAG
- Multi-level document structure
- Search at different levels of detail
- More flexible organization

#### 3. Hybrid Search RAG
- Combines keyword search (BM25) + semantic search (embeddings)
- Captures both exact matches and meaning
- Better for mixed query types

#### 4. Corrective RAG (CRAG)
- Evaluates retrieved documents for relevance
- Retrieves additional sources if needed
- Ensures high-quality context

## Implementation Components

### 1. Document Processing

**Chunking Strategies**:
```python
# Simple fixed-size chunks
chunks = split_text(doc, chunk_size=1000, overlap=100)

# Semantic chunks (group by meaning)
chunks = semantic_chunking(doc, max_tokens=512)

# Hierarchical chunks (different levels)
chapters = split_by_heading(doc)
chunks = split_each_chapter(chapters, size=1000)
```

**Key Considerations**:
- Chunk size affects retrieval quality and cost
- Overlap helps maintain context between chunks
- Semantic chunking preserves meaning better

### 2. Embedding Generation

**Popular Embedding Models**:
- OpenAI: `text-embedding-3-small`, `text-embedding-3-large`
- Open Source: `all-MiniLM-L6-v2`, `all-mpnet-base-v2`
- Domain-Specific: Domain-trained embeddings for specialized knowledge

**Best Practices**:
- Use consistent embedding model for retrieval and queries
- Store embeddings with normalized vectors
- Update embeddings when documents change

### 3. Vector Databases

**Popular Options**:
- **Pinecone**: Managed, serverless, easy to scale
- **Weaviate**: Open-source, self-hosted, flexible
- **Milvus**: Open-source, high performance
- **Chroma**: Lightweight, good for prototypes
- **Qdrant**: Production-grade, high-performance

**Selection Criteria**:
- Scale requirements (data volume, queries per second)
- Latency needs (real-time vs batch)
- Cost considerations
- Deployment preferences (managed vs self-hosted)

### 4. Retrieval Strategies

**Retrieval Methods**:
```python
# Similarity search (most common)
results = vector_db.query(question_embedding, k=5)

# Hybrid search (keyword + semantic)
keyword_results = bm25.search(question, k=3)
semantic_results = vector_db.query(embedding, k=3)
results = combine_and_rank(keyword_results, semantic_results)

# Reranking (improve relevance)
retrieved = initial_retrieval(query)
reranked = rerank_by_relevance(retrieved, query)
```

**Retrieval Parameters**:
- **k** (number of results): Balance between context and relevance
- **Similarity threshold**: Filter out low-relevance results
- **Diversity**: Return varied results vs best matches

### 5. Context Integration

**Context Window Management**:
```python
# Fit retrieved documents into context window
def prepare_context(retrieved_docs, max_tokens=3000):
    context = ""
    for doc in retrieved_docs:
        if len(tokenize(context + doc)) <= max_tokens:
            context += doc
        else:
            break
    return context
```

**Prompt Design**:
```
You are a helpful assistant. Answer the question based on the provided context.

Context:
{retrieved_documents}

Question: {user_question}

Answer:
```

### 6. Response Generation

**Generation Strategies**:
- **Direct Generation**: LLM answers from context
- **Summarization**: Summarize multiple retrieved docs first
- **Fact-Grounding**: Ensure answer cites sources
- **Iterative Refinement**: Refine based on user feedback

## Implementation Patterns

### Pattern 1: Basic RAG

Simplest RAG implementation:
1. Split documents into chunks
2. Generate embeddings for each chunk
3. Store in vector database
4. Retrieve top-k similar chunks for query
5. Pass to LLM with context

**Pros**: Simple, fast, works well for straightforward QA
**Cons**: May miss relevant context, no refinement

### Pattern 2: Agentic RAG

Agent controls retrieval:
1. Agent receives user question
2. Decides whether to retrieve documents
3. Formulates retrieval query (may differ from original)
4. Retrieves relevant documents
5. Can iterate or use tools
6. Generates final answer

**Pros**: Better for complex questions, iterative improvement
**Cons**: More complex, higher costs

### Pattern 3: Corrective RAG (CRAG)

Validates retrieved documents:
1. Retrieve documents for question
2. Grade each document for relevance
3. If poor relevance:
   - Try different retrieval strategy
   - Expand search scope
   - Retrieve from different sources
4. Generate answer from validated context

**Pros**: Higher quality answers, adapts to failures
**Cons**: More API calls, slower

## Popular Frameworks

### LangChain
```python
from langchain.document_loaders import PDFLoader
from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import Pinecone
from langchain.chains import RetrievalQA

# Load documents
loader = PDFLoader("document.pdf")
docs = loader.load()

# Create RAG chain
embeddings = OpenAIEmbeddings()
vectorstore = Pinecone.from_documents(docs, embeddings)
qa = RetrievalQA.from_chain_type(
    llm=ChatOpenAI(),
    chain_type="stuff",
    retriever=vectorstore.as_retriever()
)

answer = qa.run("What is the document about?")
```

### LlamaIndex
```python
from llama_index import GPTVectorStoreIndex, SimpleDirectoryReader

# Load documents
documents = SimpleDirectoryReader("./data").load_data()

# Create index
index = GPTVectorStoreIndex.from_documents(documents)

# Query
response = index.as