rag
Implements Retrieval-Augmented Generation for AI models to fetch and use external knowledge.
What this skill does
# rag
## Purpose
This skill implements Retrieval-Augmented Generation (RAG) for OpenClaw, enabling AI models to query external knowledge bases and integrate results into responses, enhancing accuracy for tasks like question-answering.
## When to Use
Use this skill when your AI needs dynamic access to external data, such as querying a vector database for real-time information in NLP tasks, handling knowledge gaps in models, or augmenting responses in chatbots. Avoid it for purely generative tasks without external dependencies.
## Key Capabilities
- Fetches documents from vector databases (e.g., Pinecone, FAISS) using similarity search.
- Integrates retrieved content into AI prompts for generation.
- Supports embedding models for query vectorization (e.g., via Hugging Face transformers).
- Handles chunking of large documents and relevance scoring.
- Configurable via JSON files for custom sources and thresholds.
## Usage Patterns
Always set the API key via environment variable: `export OPENCLAW_API_KEY=$SERVICE_API_KEY`. For CLI, use `openclaw rag` with required flags. In code, import the skill and call methods like `rag.retrieve()`. Pattern: Query -> Retrieve -> Augment -> Generate. Ensure queries are under 512 tokens to avoid truncation.
## Common Commands/API
- CLI Command: `openclaw rag query --db pinecone --index myindex --query "What is RAG?" --top-k 5`
- Flags: `--db` specifies database (e.g., pinecone, faiss), `--index` for collection name, `--top-k` for result count, `--query` for search string.
- API Endpoint: POST /v1/rag/retrieve with JSON body: `{"query": "Explain AI", "db": "pinecone", "top_k": 3}`
- Response: JSON object with keys like `"results"` (array of documents) and `"scores"`.
- Code Snippet (Python):
```python
import openclaw
client = openclaw.Client(api_key=os.environ['OPENCLAW_API_KEY'])
results = client.rag.retrieve(query="What is NLP?", db="faiss", top_k=4)
```
- Config Format: JSON file (e.g., rag_config.json):
```
{
"db": "pinecone",
"api_endpoint": "https://api.pinecone.io",
"embedding_model": "text-embedding-ada-002"
}
```
Load it via: `openclaw rag config load --file rag_config.json`.
## Integration Notes
Integrate by wrapping RAG calls around your AI pipeline: First, call `rag.retrieve()` to get context, then pass it to your model's prompt. For multi-skill workflows, chain with "aiml" skills by piping outputs (e.g., use RAG results as input to a generation skill). Handle asynchronous calls with `await client.rag.retrieve_async()` in async environments. Test integrations in a sandbox with mock databases to verify data flow.
## Error Handling
Check for common errors like authentication failures (e.g., "401 Unauthorized" if $OPENCLAW_API_KEY is invalid) by verifying env vars first. For query errors, catch exceptions like `RetrievalError` and retry with exponential backoff:
```python
try:
results = client.rag.retrieve(query=query)
except openclaw.RetrievalError as e:
if e.status_code == 404:
print("Database not found; create index first.")
else:
raise
```
Log all errors with details (e.g., error codes, messages) and use `--debug` flag in CLI for verbose output. Always validate inputs (e.g., ensure query is a string) before calling.
## Concrete Usage Examples
1. **Example 1: CLI Query for Knowledge Retrieval**
Use to answer user questions: Run `openclaw rag query --db faiss --index docs_index --query "Summarize RAG technique" --top-k 3`. This retrieves top 3 documents from the "docs_index" database and outputs them. Pipe the result: `openclaw rag query ... | openclaw aiml generate --prompt "Use this context:"`.
2. **Example 2: Code Integration for AI Response**
In a Python script, augment a chatbot:
```python
query = "What is machine learning?"
context = client.rag.retrieve(query=query, db="pinecone", top_k=2)
full_prompt = f"Context: {context}\nAnswer: {query}"
response = client.aiml.generate(prompt=full_prompt)
print(response)
```
This fetches relevant context and passes it to the AI for a informed response.
## Graph Relationships
- Related to: aiml (for generation integration), nlp (for text processing), vector-db (for data storage dependencies).
- Depends on: embedding skills for vectorization.
- Used by: knowledge-base skills for external data access.
Related in AI Agents
skill-development
IncludedComprehensive meta-skill for creating, managing, validating, auditing, and distributing Claude Code skills and slash commands (unified in v2.1.3+). Provides skill templates, creation workflows, validation patterns, audit checklists, naming conventions, YAML frontmatter guidance, progressive disclosure examples, and best practices lookup. Use when creating new skills, validating existing skills, auditing skill quality, understanding skill architecture, needing skill templates, learning about YAML frontmatter requirements, progressive disclosure patterns, tool restrictions (allowed-tools), skill composition, skill naming conventions, troubleshooting skill activation issues, creating custom slash commands, configuring command frontmatter, using command arguments ($ARGUMENTS, $1, $2), bash execution in commands, file references in commands, command namespacing, plugin commands, MCP slash commands, Skill tool configuration, or deciding between skills vs slash commands. Delegates to docs-management skill for official documentation.
reprompter
IncludedTransform messy prompts into well-structured, effective prompts — single or multi-agent. Use when: "reprompt", "reprompt this", "clean up this prompt", "structure my prompt", rough text needing XML tags and best practices, "reprompter teams", "repromptception", "run with quality", "smart run", "smart agents", multi-agent tasks, audits, parallel work, anything going to agent teams. Don't use when: simple Q&A, pure chat, immediate execution-only tasks. See "Don't Use When" section for details. Outputs: Structured XML/Markdown prompt, quality score (before/after), optional team brief + per-agent sub-prompts, agent team output files. Success criteria: Single mode quality score ≥ 7/10; Repromptception per-agent prompt quality score 8+/10; all required sections present, actionable and specific.
adaptive-compaction
IncludedAdaptive add-on policy and recovery layer that decides WHEN to compact, prune, snapshot, or fork -- replacing fixed-percent auto-compaction across Claude Code, Codex, and MCP-capable hosts. Trigger on auto-compact timing or damage: "when should I compact", "is it safe to compact now or start a fresh session", "auto-compact fires too early/mid-task", "switching to an unrelated task but the window still has space", "context rot", "answers get worse the longer the session runs", "the agent forgot the plan or my decisions after it summarized", "add a layer on top that manages context without changing the agent", raising autoCompactWindow to give the policy room, or installing/tuning a cross-tool compaction policy or PreCompact hook -- even when "compaction" is never said but the problem is context-window pressure or post-summarization memory loss. Do NOT use to summarize a conversation, build RAG, write a summarization prompt (decides WHEN not HOW), or answer max-context-length trivia.
agent-skill-creator
IncludedCreate cross-platform agent skills from workflow descriptions. Activates when users ask to create an agent, automate a repetitive workflow, create a custom skill, or need advanced agent creation. Triggers on phrases like create agent for, automate workflow, create skill for, every day I have to, daily I need to, turn process into agent, need to automate, create a cross-platform skill, validate this skill, export this skill, migrate this skill. Supports single skills, multi-agent suites, transcript processing, template-based creation, interactive configuration, cross-platform export, and spec validation.
llm-wiki
IncludedUse when building or maintaining a persistent personal knowledge base (second brain) in Obsidian where an LLM incrementally ingests sources, updates entity/concept pages, maintains cross-references, and keeps a synthesis current. Triggers include "second brain", "Obsidian wiki", "personal knowledge management", "ingest this paper/article/book", "build a research wiki", "compound knowledge", "Memex", or whenever the user wants knowledge to accumulate across sessions instead of being re-derived by RAG on every query.
skill-master
IncludedAgent Skills authoring, evaluation, and optimization. Create, edit, validate, benchmark, and improve skills following the agentskills.io specification. Use when designing SKILL.md files, structuring skill folders (references, scripts, assets), ingesting external documentation into skills, running trigger evals, benchmarking skill quality, optimizing descriptions, or performing blind A/B comparisons. Keywords: agentskills.io, SKILL.md, skill authoring, eval, benchmark, trigger optimization.