google-gemini-file-search

What this skill does

# Google Gemini File Search Setup

## Overview

Google Gemini File Search is a fully managed RAG (Retrieval-Augmented Generation) system that eliminates the need for separate vector databases, custom chunking logic, or embedding generation code. Upload documents (PDFs, Word, Excel, code files, etc.) and query them using natural language—Gemini automatically handles intelligent chunking, embedding with its optimized model, semantic search, and citation generation.

**What This Skill Provides:**
- Complete setup guide for @google/genai File Search API
- TypeScript/JavaScript SDK configuration patterns
- Working templates for 3 deployment scenarios (Node.js, Cloudflare Workers, Next.js)
- 8 documented common errors with prevention strategies
- Chunking best practices for optimal retrieval
- Cost optimization techniques
- Comparison guide (vs Cloudflare Vectorize, OpenAI Files API, Claude MCP)

**Key Features of File Search:**
- **100+ File Formats**: PDF, Word (.docx), Excel (.xlsx), PowerPoint (.pptx), Markdown, JSON, CSV, code files (Python, JavaScript, TypeScript, Java, C++, Go, Rust, etc.)
- **Automatic Embeddings**: Uses Google's Gemini Embedding model (no custom embedding code required)
- **Semantic Search**: Vector-based search understands meaning and context, not just keywords
- **Built-in Citations**: Grounding metadata automatically points to specific document sections
- **Custom Metadata**: Filter queries by up to 20 custom key-value pairs per document
- **Configurable Chunking**: Control chunk size (tokens) and overlap for precision tuning
- **Cost-Effective**: $0.15/1M tokens for one-time indexing, free storage (up to limits), free query-time embeddings

## When to Use This Skill

**Ideal Use Cases:**
- Building customer support knowledge bases (manuals, FAQs, troubleshooting guides)
- Creating internal documentation search (company wikis, policies, procedures)
- Legal/compliance document analysis (contracts, regulations, case law)
- Research tools (academic papers, articles, textbooks)
- Code documentation search (API docs, SDK references, examples)
- Product information retrieval (specs, datasheets, user guides)

**Use File Search When:**
- ✅ You want a fully managed RAG solution (no vector DB setup)
- ✅ Cost predictability matters (pay-per-indexing, not continuous storage fees)
- ✅ You need broad file format support (100+ types out of the box)
- ✅ Citations are important (built-in grounding metadata)
- ✅ Simple deployment is priority (single API setup)
- ✅ Documents are relatively static (updates are infrequent)

**Use Cloudflare Vectorize/AutoRAG Instead When:**
- ✅ Global edge performance is critical (low-latency worldwide)
- ✅ Building full-stack apps on Cloudflare (Workers, R2, D1)
- ✅ You need custom embedding models or retrieval logic
- ✅ Real-time data updates from R2 or external sources

**Use OpenAI Files API Instead When:**
- ✅ Already using OpenAI Assistants API (conversational threads)
- ✅ Need to attach knowledge to persistent assistant threads
- ✅ Working with very large file collections (10,000+ files per store)
- ✅ Prefer storage-based pricing model ($0.10/GB/day)

## When NOT to Use This Skill

Skip this skill if:
- ❌ Need custom embedding models (File Search locks you to Gemini Embeddings)
- ❌ Documents update frequently (no streaming updates, must delete+re-upload)
- ❌ Building conversational AI agents (use OpenAI Assistants or Claude MCP instead)
- ❌ Need BM25/hybrid search (File Search is vector-only)
- ❌ Require advanced reranking configuration (automatic only)
- ❌ Need to parse images/tables from PDFs (text extraction only)

## Prerequisites

### 1. Google AI API Key

Create an API key at https://aistudio.google.com/apikey

**Free Tier Limits:**
- 1 GB storage (total across all file search stores)
- 1,500 requests per day
- 1 million tokens per minute

**Paid Tier Pricing:**
- Indexing: $0.15 per 1M input tokens (one-time)
- Storage: Free (Tier 1: 10 GB, Tier 2: 100 GB, Tier 3: 1 TB)
- Query-time embeddings: Free (retrieved context counts as input tokens)

### 2. Node.js Environment

**Minimum Version:** Node.js 18+ (v20+ recommended)

```bash
node --version  # Should be >=18.0.0
```

### 3. Install @google/genai SDK

```bash
npm install @google/genai
# or
pnpm add @google/genai
# or
yarn add @google/genai
```

**Current Stable Version:** 0.21.0+ (verify with `npm view @google/genai version`)

### 4. TypeScript Configuration (Optional but Recommended)

```json
{
  "compilerOptions": {
    "target": "ES2020",
    "module": "ESNext",
    "moduleResolution": "node",
    "esModuleInterop": true,
    "strict": true,
    "skipLibCheck": true
  }
}
```

## Common Errors Prevented

This skill prevents 8 common errors encountered when implementing File Search:

### Error 1: Document Immutability

**Symptom:**
```
Error: Documents cannot be modified after indexing
```

**Cause:** Documents are immutable once indexed. There is no PATCH or UPDATE operation.

**Prevention:**
Use the delete+re-upload pattern for updates:

```typescript
// ❌ WRONG: Trying to update document (no such API)
await ai.fileSearchStores.documents.update({
  name: documentName,
  customMetadata: { version: '2.0' }
})

// ✅ CORRECT: Delete then re-upload
const docs = await ai.fileSearchStores.documents.list({
  parent: fileStore.name
})

const oldDoc = docs.documents.find(d => d.displayName === 'manual.pdf')
if (oldDoc) {
  await ai.fileSearchStores.documents.delete({
    name: oldDoc.name,
    force: true
  })
}

await ai.fileSearchStores.uploadToFileSearchStore({
  name: fileStore.name,
  file: fs.createReadStream('manual-v2.pdf'),
  config: { displayName: 'manual.pdf' }
})
```

**Source:** https://ai.google.dev/api/file-search/documents

### Error 2: Storage Quota Exceeded

**Symptom:**
```
Error: Quota exceeded. Expected 1GB limit, but 3.2GB used.
```

**Cause:** Storage calculation includes input files + embeddings + metadata. Total storage ≈ 3x input size.

**Prevention:**
Calculate storage before upload:

```typescript
// ❌ WRONG: Assuming storage = file size
const fileSize = fs.statSync('data.pdf').size // 500 MB
// Expect 500 MB usage → WRONG

// ✅ CORRECT: Account for 3x multiplier
const fileSize = fs.statSync('data.pdf').size // 500 MB
const estimatedStorage = fileSize * 3 // 1.5 GB (embeddings + metadata)
console.log(`Estimated storage: ${estimatedStorage / 1e9} GB`)

// Check if within quota before upload
if (estimatedStorage > 1e9) {
  console.warn('⚠️ File may exceed free tier 1 GB limit')
}
```

**Source:** https://blog.google/technology/developers/file-search-gemini-api/

### Error 3: Incorrect Chunking Configuration

**Symptom:**
Poor retrieval quality, irrelevant results, or context cutoff mid-sentence.

**Cause:** Default chunking may not be optimal for your content type.

**Prevention:**
Use recommended chunking strategy:

```typescript
// ❌ WRONG: Using defaults without testing
await ai.fileSearchStores.uploadToFileSearchStore({
  name: fileStore.name,
  file: fs.createReadStream('docs.pdf')
  // Default chunking may be too large or too small
})

// ✅ CORRECT: Configure chunking for precision
await ai.fileSearchStores.uploadToFileSearchStore({
  name: fileStore.name,
  file: fs.createReadStream('docs.pdf'),
  config: {
    chunkingConfig: {
      whiteSpaceConfig: {
        maxTokensPerChunk: 500,  // Smaller chunks = more precise retrieval
        maxOverlapTokens: 50     // 10% overlap prevents context loss
      }
    }
  }
})
```

**Chunking Guidelines:**
- **Technical docs/code:** 500 tokens/chunk, 50 overlap
- **Prose/articles:** 800 tokens/chunk, 80 overlap
- **Legal/contracts:** 300 tokens/chunk, 30 overlap (high precision)

**Source:** https://www.philschmid.de/gemini-file-search-javascript

### Error 4: Metadata Limits Exceeded

**Symptom:**
```
Error: Maximum 20 custom metadata key-value pairs allowed
```

**Cause:** Each document can have at most 20 metadata fields.

**Prevention:**
Design compact metadata schema:

```