gemini-batch

What this skill does

# Gemini Batch Processing

Process large volumes of requests efficiently using Gemini Batch API through executable scripts for cost savings and high throughput.

## When to Use This Skill

Use this skill when you need to:
- Process hundreds/thousands of requests
- Generate content in bulk (blogs, emails, descriptions)
- Reduce costs for high-volume tasks
- Run async jobs without blocking
- Process large datasets with AI
- Generate multiple documents at once
- Create scalable content pipelines
- Process requests that don't need real-time responses

## Available Scripts

### scripts/create_batch.js
**Purpose**: Create a batch job from a JSONL file

**When to use**:
- Starting any batch processing task
- Uploading multiple requests for processing
- Creating async jobs for large workloads

**Key parameters**:
| Parameter | Description | Example |
|-----------|-------------|---------|
| `input_file` | JSONL file path (required) | `requests.jsonl` |
| `--model`, `-m` | Model to use | `gemini-3-flash-preview` |
| `--name`, `-n` | Display name for job | `"my-batch-job"` |

**Output**: Job name/ID to track with check_status.js

### scripts/check_status.js
**Purpose**: Monitor batch job progress and completion

**When to use**:
- Checking if a batch job is complete
- Polling job status until finished
- Monitoring async job execution

**Key parameters**:
| Parameter | Description | Example |
|-----------|-------------|---------|
| `job_name` | Batch job name/ID (required) | `batches/abc123` |
| `--wait`, `-w` | Poll until completion | Flag |

**Output**: Job status and final state

### scripts/get_results.js
**Purpose**: Retrieve completed batch job results

**When to use**:
- Downloading completed batch results
- Parsing batch job output
- Extracting generated content

**Key parameters**:
| Parameter | Description | Example |
|-----------|-------------|---------|
| `job_name` | Batch job name/ID (required) | `batches/abc123` |
| `--output`, `-o` | Output file path | `results.jsonl` |

**Output**: Results content or file

## Workflows

### Workflow 1: Basic Batch Processing
```bash
# 1. Create JSONL file
echo '{"key": "req1", "request": {"contents": [{"parts": [{"text": "Explain photosynthesis"}]}]}}' > requests.jsonl
echo '{"key": "req2", "request": {"contents": [{"parts": [{"text": "What is gravity?"}]}}]}' >> requests.jsonl

# 2. Create batch job
node scripts/create_batch.js requests.jsonl --name "science-questions"

# 3. Check status
node scripts/check_status.js <job-name> --wait

# 4. Get results
node scripts/get_results.js <job-name> --output results.jsonl
```
- Best for: Basic bulk processing, cost efficiency
- Typical time: Minutes to hours depending on job size

### Workflow 2: Bulk Content Generation
```bash
# 1. Generate JSONL with content requests
python3 << 'EOF'
import json

topics = ["sustainable energy", "AI in healthcare", "space exploration"]
with open("content-requests.jsonl", "w") as f:
    for i, topic in enumerate(topics):
        req = {
            "key": f"blog-{i}",
            "request": {
                "contents": [{
                    "parts": [{
                        "text": f"Write a 500-word blog post about {topic}"
                    }]
                }]
            }
        }
        f.write(json.dumps(req) + "\n")
EOF

# 2. Process batch
node scripts/create_batch.js content-requests.jsonl --name "blog-posts" --model gemini-3-flash-preview
node scripts/check_status.js <job-name> --wait
node scripts/get_results.js <job-name> --output blog-posts.jsonl
```
- Best for: Blog generation, article creation, bulk writing
- Combines with: gemini-text for content needs

### Workflow 3: Dataset Processing
```bash
# 1. Load dataset and create batch requests
python3 << 'EOF'
import json

# Your dataset
data = [
    {"product": "laptop", "features": ["fast", "lightweight"]},
    {"product": "headphones", "features": ["wireless", "noise-cancelling"]},
]

with open("product-descriptions.jsonl", "w") as f:
    for item in data:
        features = ", ".join(item["features"])
        prompt = f"Write a product description for {item['product']} with these features: {features}"
        req = {
            "key": item["product"],
            "request": {
                "contents": [{"parts": [{"text": prompt}]}]
            }
        }
        f.write(json.dumps(req) + "\n")
EOF

# 2. Process
node scripts/create_batch.js product-descriptions.jsonl
node scripts/check_status.js <job-name> --wait
node scripts/get_results.js <job-name> --output results.jsonl
```
- Best for: Product descriptions, dataset enrichment, bulk analysis

### Workflow 4: Email Campaign Generation
```bash
# 1. Create personalized email requests
python3 << 'EOF'
import json

customers = [
    {"name": "Alice", "product": "premium plan"},
    {"name": "Bob", "product": "basic plan"},
]

with open("emails.jsonl", "w") as f:
    for cust in customers:
        prompt = f"Write a personalized email to {cust['name']} about upgrading to our {cust['product']}"
        req = {
            "key": f"email-{cust['name'].lower()}",
            "request": {
                "contents": [{"parts": [{"text": prompt}]}]
            }
        }
        f.write(json.dumps(req) + "\n")
EOF

# 2. Process batch
node scripts/create_batch.js emails.jsonl --name "email-campaign"
node scripts/check_status.js <job-name> --wait
node scripts/get_results.js <job-name> --output email-results.jsonl
```
- Best for: Marketing campaigns, personalized outreach
- Combines with: gemini-text for email content

### Workflow 5: Async Job Monitoring
```bash
# 1. Create job
node scripts/create_batch.js large-batch.jsonl --name "big-job"

# 2. Check status periodically (non-blocking)
while true; do
    node scripts/check_status.js <job-name>
    sleep 60  # Check every minute
done

# 3. Get results when done
node scripts/get_results.js <job-name> --output final-results.jsonl
```
- Best for: Long-running jobs, background processing
- Use when: You don't need immediate results

### Workflow 6: Cost-Optimized Bulk Processing
```bash
# 1. Use flash model for cost efficiency
node scripts/create_batch.js requests.jsonl --model gemini-3-flash-preview --name "cost-optimized"

# 2. Monitor and retrieve
node scripts/check_status.js <job-name> --wait
node scripts/get_results.js <job-name>
```
- Best for: High-volume, cost-sensitive applications
- Savings: Batch API typically 50%+ cheaper than real-time

### Workflow 7: Multi-Stage Pipeline
```bash
# Stage 1: Generate content
node scripts/create_batch.js content-requests.jsonl --name "stage1-content"
node scripts/check_status.js <job1> --wait

# Stage 2: Summarize content
node scripts/create_batch.js summaries.jsonl --name "stage2-summaries"
node scripts/check_status.js <job2> --wait

# Stage 3: Convert to audio (gemini-tts)
# Process results from stage 2
```
- Best for: Complex workflows, multi-step processing
- Combines with: Other Gemini skills for complete pipelines

## Parameters Reference

### JSONL Format

Each line is a separate JSON object:

```json
{
  "key": "unique-identifier",
  "request": {
    "contents": [
      {
        "parts": [
          {
            "text": "Your prompt here"
          }
        ]
      }
    ]
  }
}
```

### Model Selection

| Model | Best For | Cost | Speed |
|-------|----------|------|-------|
| `gemini-3-flash-preview` | General bulk processing | Lowest | Fast |
| `gemini-3-pro-preview` | Complex reasoning tasks | Medium | Medium |
| `gemini-2.5-flash` | Stable, reliable | Low | Fast |
| `gemini-2.5-pro` | Code/math/STEM | Medium | Slow |

### Job States

| State | Description |
|-------|-------------|
| `JOB_STATE_PENDING` | Job queued, waiting to start |
| `JOB_STATE_RUNNING` | Job actively processing |
| `JOB_STATE_SUCCEEDED` | Job completed successfully |
| `JOB_STATE_FAILED` | Job failed (check error message) |
| `JOB_STATE_CANCELLED` | Job was cancelled |
| `JOB_STATE_EXPIRED` | Job timed out |

### Size Limits

| Method | Max