video-generation

What this skill does

# Video Generation Skill

Generate videos using AI (Google Veo 3.1, OpenAI Sora).

**Capabilities:**
- 🎬 **Text-to-Video**: Create videos from text descriptions
- 🖼️ **Image-to-Video**: Animate images as the first frame
- 🔊 **Audio Generation**: Dialogue, sound effects, ambient sounds (Veo 3+)
- 🎭 **Reference Images**: Guide video content with up to 3 reference images (Veo 3.1)

## Prerequisites

### Default: Vertex AI (10 requests/minute) ⭐

Vertex AI is the default backend with 1400x higher rate limits:

```bash
# 1. Set your project
export GOOGLE_CLOUD_PROJECT=your-project-id

# 2. Authenticate (opens browser)
gcloud auth application-default login

# 3. Enable the API (one-time)
gcloud services enable aiplatform.googleapis.com
```

Add to your `.env` file:
```
GOOGLE_CLOUD_PROJECT=your-project-id
GOOGLE_CLOUD_LOCATION=us-central1
```

### Fallback: AI Studio (10 requests/day)

Only use if you don't have a GCP project:

- `GOOGLE_API_KEY` - Get from https://aistudio.google.com/apikey

### For Sora (OpenAI)

- `OPENAI_API_KEY` - For OpenAI Sora

## Available Models

### Google Veo Models

| Model | Description | Best For |
|-------|-------------|----------|
| `veo-3.1` | Highest quality (default) | Professional videos, dialogue, reference images |
| `veo-3.1-fast` | Faster processing | Quick iterations, batch generation |

**Both models include:**
- 720p/1080p resolution
- 4, 6, or 8 second duration
- Native audio (dialogue, SFX, ambient)
- Image-to-video (animate images)
- Reference images (up to 3)
- Video extension
- Batch/parallel generation

### OpenAI Sora
- **Best for**: Creative videos, cinematic quality, complex motion
- **Resolutions**: 480p, 720p, 1080p
- **Durations**: 5s, 10s, 15s, 20s
- **Features**: Text-to-video, image-to-video

## Workflow

### Step 1: Gather Requirements (REQUIRED)

⚠️ **Use interactive questioning — ask ONE question at a time.**

#### Question Flow

⚠️ **Use the `AskUserQuestion` tool for each question below.** Do not just print questions in your response — use the tool to create interactive prompts with the options shown.

**Q1: Image**
> "I'll generate that video for you! First — **do you have an image to animate?**
> 
> - Yes (provide path — I'll use it as the first frame)
> - No, generate from scratch"

*Wait for response.*

**Q2: Audio**
> "What **audio preference**?
> 
> - With audio (default) — Veo 3.1 generates dialogue, SFX, ambient
> - Silent video — no audio"

*Wait for response.*

**Q3: Model**
> "Which **model** would you like?
> 
> - `veo-3.1` — Latest, highest quality with audio (default)
> - `veo-3.1-fast` — Faster processing with audio
> - `veo-3` / `veo-3-fast` — Previous generation with audio
> - `sora` — OpenAI, up to 20 seconds, no audio"

*Wait for response.*

**Q4: Duration**
> "What **duration**?
> 
> - 4 seconds
> - 6 seconds
> - 8 seconds (default)"

*Wait for response.*

**Q5: Format**
> "What **aspect ratio and resolution**?
> 
> - 16:9 landscape, 720p
> - 16:9 landscape, 1080p
> - 9:16 portrait, 720p
> - 9:16 portrait, 1080p
> - Or specify"

*Wait for response.*

#### Quick Reference

| Question | Determines |
|----------|------------|
| Image | Image-to-video vs text-to-video |
| Audio | With/without audio generation |
| Model | Quality and speed tradeoff |
| Duration | Clip length |
| Format | Aspect ratio and resolution |

---

### Step 2: Craft the Prompt

Transform the user request into an effective video prompt:

1. **Describe the scene**: Set the visual context
2. **Specify action**: What moves, changes, happens
3. **Include camera work**: "slow pan", "tracking shot", "dolly shot"
4. **Add audio cues** (Veo 3+): Use quotes for dialogue, describe sounds
5. **Set the mood**: Lighting, atmosphere, time of day

**Example with dialogue (Veo 3.1):**
- User: "a person discovering treasure"
- Enhanced: "Close-up of a treasure hunter's face as torchlight flickers. He murmurs 'This must be it...' while brushing dust off an ancient chest. Sound of creaking hinges as he opens it, revealing golden light on his awestruck face. Cinematic, dramatic shadows."

**Example without dialogue:**
- User: "a dog running on a beach"
- Enhanced: "Cinematic slow-motion shot of a golden retriever running joyfully along a beach at sunset, waves lapping, warm golden hour lighting, shallow depth of field"

### Step 3: Select the Model

**Default: veo-3.1** (highest quality, with audio)

| Use Case | Recommended Model | Reason |
|----------|------------------|--------|
| Best quality | veo-3.1 (default) | Highest quality, audio |
| Quick iteration | veo-3.1-fast | Faster processing |
| Batch generation | veo-3.1-fast | Speed matters for multiple clips |
| Longer videos (>8s) | sora | Supports up to 20s |

### Step 4: Generate the Video

Execute the appropriate script from `${CLAUDE_PLUGIN_ROOT}/skills/video-generation/scripts/`:

**For Google Veo 3.1 (default, with audio):**
```bash
python3 ${CLAUDE_PLUGIN_ROOT}/skills/video-generation/scripts/veo.py \
  --prompt "your enhanced prompt with 'dialogue in quotes'" \
  --model "veo-3.1" \
  --duration 8 \
  --aspect-ratio "16:9" \
  --resolution "720p"
```

**For Google Veo 3.1 with image input:**
```bash
python3 ${CLAUDE_PLUGIN_ROOT}/skills/video-generation/scripts/veo.py \
  --prompt "The cat slowly opens its eyes and yawns" \
  --image "/path/to/cat.jpg" \
  --model "veo-3.1" \
  --duration 8
```

**For faster generation:**
```bash
python3 ${CLAUDE_PLUGIN_ROOT}/skills/video-generation/scripts/veo.py \
  --prompt "your prompt" \
  --model "veo-3.1-fast"
```

**For OpenAI Sora (longer videos):**
```bash
python3 ${CLAUDE_PLUGIN_ROOT}/skills/video-generation/scripts/sora.py \
  --prompt "your enhanced prompt" \
  --duration 20 \
  --resolution "1080p"
```

**List available models:**
```bash
python3 ${CLAUDE_PLUGIN_ROOT}/skills/video-generation/scripts/veo.py --list-models
```

## Video Extension (For Long-Form Continuity)

**The `--extend` flag creates TRUE visual continuity** by continuing from where a previous Veo video ended. This is the best approach for long-form videos.

**Basic extension:**
```bash
# First, generate initial clip
python3 ${CLAUDE_PLUGIN_ROOT}/skills/video-generation/scripts/veo.py \
  --prompt "A person walks through a forest at sunrise" \
  --duration 8

# Extend it with new content (adds ~7 seconds)
python3 ${CLAUDE_PLUGIN_ROOT}/skills/video-generation/scripts/veo.py \
  --extend veo_veo-3.1_20260104_120000.mp4 \
  --prompt "Continue walking, discover a hidden stream"
```

**Multiple extensions (for longer videos):**
```bash
# Extend 5 times (adds ~35 seconds of continuation)
python3 ${CLAUDE_PLUGIN_ROOT}/skills/video-generation/scripts/veo.py \
  --extend initial_clip.mp4 \
  --prompt "Keep exploring the forest, encounter wildlife" \
  --extend-times 5
```

**Extension vs Stitching:**

| Approach | Result | Use Case |
|----------|--------|----------|
| **Extension** | True continuity, same characters/scene | Long continuous shots |
| **Stitching** | Separate clips with transitions | Scene changes, montages |

**Extension Limits:**
- Input video must be Veo-generated (max 141 seconds)
- Each extension adds ~7 seconds
- Maximum 20 extensions total (~2.5 minutes)
- Output resolution is 720p

## Batch Generation (Parallel)

**Generate multiple videos simultaneously** for faster multi-scene workflows. Instead of waiting 15+ minutes for 5 sequential videos, generate them all in parallel (~3 minutes total).

### Create a scenes.json file:

```json
[
  {"prompt": "Scene 1: Cinematic hero shot of wireless earbuds on dark surface", "duration": 6, "output": "scene1_hero.mp4"},
  {"prompt": "Scene 2: Sound waves visualization, person enjoying music", "duration": 8, "output": "scene2_sound.mp4"},
  {"prompt": "Scene 3: Close-up of earbud in ear, person exercising", "duration": 8, "output": "scene3_comfort.mp4"},
  {"prompt": "Scene 4: Lifestyle montage, various settings", "duration": 8, "output": "scene4_lifestyle.mp4"},
  {"prompt": "Scene 5: Pr