document-to-narration

What this skill does

# Document to Narration

Convert written documents into narrated video scripts with precise word-level timing.

## Core Principle

**The agent interprets; the document guides.** Rather than rigid template-based splits, this skill uses agent judgment to find where the content naturally breathes, argues, and transitions. The document's argument flow determines scene breaks, not a predetermined structure.

## When to Use This Skill

Use this skill when:
- Converting a blog post or essay to video narration
- Preparing content for TTS audio generation
- Breaking long-form content into digestible scenes
- Creating word-level synchronized captions for video

Do NOT use this skill when:
- The content is already in scene/script format
- You need real-time voice synthesis (this is batch processing)
- Working with dialogue or multi-speaker content (single voice only)

## Prerequisites

- **Deno** installed (https://deno.land/)
- **Python 3.12** with venv support
- **ffmpeg** for audio conversion
- **whisper-cpp** (installed via @remotion/install-whisper-cpp)
- **TTS model** at `tts/model/` (not in git due to size - see Model Setup below)

## Complete Pipeline

There are two approaches: **per-scene** (legacy) and **full narration** (recommended).

### Full Narration Pipeline (Recommended)

Generates a single audio file for consistent volume and pacing:

```
Document (.md)
    ↓ [agent interprets scene breaks]
Scene .txt files (01-scene-name.txt, 02-scene-name.txt, ...)
    ↓ [TTS via narrate-full.py - SINGLE PASS]
full-narration.wav (one consistent audio file)
    ↓ [Whisper via transcribe-full.py]
full-narration.json + full-narration.vtt (word-level timing)
    ↓ [extract-scene-boundaries.py]
Scene timing boundaries for video composition
```

### Per-Scene Pipeline (Legacy)

Generates separate audio per scene - **can cause volume inconsistencies**:

```
Scene .txt files
    ↓ [TTS via narrate-scenes.py - MULTIPLE PASSES]
Scene .wav files (volume may vary between scenes)
    ↓ [concatenate]
Combined audio (may have clipping at boundaries)
```

> **Warning:** Per-scene TTS generates audio with different volume levels and pacing. When concatenated, this causes audible jumps and clipping. Use the full narration pipeline instead.

## Quick Start

### Full Narration Pipeline (Recommended)

```bash
cd .claude/skills/document-to-narration
source tts/.venv/bin/activate

# 1. Split document into scenes (manual or scripted)
deno run --allow-read --allow-write scripts/split-to-scenes.ts input.md --output ./output/

# 2. Generate single audio file
python scripts/narrate-full.py ./output/scenes/

# 3. Transcribe with word-level timestamps
python scripts/transcribe-full.py ./output/full-narration.wav

# 4. Extract scene boundaries for video timing
python scripts/extract-scene-boundaries.py ./output/scenes/ ./output/full-narration.json --typescript
```

### Legacy Per-Scene Pipeline

```bash
# 1. Split document into scenes
deno run --allow-read --allow-write scripts/split-to-scenes.ts input.md --output ./output/

# 2. Generate audio per scene (may have volume inconsistencies)
source tts/.venv/bin/activate
python scripts/narrate-scenes.py ./output/scenes/

# 3. Transcribe (DEPRECATED: transcribe-scenes.ts requires whisper-cpp)
# Use transcribe-full.py instead after concatenating audio
```

## Instructions

### Step 1: Setup (First Time Only)

#### Create Python Virtual Environment

```bash
cd .claude/skills/document-to-narration/tts
python3.12 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```

#### TTS Model Setup

The fine-tuned voice model (~7.8GB) is not included in git due to size.
Place your Qwen3-TTS model files in `tts/model/`:

```
tts/model/
├── config.json
├── generation_config.json
├── model.safetensors      # Main model weights
├── tokenizer_config.json
├── vocab.json
├── merges.txt
└── speech_tokenizer/
    └── ...
```

#### Install Whisper (if not already installed)

The @remotion/install-whisper-cpp package handles this:
```typescript
import { installWhisperCpp, downloadWhisperModel } from '@remotion/install-whisper-cpp';

await installWhisperCpp({ to: './whisper-cpp', version: '1.5.5' });
await downloadWhisperModel({ model: 'medium', folder: './whisper-cpp' });
```

### Step 2: Prepare Your Document

The skill works best with:
- Markdown documents with clear heading structure (H1, H2)
- Well-structured arguments with distinct sections
- Content that reads naturally aloud

### Step 3: Run the Pipeline

```bash
deno run -A scripts/full-pipeline.ts /path/to/essay.md --output ./output/essay-name/
```

### Step 4: Review Output

```
output/essay-name/
├── scenes/
│   ├── 01-opening-hook.txt      # Scene script
│   ├── 01-opening-hook.wav      # Generated audio
│   ├── 01-opening-hook.vtt      # Word-level captions
│   ├── 02-core-argument.txt
│   ├── 02-core-argument.wav
│   ├── 02-core-argument.vtt
│   └── ...
└── manifest.json                # Complete timing data
```

## Scene Boundary Heuristics

The agent identifies scene breaks using these heuristics:

### Strong Boundaries (Almost Always Break)
- H2 heading changes
- "Here's the thing" / "The point is" pivot statements
- Major metaphor introduction
- Explicit enumeration ("First...", "Second...")
- Significant perspective shifts

### Moderate Boundaries (Consider Breaking)
- Long paragraph after short ones (or vice versa)
- Example-to-principle transitions
- "But" / "However" / "Meanwhile" at paragraph start
- Question-then-answer patterns

### Weak Boundaries (Usually Keep Together)
- Paragraph-to-paragraph within same example
- Sequential evidence for same point
- Build-up to a punchline/reveal

### Scene Length Guidance
- **Target**: 100-300 words per scene (30-90 seconds of audio)
- **Minimum**: 50 words (avoid micro-scenes)
- **Maximum**: 500 words (avoid cognitive overload)

## Anti-Patterns

### The Paragraph Slicer
**Pattern:** Breaking at every paragraph or heading mechanically.
**Problem:** Ignores argument flow. Scenes feel choppy and disconnected.
**Fix:** Look for rhetorical units, not structural units. Multiple paragraphs often form one scene.

### The Wall of Text
**Pattern:** Keeping entire sections as single scenes.
**Problem:** Creates TTS audio that's too long. Loses natural breathing room.
**Fix:** Target 100-300 words. Find the natural pause point within sections.

### The Verbatim Transcriber
**Pattern:** Copying written text exactly without spoken adaptation.
**Problem:** Written conventions don't work when spoken. Parentheticals, complex punctuation, and nested clauses confuse TTS and listeners.
**Fix:** Apply adaptation rules. Read it aloud mentally.

### The Over-Adapter
**Pattern:** Rewriting content so heavily it loses the author's voice.
**Problem:** The result doesn't sound like the original author.
**Fix:** Preserve voice, adjust mechanics. If the author uses rhetorical questions, keep them.

## Available Scripts

### scripts/split-to-scenes.ts

Parse a markdown document and output scene text files.

```bash
deno run --allow-read --allow-write scripts/split-to-scenes.ts input.md --output ./output/
deno run --allow-read --allow-write scripts/split-to-scenes.ts input.md --output ./output/ --adapt
deno run --allow-read scripts/split-to-scenes.ts input.md --dry-run
```

**Options:**
- `--output` - Directory for scene files (created if doesn't exist)
- `--adapt` - Apply spoken adaptation rules
- `--dry-run` - Preview scene breaks without writing files

**Output:** Numbered `.txt` files and initial `manifest.json`

### scripts/narrate-full.py (Recommended)

Generate a single TTS audio file from all scene files. Produces consistent volume and pacing.

```bash
python scripts/narrate-full.py ./output/scenes/
python scripts/narrate-full.py ./output/scenes/ --force
python scripts/narrate-full.py ./output/scenes/ --speaker jwynia
python scripts/narrate-full.py ./output/scenes/ --output ./custom/path/audio.wav
```

**Options:**
- `--force` - Regenerate even if out