ffmpeg-captions-subtitles

What this skill does

## CRITICAL GUIDELINES

### Windows File Path Requirements

**MANDATORY: Always Use Backslashes on Windows for File Paths**

When using Edit or Write tools on Windows, you MUST use backslashes (`\`) in file paths, NOT forward slashes (`/`).

### Documentation Guidelines

**NEVER create new documentation files unless explicitly requested by the user.**

---

## Quick Reference

| Task | Command |
|------|---------|
| Burn SRT | `ffmpeg -i video.mp4 -vf "subtitles=subs.srt" output.mp4` |
| Burn ASS | `ffmpeg -i video.mp4 -vf "ass=subs.ass" output.mp4` |
| Add soft sub | `ffmpeg -i video.mp4 -i subs.srt -c copy -c:s mov_text output.mp4` |
| Extract sub | `ffmpeg -i video.mkv -map 0:s:0 output.srt` |
| Style subs | `-vf "subtitles=s.srt:force_style='FontSize=24,PrimaryColour=&HFFFFFF'"` |
| Text overlay | `-vf "drawtext=text='Hello':x=10:y=10:fontsize=24:fontcolor=white"` |

| Format | Extension | Best For |
|--------|-----------|----------|
| SRT | .srt | Simple, universal |
| ASS | .ass | Styled, animated, anime |
| VTT | .vtt | Web/HTML5 video |

## When to Use This Skill

Use for **subtitle and caption operations**:
- Hardcoding (burning) subtitles into video
- Adding soft subtitle tracks to containers
- Extracting subtitles from MKV/MP4
- Styling captions (font, color, position)
- Dynamic text overlays

---

# FFmpeg Captions and Subtitles (2025)

Complete guide to working with subtitles, closed captions, and text overlays using FFmpeg.

## Subtitle Format Reference

### Supported Formats

| Format | Extension | Features | Use Case |
|--------|-----------|----------|----------|
| SubRip | .srt | Simple timing + text | Universal, web |
| Advanced SubStation Alpha | .ass/.ssa | Rich styling, positioning, effects | Anime, styled subs |
| WebVTT | .vtt | Web standard, cues, styling | HTML5 video |
| TTML/DFXP | .ttml/.dfxp | Broadcast, accessibility | Streaming services |
| MOV Text | .mov (embedded) | QuickTime native | Apple ecosystem |
| DVB Subtitle | (embedded) | Bitmap-based | European broadcast |
| PGS | .sup | Blu-ray bitmap subtitles | Blu-ray |
| CEA-608/708 | (embedded) | Closed captions | US broadcast, streaming |

### Format Characteristics

```bash
SRT (SubRip):
- Simple text-based format
- Supports basic HTML tags (<b>, <i>, <u>)
- Widely compatible
- No positioning or advanced styling

ASS/SSA:
- Advanced styling (fonts, colors, outlines)
- Precise positioning anywhere on screen
- Animation and effects support
- Karaoke timing

WebVTT:
- HTML5 standard format
- CSS-like styling
- Cue settings for positioning
- Speaker identification
```

## Burning Subtitles (Hardcoding)

### Basic Subtitle Burn-in

```bash
# Burn SRT subtitles
ffmpeg -i video.mp4 -vf "subtitles=subs.srt" output.mp4

# Burn ASS/SSA subtitles (preserves styling)
ffmpeg -i video.mp4 -vf "ass=subs.ass" output.mp4

# Burn subtitles from MKV container
ffmpeg -i video.mkv -vf "subtitles=video.mkv" output.mp4

# Burn specific subtitle track (index 0)
ffmpeg -i video.mkv -vf "subtitles=video.mkv:si=0" output.mp4

# Burn subtitles with stream index
ffmpeg -i video.mkv -vf "subtitles=video.mkv:stream_index=1" output.mp4
```

### Styled Subtitle Burn-in

```bash
# Force style (overrides subtitle styling)
ffmpeg -i video.mp4 \
  -vf "subtitles=subs.srt:force_style='FontName=Arial,FontSize=24,PrimaryColour=&HFFFFFF,OutlineColour=&H000000,Outline=2,Shadow=1'" \
  output.mp4

# Yellow subtitles with black outline
ffmpeg -i video.mp4 \
  -vf "subtitles=subs.srt:force_style='FontSize=28,PrimaryColour=&H00FFFF,OutlineColour=&H000000,Outline=3'" \
  output.mp4

# Larger font for accessibility
ffmpeg -i video.mp4 \
  -vf "subtitles=subs.srt:force_style='FontSize=36,Bold=1'" \
  output.mp4
```

### ASS Style Parameters

| Parameter | Description | Example |
|-----------|-------------|---------|
| FontName | Font family | `FontName=Arial` |
| FontSize | Size in points | `FontSize=24` |
| PrimaryColour | Text color (AABBGGRR) | `PrimaryColour=&HFFFFFF` |
| SecondaryColour | Karaoke color | `SecondaryColour=&H00FFFF` |
| OutlineColour | Outline/border color | `OutlineColour=&H000000` |
| BackColour | Shadow/background color | `BackColour=&H80000000` |
| Bold | Bold text (0/1) | `Bold=1` |
| Italic | Italic text (0/1) | `Italic=1` |
| Underline | Underlined text (0/1) | `Underline=1` |
| Outline | Outline width | `Outline=2` |
| Shadow | Shadow depth | `Shadow=1` |
| Alignment | Position (numpad style) | `Alignment=2` |
| MarginL/R/V | Margins in pixels | `MarginV=50` |

### Color Format (ASS)

```text
ASS uses &HAABBGGRR format (Alpha, Blue, Green, Red):
- White: &HFFFFFF or &H00FFFFFF
- Black: &H000000 or &H00000000
- Yellow: &H00FFFF (00-Blue, FF-Green, FF-Red)
- Red: &H0000FF
- Blue: &HFF0000
- 50% transparent black: &H80000000
```

## Adding Subtitle Tracks (Soft Subs)

### Embed SRT as Track

```bash
# Add SRT to MP4 (MOV text)
ffmpeg -i video.mp4 -i subs.srt \
  -c copy -c:s mov_text \
  output.mp4

# Add SRT to MKV
ffmpeg -i video.mp4 -i subs.srt \
  -c copy -c:s srt \
  output.mkv

# Add SRT to WebM (WebVTT)
ffmpeg -i video.webm -i subs.srt \
  -c copy -c:s webvtt \
  output.webm
```

### Multiple Subtitle Tracks

```bash
# Add multiple languages
ffmpeg -i video.mp4 -i subs_en.srt -i subs_es.srt -i subs_fr.srt \
  -map 0:v -map 0:a -map 1 -map 2 -map 3 \
  -c copy -c:s mov_text \
  -metadata:s:s:0 language=eng -metadata:s:s:0 title="English" \
  -metadata:s:s:1 language=spa -metadata:s:s:1 title="Spanish" \
  -metadata:s:s:2 language=fra -metadata:s:s:2 title="French" \
  output.mp4

# Add ASS subtitles to MKV (preserves styling)
ffmpeg -i video.mp4 -i styled.ass \
  -map 0 -map 1 \
  -c copy -c:s ass \
  output.mkv
```

### Set Default Subtitle Track

```bash
# Set subtitle as default
ffmpeg -i video.mp4 -i subs.srt \
  -c copy -c:s mov_text \
  -disposition:s:0 default \
  output.mp4

# Set forced subtitles (always display)
ffmpeg -i video.mp4 -i forced.srt \
  -c copy -c:s mov_text \
  -disposition:s:0 forced \
  output.mp4
```

## Extracting Subtitles

### Extract to External File

```bash
# Extract first subtitle track to SRT
ffmpeg -i video.mkv -map 0:s:0 output.srt

# Extract specific subtitle stream
ffmpeg -i video.mkv -map 0:s:1 output.srt

# Extract to ASS format
ffmpeg -i video.mkv -map 0:s:0 output.ass

# Extract to WebVTT
ffmpeg -i video.mkv -map 0:s:0 output.vtt

# Extract all subtitle tracks
ffmpeg -i video.mkv -map 0:s subs_%d.srt
```

### List Available Subtitle Tracks

```bash
# Show all streams including subtitles
ffprobe -v error -show_entries stream=index,codec_name,codec_type:stream_tags=language,title \
  -of csv=p=0 video.mkv

# Show only subtitle streams
ffprobe -v error -select_streams s \
  -show_entries stream=index,codec_name:stream_tags=language,title \
  -of csv=p=0 video.mkv
```

### Convert Subtitle Formats

```bash
# SRT to ASS
ffmpeg -i subs.srt subs.ass

# ASS to SRT (loses styling)
ffmpeg -i subs.ass subs.srt

# SRT to WebVTT
ffmpeg -i subs.srt subs.vtt

# WebVTT to SRT
ffmpeg -i subs.vtt subs.srt
```

## drawtext Overlays and Whisper AI Integration

Detailed examples for `drawtext` overlays (fonts, positioning, escaping, time expressions, boxes, outlines, animated text) and Whisper AI transcription / subtitle generation workflows live in `references/drawtext-and-whisper.md`. Load that reference when creating burned-in text graphics or generating subtitles from speech.

## CEA-608/708 Closed Captions

### Extract Closed Captions

```bash
# Extract CEA-608 captions from ATSC stream
ffmpeg -f lavfi -i "movie=broadcast.ts[out0+subcc]" -map 0:1 captions.srt

# Extract from video with embedded CC
ffmpeg -i video_with_cc.mp4 \
  -filter_complex "[0:v]format=yuv420p[v];[0:v]crop=1:1:0:0[c]" \
  -map "[c]" -c:v libx264 -f null - 2>&1 | grep -A 1 "Closed caption"
```

### Add CEA-608 Captions

```bash
# Embed CEA-608 captions (requires eia608 line)
ffmpeg -i video.mp4 -i captions.scc \
  -c:v libx264 -c:a copy \
  -vf "movie=captions.scc[captions];[0:v][ca