video-use-editor

What this skill does

```markdown
---
name: video-use-editor
description: Edit videos with AI coding agents using Claude Code, ffmpeg, and ElevenLabs transcription
triggers:
  - edit this video
  - cut out filler words
  - add subtitles to my video
  - color grade my footage
  - make a highlight reel
  - trim my talking head video
  - assemble these clips into a final video
  - add animations to my video
---

# video-use: AI Video Editing Skill

> Skill by [ara.so](https://ara.so) — Daily 2026 Skills collection.

**video-use** lets AI coding agents edit video like a human editor — reading transcripts and timelines as structured text rather than processing raw frames. Drop raw footage in a folder, describe the edit, get `final.mp4` back.

---

## What video-use does

- **Cuts filler words** (`umm`, `uh`, false starts, dead air) using word-level timestamps
- **Color grades** every segment with ffmpeg filter chains (cinematic warm, neutral punch, or custom)
- **Burns subtitles** — 2-word UPPERCASE chunks by default, fully customizable
- **Generates animation overlays** via Manim, Remotion, or PIL in parallel sub-agents
- **Self-evaluates** rendered output at every cut boundary before showing you anything
- **Persists session memory** in `project.md` so future sessions pick up where you left off

The LLM never watches the video. It reads it through:
1. **Audio transcript** (ElevenLabs Scribe) — word-level timestamps, speaker diarization, audio events
2. **Visual composite on demand** — filmstrip + waveform + word labels PNG for ambiguous decisions only

---

## Installation

### Automated (paste into any coding agent)

```text
Set up https://github.com/browser-use/video-use for me.

Read install.md first to install this repo, wire up ffmpeg, register the skill with whichever agent you're running under, and set up the ElevenLabs API key — ask me to paste it when you need it. Then read SKILL.md for daily usage, and always read helpers/ because that's where the editing scripts live. After install, don't transcribe anything on your own — just tell me it's ready and wait for me to drop footage into a folder.
```

### Manual

```bash
# Clone and symlink into your agent's skills directory
git clone https://github.com/browser-use/video-use ~/Developer/video-use
ln -sfn ~/Developer/video-use ~/.claude/skills/video-use        # Claude Code
# ln -sfn ~/Developer/video-use ~/.codex/skills/video-use       # Codex

# Install Python dependencies
cd ~/Developer/video-use
uv sync                         # or: pip install -e .

# Install system dependencies
brew install ffmpeg              # required
brew install yt-dlp             # optional, for online sources

# Set up environment
cp .env.example .env
# Edit .env and add: ELEVENLABS_API_KEY=your_key_here
```

Get an ElevenLabs API key at [elevenlabs.io/app/settings/api-keys](https://elevenlabs.io/app/settings/api-keys).

---

## Environment Configuration

```bash
# .env file
ELEVENLABS_API_KEY=your_key_here    # Required for transcription
```

---

## Starting an editing session

```bash
cd /path/to/your/raw/footage
claude    # or: codex, hermes, etc.
```

Then in the session, describe what you want:

```
edit these into a launch video
```

```
cut out all the umms and uhs, keep takes under 30 seconds
```

```
make a 60-second highlight reel from the best moments
```

The agent will:
1. Inventory source files
2. Propose an editing strategy
3. Wait for your approval
4. Produce `edit/final.mp4` next to your sources

All outputs live in `<videos_dir>/edit/` — the skill directory stays clean.

---

## Pipeline

```
Transcribe ──> Pack ──> LLM Reasons ──> EDL ──> Render ──> Self-Eval
                                                              │
                                                              └─ issue? fix + re-render (max 3)
```

### Step 1: Transcription

ElevenLabs Scribe produces word-level timestamps per source file, packed into `takes_packed.md`:

```markdown
## C0103  (duration: 43.0s, 8 phrases)
  [002.52-005.36] S0 Ninety percent of what a web agent does is completely wasted.
  [006.08-006.74] S0 We fixed this.
  [007.10-009.80] S0 Uh — (pause) — let me show you what I mean.
```

~12KB of text replaces 45M tokens of frame analysis.

### Step 2: Edit Decision List (EDL)

The agent produces a structured EDL before touching any files:

```python
# Example EDL structure the agent reasons over
edl = [
    {
        "source": "C0103.mp4",
        "in":  2.52,
        "out": 5.36,
        "color_grade": "warm_cinematic",
        "audio_fade_ms": 30,
    },
    {
        "source": "C0103.mp4",
        "in":  6.08,
        "out": 6.74,
        "color_grade": "warm_cinematic",
        "audio_fade_ms": 30,
    },
]
```

### Step 3: Render via helpers/

The `helpers/` directory contains the ffmpeg scripts the agent calls. Always read this directory — it's where editing logic lives.

---

## Key helpers and scripts

### timeline_view — visual composite on demand

Called only at decision points (ambiguous pauses, retake comparisons, cut sanity checks):

```python
# helpers/timeline_view.py
# Produces: filmstrip + speaker track + waveform + word labels PNG
# Args: source file, start_time, end_time
python helpers/timeline_view.py C0103.mp4 2.0 10.0
# -> edit/timeline_C0103_2.0-10.0.png
```

### Cutting with ffmpeg (what the agent generates)

```bash
# Single segment cut with color grade and audio fade
ffmpeg -i C0103.mp4 \
  -ss 2.52 -to 5.36 \
  -vf "curves=vintage,fade=t=out:st=2.8:d=0.03:alpha=0" \
  -af "afade=t=in:st=0:d=0.03,afade=t=out:st=2.8:d=0.03" \
  -c:v libx264 -c:a aac \
  edit/seg_001.mp4
```

### Concatenating segments

```bash
# helpers/concat.py generates this automatically
ffmpeg -f concat -safe 0 -i edit/segments.txt -c copy edit/final_raw.mp4
```

### Burning subtitles

```bash
# 2-word UPPERCASE chunks, customizable via --style
python helpers/burn_subtitles.py \
  --input edit/final_raw.mp4 \
  --transcript edit/transcript.json \
  --style uppercase_2word \
  --output edit/final.mp4
```

### Color grade presets

```python
# helpers/color_grades.py
GRADES = {
    "warm_cinematic": "curves=vintage,colorbalance=rs=0.1:gs=0:bs=-0.1",
    "neutral_punch":  "eq=contrast=1.1:saturation=1.05:brightness=0.02",
    "cool_clean":     "colorbalance=rs=-0.05:gs=0:bs=0.1,curves=lighter",
    "raw":            None,   # pass-through
}
```

---

## Real code examples

### Transcribing a source file

```python
import os
from elevenlabs import ElevenLabs

client = ElevenLabs(api_key=os.environ["ELEVENLABS_API_KEY"])

with open("C0103.mp4", "rb") as f:
    transcript = client.speech_to_text.convert(
        file=f,
        model_id="scribe_v1",
        diarize=True,
        timestamps_granularity="word",
    )

# Word-level output
for word in transcript.words:
    print(f"[{word.start:.2f}-{word.end:.2f}] {word.text}")
```

### Finding filler words to cut

```python
FILLERS = {"umm", "uh", "um", "uhh", "hmm", "like", "you know"}

def find_filler_cuts(transcript_words):
    cuts = []
    for i, word in enumerate(transcript_words):
        if word.text.lower().strip(",.") in FILLERS:
            # Merge with surrounding silence if gap < 0.3s
            cut_start = word.start
            cut_end = word.end
            if i + 1 < len(transcript_words):
                gap = transcript_words[i + 1].start - word.end
                if gap < 0.3:
                    cut_end = transcript_words[i + 1].start
            cuts.append({"start": cut_start, "end": cut_end, "reason": word.text})
    return cuts
```

### Building a concat list from EDL

```python
import subprocess

def render_segment(source, t_in, t_out, grade, index, output_dir="edit"):
    vf = grade or "null"
    out_path = f"{output_dir}/seg_{index:03d}.mp4"
    cmd = [
        "ffmpeg", "-y",
        "-i", source,
        "-ss", str(t_in),
        "-to", str(t_out),
        "-vf", vf,
        "-af", f"afade=t=in:st=0:d=0.03,afade=t=out:st={t_out - t_in - 0.03:.3f}:d=0.03",