remotion-best-practices

What this skill does

## When to use

Use this skill for the complete video production pipeline — from script writing through rendering and delivery. It includes scaffolding automation, voiceover/caption/music generation scripts, and detailed rule files for every Remotion pattern you'll need.

## Prerequisites

You need a Remotion project. If you don't have one, create it with `npx create-video@latest` and select the **blank template**. The project must have a `remotion.config.ts` at its root — the scaffold script uses this to locate the project.

Required `.env` variables in the project root:
- `ELEVENLABS_API_KEY` — for voiceover and background music generation
- `ELEVENLABS_VOICE_ID` — default voice (optional if set per-project in project.json)
- `KREA_API_KEY` — for b-roll generation (if using Krea)

On first use, save the Remotion project location to memory so you can find it in future sessions.

## Scene planning

Before implementing any scenes, follow this sequence.

**API keys** (ElevenLabs, Krea, etc.) are loaded from `.env` by the scripts automatically. NEVER pass API keys on the command line — they will be visible in terminal output. Always use the provided scripts:
- `npx tsx src/<Name>/generate-voiceover.ts` — generates voiceover audio (reads scenes from project.json)
- `npx tsx src/<Name>/generate-captions.ts` — transcribes voiceover to captions (reads scene IDs from project.json)
- `npx tsx src/<Name>/generate-background-music.ts` — generates background music (reads mood from project.json)
- `npx tsx src/generate-broll.ts --output "public/<name>/broll/" --prompts prompts.json` — generates b-roll clips

### Content safe zone (default)

All main content (headlines, key text, CTAs, stats, graphics) must be **centered vertically and horizontally**, building outward from dead center. This is the default unless explicitly overridden.

B-roll, backgrounds, decorative elements, and captions are NOT restricted to the safe zone — they fill the full frame.

If the prompt doesn't specify landscape, default to portrait (9:16).

**Portrait (9:16) — 1080x1920 canvas:**

Universal cross-platform safe zone (works on TikTok, Reels, and Shorts):
- **Top:** 210px (status bar, search, platform header)
- **Bottom:** 320px (action buttons, captions, CTA overlays)
- **Left:** 60px
- **Right:** 120px (action icons on TikTok/Reels)
- **Safe area:** 900x1400px centered

Platform UI is updated frequently — these values are current as of early 2026.

**Landscape (16:9):** Full frame is usable, no safe zone restriction.

**First frame rule:** Scene 1's first animation must start at frame 0 with no delay AND start at visible values (e.g. scale 0.8, opacity 0.5). Spring animations start from 0 and take several frames to reach visible values — if starting from scale 0 or opacity 0, the first frames are blank which looks broken as a thumbnail and on autoplay.

### Step 1: Scaffold the composition
Run the scaffold script from the Remotion project root. The script is located in the plugin's `scripts/` directory:

```bash
python3 <path-to-plugin>/scripts/scaffold.py <CompositionName>
```

This automatically creates the full composition structure:
- `src/<Name>/` — index.tsx, Scene1.tsx (placeholder), get-audio-duration.ts, generate-voiceover.ts, generate-captions.ts, generate-background-music.ts, Captions.tsx, project.json
- `public/<name>/voiceover/`, `public/<name>/broll/`, `public/<name>/captions/`
- Registers the composition in `src/Root.tsx`

After scaffolding, **start (or restart) Remotion Studio** so it picks up the new composition:

```bash
npx remotion studio
```

The placeholder scene is immediately previewable. Do NOT manually create these files — use the script.

The scaffold creates `project.json` in the composition directory. This file is the **single source of truth** — all scripts read from it, and you update it at every step. Every creative decision, every configuration value, every approval status goes into this file.

### Step 2: Script and questionnaire
First, decide how many scenes the video needs (typically 3-5). Then write the voiceover script for all scenes. Audio durations drive scene lengths (not the other way around).

When presenting the script for approval, ALWAYS show for every scene:
- Headline (what appears on screen)
- Voiceover (exact words the narrator says)
- Visual description

After the script, ask the user:
- What's your company/brand name and website? (for captions proofreading and on-screen text)
- What voice should we use? (provide the ElevenLabs voice ID, or leave blank to use ELEVENLABS_VOICE_ID from .env)
- Background music? (yes/no, mood: upbeat/corporate/cinematic/ambient)
- Auto-render when complete? (defaults to no)
- Publishing to YouTube? → if yes:
  - What links should go in the description? (website, socials, relevant pages)
  - Tags and category?
  - Auto-publish or manual approval? (defaults to manual)

Use these exact values for approval gates: `"pending"` (wait for approval), `"approved"` (auto-proceed), `"completed"` (done). Do not use booleans or yes/no.

**Update project.json:** Fill in:
- `branding.company`, `branding.website`, `branding.socials` — from the user's answers
- `voice.voice_id` — the user's chosen voice ID (empty = use env var)
- `background_music.enabled` and `background_music.mood` — from the user's answer
- `render` — "pending" or "approved"
- `youtube.publish` — "pending" or "approved" (only if publishing to YouTube)
- `youtube.links`, `youtube.tags`, `youtube.category` — only if publishing to YouTube
- Each scene's `headline`, `voiceover`, and `visual` fields

If the scene count changed, duplicate scene entries in the `scenes` array to match. Also duplicate the placeholder Scene1.tsx for each additional scene (rename the component export to Scene2, Scene3, etc.) and update index.tsx to import all scenes and wire them into the TransitionSeries with fade transitions between each. Load [./rules/compositions.md](./rules/compositions.md) for composition patterns and [./rules/transitions.md](./rules/transitions.md) for the TransitionSeries wiring pattern.

### Step 3: Voiceover generation
The voiceover script reads directly from `project.json` — no need to edit the script itself. Just make sure the voiceover text is filled in for each scene in project.json (Step 2), then run:

```bash
npx tsx src/<Name>/generate-voiceover.ts
```

After voiceover files are generated, **update `index.tsx`**: replace the placeholder `calculateMetadata` with one that reads actual audio durations using `getAudioDuration`. The composition duration should be driven by the voiceover audio, not hardcoded frame counts. Load [./rules/voiceover.md](./rules/voiceover.md) for the exact calculateMetadata pattern.

### Step 4: B-roll
Decide which scenes get b-roll backgrounds. Allocate ~2 b-roll clips per 30 seconds of video. Any scene can have b-roll — it's a background layer independent of the foreground content (text, charts, animated diagrams, anything).

**Update project.json:** For each scene, fill in `broll.type` ("image", "video", or "none") and `broll.prompt`.

Then generate the b-roll clips. Create a prompts JSON file from the project.json b-roll entries and run:

```bash
npx tsx src/generate-broll.ts --output "public/<name>/broll/" --prompts prompts.json
```

The prompts.json format: `[{"name": "scene-02", "prompt": "description..."}, ...]`

The script submits jobs to Krea.ai, polls until complete, downloads the clips, and extends them to 15s with ffmpeg.

When coding scenes with b-roll, you MUST follow the compositing pattern in [./rules/b-roll.md](./rules/b-roll.md):
- B-roll layer: `AbsoluteFill` with `filter: "brightness(0.35)"` and `overflow: "hidden"`, OffthreadVideo with Ken Burns bounce zoom
- Content layer: `AbsoluteFill` with safe zone padding and `justifyContent: "center"`
- All text/elements over b-roll MUST have dark semi-transparent card backgrounds (`backgroundColor: "rgba(10,10,10,0.7)"`) for readability

### Step 5: Transit