rw-integrate-video

What this skill does

# Integrate Video Generation

> **PREREQUISITE:** Run `+rw-check-compatibility` first. Run `+rw-fetch-api-reference` to load the latest API reference before integrating. Requires `+rw-setup-api-key` for API credentials. Requires `+rw-integrate-uploads` when the user has local files to use as input.

Help users add Runway video generation to their server-side code.

## Available Models

| Model | Best For | Input | Cost | Speed |
|-------|----------|-------|------|-------|
| `seedance2` | Reference image and video, long duration | Text, Image, and/or Video | 36 credits/sec | Standard |
| `gen4.5` | High quality, general purpose | Text and/or Image | 12 credits/sec | Standard |
| `gen4_turbo` | Fast, image-driven | Image required | 5 credits/sec | Fast |
| `gen4_aleph` | Video editing/transformation | Video + Text/Image | 15 credits/sec | Standard |
| `veo3` | Premium Google model | Text/Image | 40 credits/sec | Standard |
| `veo3.1` | High quality Google model | Text/Image | 20-40 credits/sec | Standard |
| `veo3.1_fast` | Fast Google model | Text/Image | 10-15 credits/sec | Fast |

**Model selection guidance:**
- Default recommendation: **`gen4.5`** — best balance of quality and cost
- **Product ads / e-commerce:** **`seedance2`** — up to 15s, supports reference image and video
- Budget-conscious: **`gen4_turbo`** (requires image) or **`veo3.1_fast`**
- Highest quality: **`veo3`** (most expensive)
- Video-to-video editing: **`gen4_aleph`** or **`seedance2`**

## Security

`promptImage`, `promptVideo`, `videoUri`, and `references[].uri` are **fetched server-side by the Runway API** — treat them like any outbound fetch:

- **Prefer `runway://` URIs** from `+rw-integrate-uploads` — scoped to your account, no arbitrary web content.
- **If accepting URLs from clients**, validate first: require `https://`, allowlist trusted hosts, reject private addresses. See the Express.js example below.
- **Never forward `req.body.imageUrl`** (or similar) straight into `promptImage` / `promptVideo`. The SDK snippets below use raw URLs for brevity — they aren't production templates.
- Treat generated outputs as untrusted when piping into downstream automations — ingested media influences the result.

## Endpoints

### Text-to-Video: `POST /v1/text_to_video`

Generate video from a text prompt only.

**Compatible models:** `seedance2`, `gen4.5`, `veo3`, `veo3.1`, `veo3.1_fast`

```javascript
// Node.js SDK
import RunwayML from '@runwayml/sdk';

const client = new RunwayML();

const task = await client.textToVideo.create({
  model: 'gen4.5',
  promptText: 'A golden retriever running through a field of wildflowers at sunset',
  ratio: '1280:720',
  duration: 5
}).waitForTaskOutput();

// task.output is an array of signed URLs
const videoUrl = task.output[0];
```

```python
# Python SDK
from runwayml import RunwayML

client = RunwayML()

task = client.text_to_video.create(
    model='gen4.5',
    prompt_text='A golden retriever running through a field of wildflowers at sunset',
    ratio='1280:720',
    duration=5
).wait_for_task_output()

video_url = task.output[0]
```

### Image-to-Video: `POST /v1/image_to_video`

Animate a still image into a video.

**Compatible models:** `seedance2`, `gen4.5`, `gen4_turbo`, `veo3`, `veo3.1`, `veo3.1_fast`

**Recommended:** upload via `+rw-integrate-uploads` and pass the returned `runway://` URI.

```javascript
// Node.js SDK — preferred flow
import fs from 'fs';

const upload = await client.uploads.createEphemeral(
  fs.createReadStream('/path/to/image.jpg')
);

const task = await client.imageToVideo.create({
  model: 'gen4.5',
  promptImage: upload.runwayUri,
  promptText: 'The scene comes to life with gentle wind',
  ratio: '1280:720',
  duration: 5
}).waitForTaskOutput();
```

External URLs also work — only pass origins you control (see Security):

```javascript
const task = await client.imageToVideo.create({
  model: 'gen4.5',
  promptImage: 'https://cdn.yourapp.com/landscape.jpg',
  promptText: 'Camera slowly pans right revealing a mountain range',
  ratio: '1280:720',
  duration: 5
}).waitForTaskOutput();
```

```python
# Python SDK
task = client.image_to_video.create(
    model='gen4.5',
    prompt_image='https://cdn.yourapp.com/landscape.jpg',
    prompt_text='Camera slowly pans right revealing a mountain range',
    ratio='1280:720',
    duration=5
).wait_for_task_output()
```

### Video-to-Video: `POST /v1/video_to_video`

Transform an existing video with a text prompt and/or reference image.

**Compatible models:** `gen4_aleph`, `seedance2`

```javascript
// Node.js SDK — gen4_aleph
const task = await client.videoToVideo.create({
  model: 'gen4_aleph',
  videoUri: 'https://cdn.yourapp.com/source.mp4',
  promptText: 'Transform into an animated cartoon style',
}).waitForTaskOutput();
```

```javascript
// Node.js SDK — seedance2 video-to-video (with optional image reference)
const task = await client.videoToVideo.create({
  model: 'seedance2',
  promptVideo: 'https://cdn.yourapp.com/input.mp4',
  promptText: 'Transform into a warm golden sunset scene',
  references: [{ type: 'image', uri: 'https://cdn.yourapp.com/style_ref.jpg' }]
}).waitForTaskOutput();
```

> **seedance2 VTV input requirements:** max 15 seconds, max 32 MB, min 720p resolution, MP4 recommended.

### Seedance 2

Seedance 2 supports text-to-video, image-to-video (two modes), and video-to-video. It uses pixel-based ratios: `1280:720`, `720:1280`, `960:960`, `1112:834`, `834:1112`, `1470:630`, `992:432`, `864:496`, `752:560`, `640:640`, `560:752`, `496:864`.

#### Text-to-Video

```javascript
const task = await client.textToVideo.create({
  model: 'seedance2',
  promptText: 'A calm ocean wave gently crashing on a sandy beach at sunset',
  duration: 5,
  ratio: '1280:720'
}).waitForTaskOutput();
```

#### Image-to-Video — Mode 1: First / Last Frame

Use a specific image as the first and/or last frame. The `references` field **cannot** be used in this mode.

```javascript
const task = await client.imageToVideo.create({
  model: 'seedance2',
  promptText: 'Smooth transition from day to night in a cozy mountain cabin',
  promptImage: [
    { uri: 'https://cdn.yourapp.com/image.jpg', position: 'first' },
    { uri: 'https://cdn.yourapp.com/image2.jpg', position: 'last' }
  ],
  duration: 4,
  ratio: '1280:720'
}).waitForTaskOutput();
```

`promptImage` is an array of objects with `uri` (required) and `position` (`"first"` or `"last"`, defaults to first).

#### Image-to-Video — Mode 2: Image Reference

Use an image as a stylistic/content reference rather than a literal frame. `promptImage` is still required (as a URI string or single-item array).

```javascript
const task = await client.imageToVideo.create({
  model: 'seedance2',
  promptText: 'Smooth transition from day to night in a cozy mountain cabin',
  promptImage: 'https://cdn.yourapp.com/image.jpg',
  references: [{ type: 'image', uri: 'https://cdn.yourapp.com/reference.jpg' }],
  duration: 4,
  ratio: '1280:720'
}).waitForTaskOutput();
```

> These two ITV modes are **mutually exclusive** — you cannot use `position` in `promptImage` and `references` in the same request.

#### Video-to-Video

Transform an existing video guided by a text prompt, optionally with an image reference.

```python
task = client.video_to_video.create(
    model='seedance2',
    prompt_video='https://cdn.yourapp.com/input.mp4',
    prompt_text='Transform into a warm golden sunset scene',
    references=[{'type': 'image', 'uri': 'https://cdn.yourapp.com/style_ref.jpg'}]
).wait_for_task_output()
```

> **VTV input requirements:** max 15 seconds, max 32 MB, min 720p resolution, MP4 recommended.

#### Seedance 2 Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `model` | string | Yes | Must be `"seedance2"` |
| `promptText` | string | Yes | Text description of the desired video |
| `duration` | number | Yes (TTV/ITV) | Duration in seconds |
| `ratio` | string | Yes (TTV/ITV) | `1280:7