Video Generation

What this skill does

# Video Generation Skill

This skill guides the implementation of video generation functionality using the z-ai-web-dev-sdk package, enabling AI models to create videos from text descriptions or images through asynchronous task processing.

## Skills Path

**Skill Location**: `{project_path}/skills/video-generation`

This skill is located at the above path in your project.

**Reference Scripts**: Example test scripts are available in the `{Skill Location}/scripts/` directory for quick testing and reference. See `{Skill Location}/scripts/video.ts` for a working example.

## Overview

Video Generation allows you to build applications that can create video content from text prompts or images, with customizable parameters like resolution, frame rate, duration, and quality settings. The API uses an asynchronous task model where you create a task and poll for results.

**IMPORTANT**: z-ai-web-dev-sdk MUST be used in backend code only. Never use it in client-side code.

## Prerequisites

The z-ai-web-dev-sdk package is already installed. Import it as shown in the examples below.

## CLI Usage (For Simple Tasks)

For simple video generation tasks, you can use the z-ai CLI instead of writing code. The CLI handles task creation and polling automatically, making it ideal for quick tests and simple automation.

### Basic Text-to-Video

```bash
# Generate video with automatic polling
z-ai video --prompt "A cat playing with a ball" --poll

# Using short options
z-ai video -p "Beautiful landscape with mountains" --poll
```

### Custom Quality and Settings

```bash
# Quality mode (speed or quality)
z-ai video -p "Ocean waves at sunset" --quality quality --poll

# Custom resolution and FPS
z-ai video \
  -p "City timelapse" \
  --size "1920x1080" \
  --fps 60 \
  --poll

# Custom duration (5 or 10 seconds)
z-ai video -p "Fireworks display" --duration 10 --poll
```

### Image-to-Video

**IMPORTANT**: For `image_url` parameter, it is **strongly recommended to use base64-encoded image data** instead of URLs. This approach is more reliable and avoids potential network issues or access restrictions.

**Note**: Match the MIME type in the data URI to your actual image format (image/jpeg, image/png, image/webp, etc.) to avoid decoding issues.

```bash
# Generate video from single image using base64 (RECOMMENDED)
# Convert your image to base64 with correct MIME type

# For PNG images
IMAGE_BASE64=$(base64 -i image.png)
z-ai video \
  --image-url "data:image/png;base64,${IMAGE_BASE64}" \
  --prompt "Make the scene come alive" \
  --poll

# For JPEG images
IMAGE_BASE64=$(base64 -i photo.jpg)
z-ai video \
  --image-url "data:image/jpeg;base64,${IMAGE_BASE64}" \
  --prompt "Make the scene come alive" \
  --poll

# For WebP images
IMAGE_BASE64=$(base64 -i image.webp)
z-ai video \
  --image-url "data:image/webp;base64,${IMAGE_BASE64}" \
  --prompt "Make the scene come alive" \
  --poll

# Using URL (less recommended, may have reliability issues)
z-ai video \
  -i "https://example.com/photo.jpg" \
  -p "Add motion to this scene" \
  --poll
```

### First-Last Frame Mode

**IMPORTANT**: For best reliability, use base64-encoded images instead of URLs. Ensure the MIME type matches your actual image format.

```bash
# Generate video between two frames using base64 (RECOMMENDED)
# Make sure to use the correct MIME type for each image

# Example with PNG images
START_BASE64=$(base64 -i start.png)
END_BASE64=$(base64 -i end.png)
z-ai video \
  --image-url "data:image/png;base64,${START_BASE64},data:image/png;base64,${END_BASE64}" \
  --prompt "Smooth transition between frames" \
  --poll

# Example with JPEG images
START_BASE64=$(base64 -i start.jpg)
END_BASE64=$(base64 -i end.jpg)
z-ai video \
  --image-url "data:image/jpeg;base64,${START_BASE64},data:image/jpeg;base64,${END_BASE64}" \
  --prompt "Smooth transition between frames" \
  --poll

# Using URLs (less recommended)
z-ai video \
  --image-url "https://example.com/start.png,https://example.com/end.png" \
  --prompt "Smooth transition between frames" \
  --poll
```

### With Audio Generation

```bash
# Generate video with AI-generated audio effects
z-ai video \
  -p "Thunder storm approaching" \
  --with-audio \
  --poll
```

### Save Output

```bash
# Save task result to JSON file
z-ai video \
  -p "Sunrise over mountains" \
  --poll \
  -o video_result.json
```

### Custom Polling Parameters

```bash
# Customize polling behavior
z-ai video \
  -p "Dancing robot" \
  --poll \
  --poll-interval 10 \
  --max-polls 30

# Create task without polling (get task ID)
z-ai video -p "Abstract art animation" -o task.json
```

### CLI Parameters

- `--prompt, -p <text>`: Optional - Text description of the video
- `--image-url, -i <data>`: Optional - **Preferably base64-encoded image data** (e.g., "data:image/png;base64,iVBORw..."). URLs are also supported but less recommended. For two images, use comma-separated values.
- `--quality, -q <mode>`: Optional - Output mode: `speed` or `quality` (default: speed)
- `--with-audio`: Optional - Generate AI audio effects (default: false)
- `--size, -s <resolution>`: Optional - Video resolution (e.g., "1920x1080")
- `--fps <rate>`: Optional - Frame rate: 30 or 60 (default: 30)
- `--duration, -d <seconds>`: Optional - Duration: 5 or 10 seconds (default: 5)
- `--model, -m <model>`: Optional - Model name to use
- `--poll`: Optional - Auto-poll until task completes
- `--poll-interval <seconds>`: Optional - Polling interval (default: 5)
- `--max-polls <count>`: Optional - Maximum poll attempts (default: 60)
- `--output, -o <path>`: Optional - Output file path (JSON format)

### Supported Resolutions

- `1024x1024`
- `768x1344`
- `864x1152`
- `1344x768`
- `1152x864`
- `1440x720`
- `720x1440`
- `1920x1080` (and other standard resolutions)

### Checking Task Status Later

If you create a task without `--poll`, you can check its status later:

```bash
# Get the task ID from the initial response
z-ai async-result --id "task-id-here" --poll
```

### When to Use CLI vs SDK

**Use CLI for:**
- Quick video generation tests
- Simple one-off video creation
- Command-line automation scripts
- Testing different prompts and settings

**Use SDK for:**
- Batch video generation with custom logic
- Integration with web applications
- Custom task queue management
- Production applications with complex workflows

## Video Generation Workflow

Video generation follows a two-step asynchronous pattern:

1. **Create Task**: Submit video generation request and receive a task ID
2. **Poll Results**: Query the task status until completion and retrieve the video URL

## Basic Video Generation Implementation

### Simple Text-to-Video Generation

```javascript
import ZAI from 'z-ai-web-dev-sdk';

async function generateVideo(prompt) {
  try {
    const zai = await ZAI.create();

    // Create video generation task
    const task = await zai.video.generations.create({
      prompt: prompt,
      quality: 'speed', // 'speed' or 'quality'
      with_audio: false,
      size: '1920x1080',
      fps: 30,
      duration: 5
    });

    console.log('Task ID:', task.id);
    console.log('Task Status:', task.task_status);

    // Poll for results
    let result = await zai.async.result.query(task.id);
    let pollCount = 0;
    const maxPolls = 60;
    const pollInterval = 5000; // 5 seconds

    while (result.task_status === 'PROCESSING' && pollCount < maxPolls) {
      pollCount++;
      console.log(`Polling ${pollCount}/${maxPolls}: Status is ${result.task_status}`);
      await new Promise(resolve => setTimeout(resolve, pollInterval));
      result = await zai.async.result.query(task.id);
    }

    if (result.task_status === 'SUCCESS') {
      // Get video URL from multiple possible fields
      const videoUrl = result.video_result?.[0]?.url ||
                      result.video_url ||
                      result.url ||
                      result.video;
      console.log('Video URL:', videoUrl);
      return videoUrl;
    } else {