happy-horse-video-generator

What this skill does

```markdown
---
name: happy-horse-video-generator
description: AI coding agent skill for Happy Horse 1.0 — native joint audio-video generation model with unified Transformer architecture
triggers:
  - generate video with Happy Horse
  - use Happy Horse AI video model
  - happy horse video generation
  - joint audio video generation happy horse
  - run happy horse inference
  - happy horse text to video
  - happy horse image to video
  - happyhorses.io model
---

# Happy Horse 1.0 — AI Video Generator

> Skill by [ara.so](https://ara.so) — Daily 2026 Skills collection.

## ⚠️ Release Status

Happy Horse 1.0 has **not yet been officially open-sourced** as of this skill's writing. Model weights, inference code, and the official repository have not been published. This skill covers:

- Reported architecture and capabilities (community-compiled, unverified)
- Anticipated usage patterns based on similar open-source video models (Wan 2.2, HunyuanVideo, LTX-2)
- Scaffolding code to wire up Happy Horse once weights/code drop
- Pointers to official resources

**Official resources:**
- Homepage & demo: https://happyhorses.io/
- Leaderboard context: https://artificialanalysis.ai/video/arena

---

## What Happy Horse 1.0 Does

Happy Horse 1.0 is a ~15B-parameter unified self-attention Transformer that generates **video frames and audio together in a single forward pass** — no silent video + separate dubbing pipeline required.

Key reported capabilities:
- **Native joint audio-video generation** (dialogue, Foley, ambient sound)
- **1080p output** in ~38 seconds on NVIDIA H100 (distilled model)
- **8 denoising steps, no classifier-free guidance** (DMD-2 distillation)
- **Text-to-video and image-to-video** via a single set of weights
- **Native lip-sync** in English, Mandarin Chinese, Japanese, Korean, German, French

Architectural highlights:
- 40 layers total: first 4 + last 4 are modality-specific; middle 32 share parameters across text/image/video/audio
- Per-attention-head learned scalar sigmoid gates for multimodal training stability
- No explicit timestep embeddings — noise level inferred from noisy latents
- MagiCompiler runtime for full-graph operator fusion (~1.2× speedup)

---

## Installation (Anticipated — Update When Official Repo Releases)

Watch https://github.com/brooks376/Happy-Horse-1.0 and https://happyhorses.io/ for the official release. The following scaffolding mirrors conventions used by comparable models (HunyuanVideo, Wan 2.2).

```bash
# Clone the official repo once released
git clone https://github.com/<official-org>/happy-horse-1.0
cd happy-horse-1.0

# Create isolated environment
python -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\activate

# Install dependencies (anticipated)
pip install -r requirements.txt

# Download model weights (anticipated — likely via huggingface_hub)
pip install huggingface_hub
python -c "
from huggingface_hub import snapshot_download
snapshot_download(
    repo_id='<official-org>/happy-horse-1.0',
    local_dir='./weights/happy-horse-1.0'
)
"
```

### Environment variables

```bash
# Set before running inference
export HF_TOKEN=your_huggingface_token          # if weights are gated
export HAPPY_HORSE_MODEL_PATH=./weights/happy-horse-1.0
export CUDA_VISIBLE_DEVICES=0                   # target GPU index
```

---

## Hardware Requirements

| Task | VRAM | Notes |
|---|---|---|
| 1080p generation (distilled) | 80 GB | NVIDIA H100 reference |
| 720p generation | ~40 GB (estimated) | A100 80GB likely sufficient |
| 256p preview | ~16 GB (estimated) | ~2 sec on H100 |

For multi-GPU setups, model parallelism will likely follow the same pattern as HunyuanVideo:

```bash
# Anticipated multi-GPU launch
torchrun --nproc_per_node=4 generate.py \
  --model_path $HAPPY_HORSE_MODEL_PATH \
  --prompt "A galloping horse across open plains at sunset" \
  --resolution 1080p \
  --steps 8
```

---

## Key Commands (Anticipated CLI)

Based on comparable open-source video model CLIs (Wan 2.2, HunyuanVideo):

```bash
# Text-to-video (distilled, 8 steps, no CFG)
python generate.py \
  --task t2v \
  --model_path $HAPPY_HORSE_MODEL_PATH \
  --prompt "A horse galloping through a misty forest, hooves thundering on wet earth" \
  --resolution 1080p \
  --duration 5 \
  --steps 8 \
  --output ./output/horse_forest.mp4

# Image-to-video
python generate.py \
  --task i2v \
  --model_path $HAPPY_HORSE_MODEL_PATH \
  --image ./reference.png \
  --prompt "The horse rears up and gallops away into the distance" \
  --resolution 1080p \
  --duration 5 \
  --steps 8 \
  --output ./output/horse_i2v.mp4

# Base model (50 steps, CFG — higher quality, slower)
python generate.py \
  --task t2v \
  --model_path $HAPPY_HORSE_MODEL_PATH \
  --variant base \
  --prompt "..." \
  --resolution 1080p \
  --steps 50 \
  --cfg_scale 7.5 \
  --output ./output/horse_base.mp4
```

---

## Python API (Anticipated Usage Patterns)

### Text-to-Video

```python
import os
import torch
from happy_horse import HappyHorsePipeline  # module name TBC at release

def generate_video_from_text(
    prompt: str,
    output_path: str,
    resolution: str = "1080p",
    duration_sec: int = 5,
    steps: int = 8,
    seed: int = 42,
) -> str:
    """
    Generate a video (with native audio) from a text prompt.
    Uses the distilled 8-step model by default.
    """
    model_path = os.environ["HAPPY_HORSE_MODEL_PATH"]

    pipe = HappyHorsePipeline.from_pretrained(
        model_path,
        torch_dtype=torch.bfloat16,
        variant="distilled",  # or "base" for full 50-step model
    )
    pipe = pipe.to("cuda")

    result = pipe(
        prompt=prompt,
        resolution=resolution,
        duration=duration_sec,
        num_inference_steps=steps,
        guidance_scale=1.0,   # CFG not required for distilled model
        generator=torch.Generator("cuda").manual_seed(seed),
    )

    result.save(output_path)
    print(f"Saved video to {output_path}")
    return output_path


if __name__ == "__main__":
    generate_video_from_text(
        prompt="A joyful horse running along a beach, waves crashing, seagulls calling",
        output_path="./output/beach_horse.mp4",
    )
```

### Image-to-Video

```python
import os
import torch
from PIL import Image
from happy_horse import HappyHorsePipeline

def generate_video_from_image(
    image_path: str,
    prompt: str,
    output_path: str,
    resolution: str = "1080p",
    duration_sec: int = 5,
    steps: int = 8,
    seed: int = 42,
) -> str:
    """
    Animate a reference image into video with native audio.
    """
    model_path = os.environ["HAPPY_HORSE_MODEL_PATH"]

    pipe = HappyHorsePipeline.from_pretrained(
        model_path,
        torch_dtype=torch.bfloat16,
        variant="distilled",
    )
    pipe = pipe.to("cuda")

    reference_image = Image.open(image_path).convert("RGB")

    result = pipe(
        image=reference_image,
        prompt=prompt,
        resolution=resolution,
        duration=duration_sec,
        num_inference_steps=steps,
        guidance_scale=1.0,
        generator=torch.Generator("cuda").manual_seed(seed),
    )

    result.save(output_path)
    return output_path


if __name__ == "__main__":
    generate_video_from_image(
        image_path="./horse_portrait.png",
        prompt="The horse turns its head, snorts, and trots away",
        output_path="./output/horse_portrait_anim.mp4",
    )
```

### Extracting Audio and Video Separately

```python
import os
from happy_horse import HappyHorsePipeline
import torch

def generate_and_split_av(
    prompt: str,
    video_output: str = "./output/video_only.mp4",
    audio_output: str = "./output/audio_only.wav",
) -> tuple[str, str]:
    """
    Generate joint audio-video and export video and audio as separate files.
    Useful when you need the audio track independently (e.g. for further processing).
    """
    model_path = os.environ["HAPPY_HORSE_MODEL_PATH"]

    pipe = HappyHorsePipeline.from_pretrained(