acquire

What this skill does

# /acquire

Download media from discovered sources with intelligent format selection, parallel execution, and comprehensive progress tracking.

## Purpose

The `/acquire` command orchestrates media downloads from various sources (YouTube, Internet Archive, Bandcamp, direct links) using the Acquisition Manager agent. It handles:

- Single URL or batch downloads from source plans
- Automatic format selection based on content type
- Parallel download execution with resource limits
- Real-time progress tracking and status reporting
- Error recovery with retry strategies
- Quality verification and metadata recording
- Network mount awareness for optimal performance

## Parameters

### Required (one of)

**`--plan <sources.yaml>`**
- Source plan file generated by `/find-sources` command
- YAML format containing URLs, metadata, and acquisition strategy
- Example: `.curator/sources/plan-001.yaml`

**`--url <URL>`**
- Single URL for one-off downloads
- Supports: YouTube, Internet Archive, Bandcamp, SoundCloud, Vimeo, direct links
- Example: `https://youtube.com/watch?v=abc123`

### Optional

**`--format <audio|video|best>`**
- Format preference for downloads
- `audio`: Extract audio only (Opus 128K default)
- `video`: Best video up to 1080p with audio
- `best`: Let agent decide based on content type (default)
- Example: `--format audio`

**`--output <directory>`**
- Base output directory for downloads
- Default: `./downloads/<timestamp>`
- Structure: `<output>/<artist>/<era>/audio/` and `/video/`
- Example: `--output /mnt/archive/music`

**`--parallel <N>`**
- Maximum concurrent downloads
- Range: 1-5 (default: 3)
- Lower for slow networks, higher for fast connections
- Example: `--parallel 5`

**`--local-work <directory>`**
- Local working directory (required for network mount outputs)
- Downloads happen here first, then copied to output
- Default: `/tmp/curator-work-$$`
- Example: `--local-work /fast-local-disk/tmp`

**`--verify-after`**
- Enable post-download verification
- Checks file integrity, size, media validity
- Adds ~5% overhead but prevents corruption
- Example: `--verify-after`

**`--extract-audio`**
- Automatically extract audio from downloaded videos
- Creates parallel audio files in audio/ directory
- Uses Opus 128K by default
- Example: `--extract-audio`

**`--resume <session-id>`**
- Resume interrupted download session
- Loads state from `.curator/sessions/<session-id>/state.json`
- Skips completed downloads, retries failed
- Example: `--resume 20260214-143022`

## Usage Examples

### Single URL Download

```bash
# Download single YouTube video (audio only)
/acquire --url "https://youtube.com/watch?v=abc123" --format audio

# Download single video with metadata
/acquire --url "https://youtube.com/watch?v=abc123" \
  --format video \
  --output /mnt/archive/concerts \
  --verify-after
```

### Batch Download from Plan

```bash
# Download entire source plan (generated by /find-sources)
/acquire --plan .curator/sources/plan-001.yaml

# Download to network mount (uses local working dir)
/acquire --plan .curator/sources/plan-001.yaml \
  --output /mnt/network-storage/archive \
  --local-work /fast-ssd/curator-work \
  --parallel 3
```

### Audio Extraction Workflow

```bash
# Download videos and auto-extract audio
/acquire --plan .curator/sources/plan-001.yaml \
  --format video \
  --extract-audio \
  --verify-after

# Result structure:
# downloads/artist/era/video/concert.mkv
# downloads/artist/era/audio/concert.opus
```

### Resume Interrupted Session

```bash
# Resume after network failure
/acquire --resume 20260214-143022

# Resume with different output (move completed files)
/acquire --resume 20260214-143022 \
  --output /new/location
```

## Workflow

### 1. Initialization

```bash
# Validate parameters
- Check --plan exists OR --url provided
- Verify output directory is writable
- Check required tools (yt-dlp, wget, curl, ffmpeg)
- Test network mount performance if applicable
- Create session directory structure

# Session setup
SESSION_ID=$(date +%Y%m%d-%H%M%S)
SESSION_DIR=".curator/sessions/$SESSION_ID"
mkdir -p "$SESSION_DIR"/{logs,metadata}

# Initialize state file
cat > "$SESSION_DIR/state.json" <<EOF
{
  "session_id": "$SESSION_ID",
  "started_at": "$(date -Iseconds)",
  "status": "initializing",
  "parameters": {
    "plan": "$PLAN_FILE",
    "format": "$FORMAT",
    "output": "$OUTPUT_DIR",
    "parallel": $PARALLEL
  },
  "downloads": []
}
EOF
```

### 2. Source Validation

```bash
# If --plan provided, parse YAML
if [[ -n "$PLAN_FILE" ]]; then
  # Extract URLs and metadata
  SOURCES=$(yq eval '.sources[] | .url' "$PLAN_FILE")
  TOTAL_SOURCES=$(echo "$SOURCES" | wc -l)

  # Validate each URL is accessible
  while IFS= read -r url; do
    if yt-dlp --dump-json "$url" >/dev/null 2>&1; then
      echo "VALID: $url"
    else
      echo "WARNING: Cannot access $url"
    fi
  done <<< "$SOURCES"
fi

# If --url provided, validate single URL
if [[ -n "$SINGLE_URL" ]]; then
  if ! yt-dlp --dump-json "$SINGLE_URL" >/dev/null 2>&1; then
    echo "ERROR: Cannot access URL: $SINGLE_URL"
    exit 1
  fi
fi
```

### 3. Directory Structure Creation

```bash
create_acquisition_structure() {
  local output_base="$1"
  local artist="$2"
  local era="$3"

  # Sanitize directory names
  local safe_artist=$(echo "$artist" | sed 's/[^a-zA-Z0-9_-]/_/g')
  local safe_era=$(echo "$era" | sed 's/[^a-zA-Z0-9_-]/_/g')

  # Create directory tree
  local audio_dir="$output_base/$safe_artist/$safe_era/audio"
  local video_dir="$output_base/$safe_artist/$safe_era/video"

  mkdir -p "$audio_dir/.curator"
  mkdir -p "$video_dir/.curator"
  mkdir -p "$output_base/$safe_artist/.curator"

  # Write artist metadata
  cat > "$output_base/$safe_artist/.curator/artist-info.json" <<EOF
{
  "name": "$artist",
  "era": "$era",
  "created_at": "$(date -Iseconds)",
  "session_id": "$SESSION_ID"
}
EOF

  echo "$audio_dir:$video_dir"
}
```

### 4. Launch Downloads

```bash
# Download orchestration with concurrency control
ACTIVE_DOWNLOADS=()
MAX_CONCURRENT=${PARALLEL:-3}

launch_download() {
  local url="$1"
  local output_dir="$2"
  local format="$3"
  local download_id="dl-$SESSION_ID-$(date +%s)"

  # Wait for slot availability
  while [[ ${#ACTIVE_DOWNLOADS[@]} -ge $MAX_CONCURRENT ]]; do
    check_completed_downloads
    sleep 2
  done

  # Determine target directory (audio vs video)
  local target_dir
  if [[ "$format" == "audio" || "$format" == "bestaudio" ]]; then
    target_dir="$output_dir/audio"
  else
    target_dir="$output_dir/video"
  fi

  # Launch download in background
  (
    download_with_retry "$url" "$target_dir" "$format" "$download_id"
    echo "COMPLETED:$download_id:$?" >> "$SESSION_DIR/completion.log"
  ) &

  local pid=$!
  ACTIVE_DOWNLOADS+=("$download_id:$pid")

  # Update state file
  update_state_file "add" "$download_id" "$url" "in_progress"
}

download_with_retry() {
  local url="$1"
  local target_dir="$2"
  local format="$3"
  local download_id="$4"

  local attempt=0
  local max_attempts=3

  while [[ $attempt -lt $max_attempts ]]; do
    attempt=$((attempt + 1))

    echo "[$download_id] Attempt $attempt/$max_attempts"

    if yt-dlp -f "$format" \
        --output "$target_dir/%(title)s.%(ext)s" \
        --write-info-json \
        --write-thumbnail \
        --no-playlist \
        "$url" \
        2>&1 | tee "$SESSION_DIR/logs/$download_id.log"; then

      update_state_file "complete" "$download_id" "" "completed"
      return 0
    fi

    # Check error type
    local error_type=$(classify_error "$SESSION_DIR/logs/$download_id.log")

    if [[ "$error_type" == "disk_full" || "$error_type" == "permission_denied" ]]; then
      echo "FATAL: $error_type - aborting all downloads"
      pkill -P $$  # Kill all child processes
      exit 2
    fi

    if [[ $attempt -lt $max_attempts ]]; then
      local wait_time=$((5 * attempt))
      echo "Waiting ${wait_time}s before retry..."
      sleep "$wait_time"
    fi
  done

  update_state_fil