cost-calculator

What this skill does

# ML Training Cost Calculator

**Purpose:** Provide production-ready cost estimation tools for ML training and inference across cloud GPU platforms (Modal, Lambda Labs, RunPod).

**Activation Triggers:**
- Estimating training costs for ML models
- Comparing GPU platform pricing
- Calculating GPU hours for training jobs
- Budgeting for ML projects
- Optimizing inference costs
- Evaluating cost-effectiveness of different GPU types
- Planning resource allocation

**Key Resources:**
- `scripts/estimate-training-cost.sh` - Calculate training costs based on model size, data, GPU type
- `scripts/estimate-inference-cost.sh` - Estimate inference costs for production workloads
- `scripts/calculate-gpu-hours.sh` - Convert training parameters to GPU hours
- `scripts/compare-platforms.sh` - Compare costs across Modal, Lambda, RunPod
- `templates/cost-breakdown.json` - Structured cost breakdown template
- `templates/platform-pricing.yaml` - Up-to-date platform pricing data
- `examples/training-cost-estimate.md` - Example training cost calculation
- `examples/inference-cost-estimate.md` - Example inference cost analysis

## Platform Pricing Overview

### Modal (Serverless - Pay Per Second)

**GPU Options:**
- **T4**: $0.000164/sec ($0.59/hr) - Development, small models
- **L4**: $0.000222/sec ($0.80/hr) - Cost-effective training
- **A10**: $0.000306/sec ($1.10/hr) - Mid-range training
- **A100 40GB**: $0.000583/sec ($2.10/hr) - Large model training
- **A100 80GB**: $0.000694/sec ($2.50/hr) - Very large models
- **H100**: $0.001097/sec ($3.95/hr) - Cutting-edge training
- **H200**: $0.001261/sec ($4.54/hr) - Latest generation
- **B200**: $0.001736/sec ($6.25/hr) - Maximum performance

**Free Credits:**
- Starter: $30/month free
- Startup credits: Up to $50,000 FREE

### Lambda Labs (On-Demand Hourly)

**Single GPU:**
- **1x A10**: $0.31/hr - Cheapest single GPU option
- **1x V100 16GB**: $0.55/hr - Most affordable multi-GPU base

**8x GPU Clusters:**
- **8x V100**: $4.40/hr ($0.55/GPU) - Most affordable multi-GPU
- **8x A100 40GB**: $10.32/hr ($1.29/GPU)
- **8x A100 80GB**: $14.32/hr ($1.79/GPU)
- **8x H100**: $23.92/hr ($2.99/GPU)

### RunPod (Serverless - Pay Per Minute)

**Key Features:**
- Pay-per-minute billing
- FlashBoot <200ms cold-starts
- Zero egress fees on storage
- 30+ GPU SKUs available

## Cost Estimation Scripts

### 1. Estimate Training Cost

**Script:** `scripts/estimate-training-cost.sh`

**Usage:**
```bash
bash scripts/estimate-training-cost.sh \
  --model-size 7B \
  --dataset-size 10000 \
  --epochs 3 \
  --gpu t4 \
  --platform modal
```

**Parameters:**
- `--model-size`: Model size (125M, 350M, 1B, 3B, 7B, 13B, 70B)
- `--dataset-size`: Number of training samples
- `--epochs`: Number of training epochs
- `--batch-size`: Training batch size (default: auto-calculated)
- `--gpu`: GPU type (t4, a10, a100-40gb, a100-80gb, h100)
- `--platform`: Cloud platform (modal, lambda, runpod)
- `--peft`: Use PEFT/LoRA (yes/no, default: no)
- `--mixed-precision`: Use FP16/BF16 (yes/no, default: yes)

**Output:**
```json
{
  "model": "7B",
  "dataset_size": 10000,
  "epochs": 3,
  "gpu": "T4",
  "platform": "Modal",
  "estimated_hours": 4.2,
  "cost_breakdown": {
    "compute_cost": 2.48,
    "storage_cost": 0.05,
    "total_cost": 2.53
  },
  "cost_optimizations": {
    "with_peft": 1.26,
    "savings_percentage": 50
  },
  "alternative_platforms": {
    "lambda_a10": 1.30,
    "runpod_t4": 2.40
  }
}
```

**Calculation Methodology:**
- Estimates tokens per sample (avg 500 tokens)
- Calculates total training tokens
- Applies throughput rates per GPU type
- Accounts for PEFT (90% memory reduction)
- Accounts for mixed precision (2x speedup)

### 2. Estimate Inference Cost

**Script:** `scripts/estimate-inference-cost.sh`

**Usage:**
```bash
bash scripts/estimate-inference-cost.sh \
  --requests-per-day 1000 \
  --avg-latency 2 \
  --gpu t4 \
  --platform modal \
  --deployment serverless
```

**Parameters:**
- `--requests-per-day`: Expected daily requests
- `--avg-latency`: Average inference time (seconds)
- `--gpu`: GPU type
- `--platform`: Cloud platform
- `--deployment`: Deployment type (serverless, dedicated)
- `--batch-inference`: Batch requests (yes/no, default: no)

**Output:**
```json
{
  "requests_per_day": 1000,
  "requests_per_month": 30000,
  "avg_latency_sec": 2,
  "gpu": "T4",
  "platform": "Modal Serverless",
  "cost_breakdown": {
    "daily_compute_seconds": 2000,
    "daily_cost": 0.33,
    "monthly_cost": 9.90,
    "cost_per_request": 0.00033
  },
  "scaling_analysis": {
    "requests_10k_day": 99.00,
    "requests_100k_day": 990.00
  },
  "dedicated_alternative": {
    "monthly_cost": 442.50,
    "break_even_requests_day": 4500
  }
}
```

### 3. Calculate GPU Hours

**Script:** `scripts/calculate-gpu-hours.sh`

**Usage:**
```bash
bash scripts/calculate-gpu-hours.sh \
  --model-params 7B \
  --tokens-total 30M \
  --gpu a100-40gb
```

**Parameters:**
- `--model-params`: Model parameters (125M, 350M, 1B, 3B, 7B, 13B, 70B)
- `--tokens-total`: Total training tokens
- `--gpu`: GPU type
- `--peft`: Use PEFT (yes/no)
- `--multi-gpu`: Number of GPUs (default: 1)

**GPU Throughput Benchmarks:**
```
T4 (16GB):
  - 7B full fine-tune: 150 tokens/sec
  - 7B with PEFT: 600 tokens/sec

A100 40GB:
  - 7B full fine-tune: 800 tokens/sec
  - 7B with PEFT: 3200 tokens/sec
  - 13B with PEFT: 1600 tokens/sec

A100 80GB:
  - 13B full fine-tune: 600 tokens/sec
  - 70B with PEFT: 400 tokens/sec

H100:
  - 70B with PEFT: 1200 tokens/sec
```

### 4. Compare Platforms

**Script:** `scripts/compare-platforms.sh`

**Usage:**
```bash
bash scripts/compare-platforms.sh \
  --training-hours 4 \
  --gpu-type a100-40gb
```

**Output:**
```markdown
# Platform Cost Comparison

## Training Job: 4 hours on A100 40GB

| Platform | GPU Cost | Egress Fees | Total | Notes |
|----------|----------|-------------|-------|-------|
| Modal | $8.40 | $0.00 | $8.40 | Serverless, pay-per-second |
| Lambda | $5.16 | $0.00 | $5.16 | Cheapest for dedicated |
| RunPod | $8.00 | $0.00 | $8.00 | Pay-per-minute |

## Winner: Lambda Labs ($5.16)

Savings: $3.24 (38.6% vs Modal)

Recommendation: Use Lambda for long-running dedicated training, Modal for
serverless/bursty workloads.
```

## Cost Templates

### Cost Breakdown Template

**Template:** `templates/cost-breakdown.json`

```json
{
  "project_name": "ML Training Project",
  "cost_estimate": {
    "training": {
      "model_size": "7B",
      "training_runs": 4,
      "hours_per_run": 4.2,
      "gpu_type": "T4",
      "platform": "Modal",
      "cost_per_run": 2.48,
      "total_training_cost": 9.92
    },
    "inference": {
      "deployment_type": "serverless",
      "expected_requests_month": 30000,
      "gpu_type": "T4",
      "platform": "Modal",
      "monthly_cost": 9.90
    },
    "storage": {
      "model_artifacts_gb": 14,
      "dataset_storage_gb": 5,
      "monthly_storage_cost": 0.50
    },
    "total_monthly_cost": 20.32,
    "breakdown_percentage": {
      "training": 48.8,
      "inference": 48.7,
      "storage": 2.5
    }
  },
  "cost_optimizations_applied": {
    "peft_lora": "50% training cost reduction",
    "mixed_precision": "2x faster training",
    "serverless_inference": "Pay only for actual usage",
    "batch_inference": "Up to 10x reduction in inference cost"
  },
  "potential_savings": {
    "without_optimizations": 45.00,
    "with_optimizations": 20.32,
    "total_savings": 24.68,
    "savings_percentage": 54.8
  }
}
```

### Platform Pricing Data

**Template:** `templates/platform-pricing.yaml`

```yaml
platforms:
  modal:
    billing: per-second
    free_credits: 30  # USD per month
    startup_credits: 50000  # USD for eligible startups
    gpus:
      t4:
        price_per_sec: 0.000164
        price_per_hour: 0.59
        vram_gb: 16
      a100_40gb:
        price_per_sec: 0.000583
        price_per_hour: 2.10
        vram_gb: 40
      h100:
        price_per_sec: 0.001097
        price_per_hour: 3.95