model-supply-chain-security

What this skill does

# Model Supply Chain Security

Protect models and inference components from tampering, dependency compromise, and untrusted artifact promotion.

## When to Use This Skill

Use this skill when:
- Pulling pretrained models from public registries (Hugging Face, TensorFlow Hub)
- Building model-serving containers for production deployment
- Establishing trust policies for ML artifact promotion across environments
- Responding to supply chain incidents affecting ML dependencies
- Meeting SLSA or SOC2 compliance requirements for AI systems

## Prerequisites

- `cosign` v2+ installed for signing and verification
- `syft` for SBOM generation of model-serving images
- `crane` or `skopeo` for OCI image inspection
- Container registry with signature support (GHCR, ECR, ACR, Artifact Registry)
- CI/CD pipeline with provenance generation capability

## Threats

- Poisoned pretrained weights or adapters
- Malicious model conversion tools or loaders
- Compromised build pipelines and registries
- Insecure runtime images with critical CVEs
- Typosquatting on model registries
- Deserialization attacks via pickle or custom loaders

## Control Objectives

- Verify artifact integrity end-to-end
- Prove provenance for every promoted model
- Detect vulnerable dependencies before deploy
- Restrict execution to trusted signed artifacts

## Model Signing with Cosign

### Sign a Model Artifact

```bash
# Generate a keypair (store private key securely)
cosign generate-key-pair

# Sign an OCI-packaged model image
cosign sign --key cosign.key ghcr.io/acme/ml-models/sentiment:v2.1.0

# Keyless signing with Sigstore (uses OIDC identity)
cosign sign ghcr.io/acme/ml-models/sentiment:v2.1.0

# Verify the signature
cosign verify --key cosign.pub ghcr.io/acme/ml-models/sentiment:v2.1.0

# Keyless verification (requires certificate identity)
cosign verify \
  [email protected] \
  --certificate-oidc-issuer=https://accounts.google.com \
  ghcr.io/acme/ml-models/sentiment:v2.1.0
```

### Sign Model Weight Files Directly

```bash
# For model files stored as blobs (not OCI images)
# Compute digest and sign
sha256sum model-weights.safetensors > model-weights.sha256
cosign sign-blob --key cosign.key model-weights.safetensors \
  --output-signature model-weights.sig \
  --output-certificate model-weights.crt

# Verify blob signature
cosign verify-blob --key cosign.pub \
  --signature model-weights.sig \
  model-weights.safetensors
```

## SLSA for ML Pipelines

### SLSA Level Requirements for Model Builds

```yaml
# slsa-requirements.yaml
slsa_levels:
  level_1:
    - Build process is scripted (not manual)
    - Provenance document generated automatically
  level_2:
    - Build runs on hosted CI service
    - Provenance is authenticated (signed)
    - Source is version controlled
  level_3:
    - Build environment is ephemeral and isolated
    - Provenance is non-falsifiable (hardened builder)
    - Source integrity verified (two-person review)
```

### Generate SLSA Provenance for Model Training

```yaml
# .github/workflows/model-build-slsa.yml
name: Model Build with SLSA Provenance
on:
  push:
    tags: ['model-v*']

jobs:
  train-and-package:
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read
      packages: write
    steps:
      - uses: actions/checkout@v4

      - name: Train model
        run: python train.py --config configs/production.yaml

      - name: Package model as OCI artifact
        run: |
          oras push ghcr.io/acme/ml-models/sentiment:${{ github.ref_name }} \
            model-weights.safetensors:application/vnd.acme.model.safetensors \
            model-config.json:application/json

      - name: Generate SBOM for training environment
        run: |
          syft dir:. -o cyclonedx-json > training-sbom.json

      - name: Sign and attest
        run: |
          cosign sign ghcr.io/acme/ml-models/sentiment:${{ github.ref_name }}
          cosign attest --predicate training-sbom.json \
            --type cyclonedx \
            ghcr.io/acme/ml-models/sentiment:${{ github.ref_name }}

      - name: Generate provenance
        uses: slsa-framework/slsa-github-generator/.github/workflows/[email protected]
        with:
          image: ghcr.io/acme/ml-models/sentiment
          digest: ${{ steps.push.outputs.digest }}
```

## Model Cards for Provenance

```yaml
# model-card.yaml
model_details:
  name: "sentiment-classifier-v2.1.0"
  version: "2.1.0"
  type: "text-classification"
  framework: "pytorch"
  license: "Apache-2.0"

provenance:
  training_data:
    source: "s3://acme-datasets/sentiment-v3/"
    hash: "sha256:abc123..."
    data_card_ref: "https://internal.acme.com/data-cards/sentiment-v3"
  training_config:
    source: "git://github.com/acme/ml-models@abc123"
    hyperparameters:
      learning_rate: 0.00005
      epochs: 10
      batch_size: 32
  build_environment:
    builder: "github-actions"
    runner: "ubuntu-22.04"
    python: "3.11.7"
    torch: "2.1.2"
    cuda: "12.1"
  build_id: "gh-actions-12345"
  commit_sha: "abc123def456"
  build_timestamp: "2025-01-15T10:30:00Z"
  signed_by: "[email protected]"

performance:
  accuracy: 0.94
  f1_score: 0.93
  evaluation_dataset: "s3://acme-datasets/sentiment-eval-v3/"
  evaluation_hash: "sha256:def456..."

security:
  vulnerability_scan: "clean"
  sbom_ref: "ghcr.io/acme/ml-models/sentiment:v2.1.0.sbom"
  last_security_review: "2025-01-10"
  known_limitations:
    - "May produce biased outputs for underrepresented languages"
    - "Not evaluated for adversarial robustness"
```

## Registry Scanning

```bash
# Scan model-serving image for CVEs
trivy image ghcr.io/acme/ml-models/sentiment-serving:v2.1.0

# Generate SBOM for the serving container
syft ghcr.io/acme/ml-models/sentiment-serving:v2.1.0 -o spdx-json > serving-sbom.json

# Scan SBOM for vulnerabilities
grype sbom:serving-sbom.json --fail-on critical

# Check for known-malicious model files (pickle scanning)
pip install fickling
fickling --check model.pkl
```

### Automated Registry Scan Pipeline

```yaml
# .github/workflows/registry-scan.yml
name: Nightly Registry Scan
on:
  schedule:
    - cron: '0 2 * * *'

jobs:
  scan:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        image:
          - ghcr.io/acme/ml-models/sentiment-serving:latest
          - ghcr.io/acme/ml-models/embedding-serving:latest
          - ghcr.io/acme/ml-models/rag-api:latest
    steps:
      - name: Scan image
        run: |
          trivy image --severity CRITICAL,HIGH \
            --exit-code 1 \
            --format json \
            --output scan-$(echo ${{ matrix.image }} | tr '/:' '-').json \
            ${{ matrix.image }}

      - name: Verify signatures are still valid
        run: |
          cosign verify \
            [email protected] \
            --certificate-oidc-issuer=https://accounts.google.com \
            ${{ matrix.image }}
```

## Promotion Policy Enforcement

```python
#!/usr/bin/env python3
"""model_promotion_gate.py - Verify model meets all promotion criteria."""

import subprocess
import json
import sys

def check_signature(image: str) -> bool:
    result = subprocess.run(
        ["cosign", "verify", "[email protected]",
         "--certificate-oidc-issuer=https://accounts.google.com", image],
        capture_output=True, text=True,
    )
    return result.returncode == 0

def check_vulnerabilities(image: str) -> bool:
    result = subprocess.run(
        ["trivy", "image", "--severity", "CRITICAL", "--exit-code", "1",
         "--quiet", image],
        capture_output=True, text=True,
    )
    return result.returncode == 0

def check_sbom_exists(image: str) -> bool:
    result = subprocess.run(
        ["cosign", "verify-attestation", "--type", "cyclonedx",
         "[email protected]",
         "--ce