cicd-pipeline

What this skill does

# CI/CD Pipeline (GitHub Actions)

Set up and manage CI/CD pipelines using GitHub Actions. Covers workflow creation, testing, deployment, release automation, and debugging.

## When to Use

- Setting up automated testing on push/PR
- Creating deployment pipelines (staging, production)
- Automating releases with changelogs and tags
- Debugging failing CI workflows
- Setting up matrix builds for cross-platform testing
- Managing secrets and environment variables in CI
- Optimizing CI with caching and parallelism

## Quick Start: Add CI to a Project

### Node.js project

```yaml
# .github/workflows/ci.yml
name: CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
      - run: npm ci
      - run: npm test
      - run: npm run lint
```

### Python project

```yaml
# .github/workflows/ci.yml
name: CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          cache: pip
      - run: pip install -r requirements.txt
      - run: pytest
      - run: ruff check .
```

### Go project

```yaml
# .github/workflows/ci.yml
name: CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-go@v5
        with:
          go-version: "1.22"
      - run: go test ./...
      - run: go vet ./...
```

### Rust project

```yaml
# .github/workflows/ci.yml
name: CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: dtolnay/rust-toolchain@stable
      - uses: Swatinem/rust-cache@v2
      - run: cargo test
      - run: cargo clippy -- -D warnings
```

## Common Patterns

### Matrix builds (test across versions/OSes)

```yaml
jobs:
  test:
    strategy:
      fail-fast: false
      matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]
        node-version: [18, 20, 22]
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}
      - run: npm ci
      - run: npm test
```

### Conditional jobs

```yaml
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm test

  deploy:
    needs: test
    if: github.ref == 'refs/heads/main' && github.event_name == 'push'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: ./deploy.sh
```

### Caching dependencies

```yaml
# Node.js (automatic with setup-node)
- uses: actions/setup-node@v4
  with:
    node-version: 20
    cache: npm  # or yarn, pnpm

# Generic caching
- uses: actions/cache@v4
  with:
    path: |
      ~/.cache/pip
      ~/.cargo/registry
      node_modules
    key: ${{ runner.os }}-deps-${{ hashFiles('**/package-lock.json') }}
    restore-keys: |
      ${{ runner.os }}-deps-
```

### Artifacts (save build outputs)

```yaml
- uses: actions/upload-artifact@v4
  with:
    name: build-output
    path: dist/
    retention-days: 7

# Download in another job
- uses: actions/download-artifact@v4
  with:
    name: build-output
    path: dist/
```

### Run on schedule (cron)

```yaml
on:
  schedule:
    - cron: "0 6 * * 1"  # Every Monday at 6 AM UTC
  workflow_dispatch:  # Also allow manual trigger
```

## Deployment Workflows

### Deploy to production on tag

```yaml
name: Release

on:
  push:
    tags:
      - "v*"

jobs:
  release:
    runs-on: ubuntu-latest
    permissions:
      contents: write
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
      - run: npm ci
      - run: npm run build
      - run: npm test

      # Create GitHub release
      - uses: softprops/action-gh-release@v2
        with:
          generate_release_notes: true
          files: |
            dist/*.js
            dist/*.css
```

### Deploy to multiple environments

```yaml
name: Deploy

on:
  push:
    branches: [main, staging]

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: ${{ github.ref == 'refs/heads/main' && 'production' || 'staging' }}
    steps:
      - uses: actions/checkout@v4
      - run: npm ci && npm run build
      - run: |
          if [ "${{ github.ref }}" = "refs/heads/main" ]; then
            ./deploy.sh production
          else
            ./deploy.sh staging
          fi
        env:
          DEPLOY_TOKEN: ${{ secrets.DEPLOY_TOKEN }}
```

### Docker build and push

```yaml
name: Docker

on:
  push:
    branches: [main]
    tags: ["v*"]

jobs:
  build:
    runs-on: ubuntu-latest
    permissions:
      packages: write
    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-buildx-action@v3
      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
      - uses: docker/build-push-action@v6
        with:
          push: true
          tags: |
            ghcr.io/${{ github.repository }}:latest
            ghcr.io/${{ github.repository }}:${{ github.sha }}
          cache-from: type=gha
          cache-to: type=gha,mode=max
```

### npm publish on release

```yaml
name: Publish

on:
  release:
    types: [published]

jobs:
  publish:
    runs-on: ubuntu-latest
    permissions:
      id-token: write
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          registry-url: https://registry.npmjs.org
      - run: npm ci
      - run: npm test
      - run: npm publish --provenance
        env:
          NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
```

## Secrets Management

### Set secrets via CLI

```bash
# Set a repository secret
gh secret set DEPLOY_TOKEN --body "my-secret-value"

# Set from a file
gh secret set SSH_KEY < ~/.ssh/deploy_key

# Set for a specific environment
gh secret set DB_PASSWORD --env production --body "p@ssw0rd"

# List secrets
gh secret list

# Delete a secret
gh secret delete OLD_SECRET
```

### Use secrets in workflows

```yaml
env:
  # Available to all steps in this job
  DATABASE_URL: ${{ secrets.DATABASE_URL }}

steps:
  - run: echo "Deploying..."
    env:
      # Available to this step only
      API_KEY: ${{ secrets.API_KEY }}
```

### Environment protection rules

Set up via GitHub UI or API:
- Required reviewers before deployment
- Wait timers
- Branch restrictions
- Custom deployment branch policies

```bash
# View environments
gh api repos/{owner}/{repo}/environments | jq '.environments[].name'
```

## Workflow Debugging

### Re-run failed jobs

```bash
# List recent workflow runs
gh run list --limit 10

# View a specific run
gh run view <run-id>

# View failed job logs
gh run view <run-id> --log-failed

# Re-run failed jobs only
gh run rerun <run-id> --failed

# Re-run entire workflow
gh run rerun <run-id>
```

### Debug with SSH (using tmate)

```yaml
# Add this step before the failing step
- uses: mxschmitt/action-tmate@v3
  if: failure()
  with:
    limit-access-to-actor: true
```

### Common failures and fixes

**"Permission denied" on scripts**
```yaml
- run: chmod +x ./scripts/deploy.sh && ./scripts/deploy.sh
```

**"Node modules not found"**
```yaml
# Make sure npm ci runs before npm test
- run: npm ci     # Install exact lockfile versions
- run: npm test   # Now node_modules exists
```

**"Resource not accessible by integration"**
```yaml
# Add permissions block
permissions:
  contents: write
  packages: write
  pull-requests: write
```

**Cache not restoring*