ci-cd-pipeline-patterns

What this skill does

# CI/CD Pipeline Patterns

A comprehensive skill for designing, implementing, and optimizing CI/CD pipelines using GitHub Actions and modern DevOps practices. Master workflow automation, testing strategies, deployment patterns, and release management for continuous software delivery.

## When to Use This Skill

Use this skill when:

- Setting up continuous integration and deployment pipelines for projects
- Automating build, test, and deployment workflows
- Implementing multi-environment deployment strategies (staging, production)
- Managing release automation and versioning
- Configuring matrix builds for multi-platform testing
- Securing CI/CD pipelines with secrets and OIDC
- Optimizing pipeline performance with caching and parallelization
- Building containerized applications with Docker in CI
- Deploying to cloud platforms (AWS, Azure, GCP, Vercel, Netlify)
- Implementing infrastructure as code with Terraform/CloudFormation
- Setting up monorepo CI/CD patterns
- Creating reusable workflow templates and custom actions
- Implementing deployment strategies (blue-green, canary, rolling)
- Automating changelog generation and semantic versioning
- Integrating quality gates and code coverage checks

## Core Concepts

### CI/CD Fundamentals

**Continuous Integration (CI)**: Automatically building and testing code changes as developers commit to the repository.

**Continuous Deployment (CD)**: Automatically deploying code changes to production after passing tests.

**Continuous Delivery**: Keeping code in a deployable state, with manual approval for production deployment.

### GitHub Actions Architecture

GitHub Actions provides event-driven automation directly integrated with your repository.

#### Workflows

YAML files in `.github/workflows/` that define automated processes:

```yaml
name: CI Pipeline
on: [push, pull_request]
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Build project
        run: npm run build
```

**Key Components:**
- **name**: Human-readable workflow name
- **on**: Events that trigger the workflow (push, pull_request, schedule, workflow_dispatch)
- **jobs**: Collection of steps that run in sequence or parallel
- **runs-on**: The runner environment (ubuntu-latest, windows-latest, macos-latest)

#### Jobs

Groups of steps executed on the same runner:

```yaml
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm test

  deploy:
    needs: test  # Runs after 'test' job completes
    runs-on: ubuntu-latest
    steps:
      - run: npm run deploy
```

**Job Features:**
- **needs**: Define job dependencies (sequential execution)
- **if**: Conditional execution based on expressions
- **strategy**: Matrix builds for multiple configurations
- **outputs**: Share data between jobs
- **environment**: Deployment environments with protection rules

#### Steps

Individual tasks within a job:

```yaml
steps:
  - name: Checkout code
    uses: actions/checkout@v4

  - name: Setup Node.js
    uses: actions/setup-node@v4
    with:
      node-version: '20'

  - name: Install dependencies
    run: npm ci

  - name: Run tests
    run: npm test
```

**Step Types:**
- **uses**: Run a pre-built action from marketplace or repository
- **run**: Execute shell commands
- **with**: Provide inputs to actions
- **env**: Set environment variables for the step

#### Actions

Reusable units of code that perform specific tasks:

**Official Actions:**
- `actions/checkout@v4`: Check out repository code
- `actions/setup-node@v4`: Setup Node.js environment
- `actions/cache@v4`: Cache dependencies
- `actions/upload-artifact@v4`: Upload build artifacts
- `actions/download-artifact@v4`: Download artifacts from previous jobs

**Marketplace Actions:**
- `docker/build-push-action@v5`: Build and push Docker images
- `aws-actions/configure-aws-credentials@v4`: Configure AWS credentials
- `codecov/codecov-action@v4`: Upload code coverage
- `google-github-actions/auth@v2`: Authenticate with Google Cloud

#### Secrets and Variables

**Secrets**: Encrypted sensitive data (API keys, credentials, tokens)

```yaml
steps:
  - name: Deploy to production
    env:
      API_KEY: ${{ secrets.API_KEY }}
      DATABASE_URL: ${{ secrets.DATABASE_URL }}
    run: npm run deploy
```

**Variables**: Non-sensitive configuration data

```yaml
env:
  NODE_ENV: ${{ vars.NODE_ENV }}
  API_ENDPOINT: ${{ vars.API_ENDPOINT }}
```

**Secret Types:**
- **Repository secrets**: Available to all workflows in a repository
- **Environment secrets**: Scoped to specific environments (production, staging)
- **Organization secrets**: Shared across repositories in an organization

#### Artifacts

Files produced by workflows that can be downloaded or used by other jobs:

```yaml
- name: Upload build artifacts
  uses: actions/upload-artifact@v4
  with:
    name: dist-files
    path: dist/
    retention-days: 7

- name: Download artifacts
  uses: actions/download-artifact@v4
  with:
    name: dist-files
    path: ./dist
```

### Workflow Triggers

#### Event Triggers

**Push Events:**
```yaml
on:
  push:
    branches:
      - main
      - develop
      - 'release/**'
    paths:
      - 'src/**'
      - 'package.json'
    tags:
      - 'v*'
```

**Pull Request Events:**
```yaml
on:
  pull_request:
    types: [opened, synchronize, reopened]
    branches:
      - main
    paths-ignore:
      - 'docs/**'
      - '**.md'
```

**Schedule (Cron):**
```yaml
on:
  schedule:
    - cron: '0 0 * * *'  # Daily at midnight UTC
    - cron: '0 */6 * * *'  # Every 6 hours
```

**Manual Triggers (workflow_dispatch):**
```yaml
on:
  workflow_dispatch:
    inputs:
      environment:
        description: 'Environment to deploy to'
        required: true
        type: choice
        options:
          - staging
          - production
      version:
        description: 'Version to deploy'
        required: true
        type: string
```

**Release Events:**
```yaml
on:
  release:
    types: [published, created, released]
```

**Workflow Call (Reusable Workflows):**
```yaml
on:
  workflow_call:
    inputs:
      environment:
        required: true
        type: string
    secrets:
      api-key:
        required: true
```

### Matrix Builds

Run jobs across multiple configurations in parallel:

```yaml
jobs:
  test:
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest, windows-latest, macos-latest]
        node-version: [18, 20, 22]
        include:
          - os: ubuntu-latest
            node-version: 20
            coverage: true
        exclude:
          - os: macos-latest
            node-version: 18
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}
      - run: npm test
      - if: matrix.coverage
        run: npm run coverage
```

**Matrix Features:**
- **Parallel execution**: All combinations run simultaneously
- **include**: Add specific configurations
- **exclude**: Remove specific combinations
- **fail-fast**: Stop all jobs if one fails (default: true)
- **max-parallel**: Limit concurrent jobs

### Caching Strategies

Speed up workflows by caching dependencies:

**Node.js Caching:**
```yaml
- uses: actions/setup-node@v4
  with:
    node-version: '20'
    cache: 'npm'  # Automatically caches npm dependencies
```

**Custom Caching:**
```yaml
- uses: actions/cache@v4
  with:
    path: |
      ~/.npm
      ~/.cache
      node_modules
    key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
    restore-keys: |
      ${{ runner.os }}-node-
```

**Docker Layer Caching:**
```yaml
- uses: docker/build-push-action@v5
  with:
    context: .
    cache-from: type=gha
    cache-to: type=gha,mode=max
```

## Testing Strategies in CI

### Unit Testing

Fast, isolated tests for individual components:

```yaml
jobs:
  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4