managing-pipelines

What this skill does

# Pipeline Management

Decision guidance for GitHub Actions CI/CD pipelines, deployment strategies, and infrastructure automation.

## Contents

- When to use which pattern
- Security quick reference
- Performance quick reference
- Workflow architecture quick reference
- Deployment quick reference
- Infrastructure as code quick reference
- Observability quick reference
- Cross-pipeline conventions
- Pipeline debugging checklist

## When to use which pattern

| Scenario | Reference | Why |
| --- | --- | --- |
| Hardening against supply chain attacks | Security | SHA pinning, permissions, OIDC |
| Speeding up slow CI builds | Performance | Caching, matrix builds, concurrency |
| DRY-ing up duplicated workflow YAML | Workflow architecture | Reusable workflows vs composite actions |
| Setting up staging/production deploys | Deployment | Environment promotion, protection rules |
| Adding Terraform/OpenTofu to CI | Infrastructure | Plan-on-PR, apply-on-merge, drift detection |
| Tracking pipeline reliability | Observability | OTel, DORA metrics, SLOs |
| Reviewing a PR that modifies workflows | Security + Workflow | Permissions audit, secret exposure review |
| Debugging flaky pipelines | Observability + Performance | Metrics, cache hit rates, concurrency |
| Migrating from Jenkins/CircleCI | Workflow architecture | Action patterns, reusable workflow design |
| Setting up monorepo CI | Performance | Path filtering, selective job execution |

## Security quick reference

**Use for:** Preventing supply chain attacks, minimizing credential exposure, hardening runner environments.

**Key decisions:**

- Pin all third-party actions to full commit SHAs, not tags
- Set org-level default token permissions to read-only
- Use OIDC for cloud auth instead of stored credentials
- Never use `pull_request_target` without understanding the security model

See [references/security-hardening.md](references/security-hardening.md) for attack patterns and mitigations.

## Performance quick reference

**Use for:** Reducing CI times, optimizing runner costs, parallelizing builds.

**Key decisions:**

- Cache dependency installs AND build artifacts (not just `node_modules`)
- Use `fail-fast: false` for CI matrices, `true` for deployment
- Set concurrency groups with `cancel-in-progress: true` for CI, `false` for deploys
- Use path filtering in monorepos to skip irrelevant jobs

See [references/performance-optimization.md](references/performance-optimization.md) for caching strategies and runner selection.

## Workflow architecture quick reference

**Use for:** Structuring reusable CI/CD components, managing action dependencies.

**Key decisions:**

- Reusable workflows for entire pipeline templates; composite actions for shared steps
- Pass secrets explicitly, not with `secrets: inherit`
- Automate SHA pin updates with Dependabot or Renovate
- Restrict allowed actions at the org level

See [references/workflow-architecture.md](references/workflow-architecture.md) for patterns and versioning.

## Deployment quick reference

**Use for:** Environment promotion, deployment gates, progressive delivery.

**Key decisions:**

- Use GitHub Environments with branch restrictions for production
- Release-based promotion gives the cleanest audit trail
- Progressive delivery (canary/blue-green) via Argo Rollouts or Flagger
- Custom deployment protection rules for SLO-gated deployments

See [references/deployment-strategies.md](references/deployment-strategies.md) for promotion patterns and rollback strategies.

## Infrastructure as code quick reference

**Use for:** Terraform/OpenTofu pipelines, drift detection, policy enforcement.

**Key decisions:**

- Always save plan output and apply the saved plan (never plan-then-apply without `-out`)
- Post plan output as PR comments for review
- Segment state by functional boundary, not geography
- Run scheduled drift detection separately from code-triggered deploys

See [references/infrastructure-pipelines.md](references/infrastructure-pipelines.md) for IaC workflow patterns.

## Observability quick reference

**Use for:** Pipeline reliability tracking, incident response, capacity planning.

**Key decisions:**

- Instrument pipelines with OpenTelemetry (runs as traces, jobs as spans)
- Track DORA metrics: deployment frequency, lead time, change failure rate, MTTR
- Set SLOs for pipeline reliability (e.g., 99% main branch build success)
- Monitor cache hit rates and queue times as leading indicators

See [references/pipeline-observability.md](references/pipeline-observability.md) for instrumentation and metrics.

## Pipeline debugging checklist

### Slow CI builds

1. Check cache hit rates (low = cold start overhead)
2. Look for sequential jobs that could run in parallel
3. Verify concurrency groups aren't queuing unnecessarily
4. Check runner specs (CPU-bound work on small runners)
5. Look for full-repo checkouts when sparse checkout would work

### Failed deployments

1. Check environment protection rule approvals
2. Verify OIDC token audience and subject claims
3. Check if concurrency group blocked/cancelled the run
4. Review Terraform plan output for unexpected changes
5. Check if deployment protection rules (Datadog, etc.) rejected

### Security incidents

1. Audit recent changes to workflow files and action versions
2. Check for new `pull_request_target` usage
3. Review GITHUB_TOKEN permissions in affected workflows
4. Scan for secrets in workflow logs (step outputs, artifacts)
5. Check if any action SHAs were recently changed

### Flaky pipelines

1. Check if tests have timing dependencies (see `condition-based-waiting` skill)
2. Look for shared state between matrix jobs
3. Verify caches aren't corrupted (clear and rebuild)
4. Check for rate limiting from external services
5. Review runner availability (self-hosted runner capacity)