managing-secrets

What this skill does

# Managing Secrets

Secure storage, rotation, and delivery of secrets (API keys, database credentials, TLS certificates) for applications and infrastructure.

## When to Use This Skill

Use when:
- Storing API keys, database credentials, or encryption keys
- Implementing secret rotation (manual or automatic)
- Syncing secrets from external stores to Kubernetes
- Setting up dynamic secrets (database, cloud providers)
- Scanning code for leaked secrets
- Implementing zero-knowledge patterns
- Meeting compliance requirements (SOC 2, ISO 27001, PCI DSS)

## Quick Decision Frameworks

### Framework 1: Choosing a Secret Store

| Scenario | Primary Choice | Alternative |
|----------|----------------|-------------|
| Kubernetes + Multi-Cloud | Vault + ESO | Cloud Secret Manager + ESO |
| Kubernetes + Single Cloud | Cloud Secret Manager + ESO | Vault + ESO |
| Serverless (AWS Lambda) | AWS Secrets Manager | AWS Parameter Store |
| Multi-Cloud Enterprise | HashiCorp Vault | Doppler (SaaS) |
| Small Team (<10 apps) | Doppler, Infisical | 1Password Secrets Automation |
| GitOps-Centric | SOPS (git-encrypted) | Sealed Secrets (K8s-only) |

**Decision Tree:**
- Kubernetes? → External Secrets Operator (ESO) with chosen backend
- Single cloud? → Cloud-native (AWS/GCP/Azure)
- Multi-cloud/on-prem? → HashiCorp Vault
- GitOps? → SOPS or Sealed Secrets

### Framework 2: Static vs. Dynamic Secrets

| Secret Type | Use Dynamic? | TTL | Solution |
|-------------|-------------|-----|----------|
| Database credentials | YES | 1 hour | Vault DB engine |
| Cloud IAM (AWS/GCP) | YES | 15 min | Vault cloud engine |
| SSH/RDP access | YES | 5 min | Vault SSH engine |
| TLS certificates | YES | 24 hours | Vault PKI / cert-manager |
| Third-party API keys | NO | Quarterly | Vault KV v2 (manual rotation) |

### Framework 3: Kubernetes Secret Delivery

| Method | Use Case | Rotation | Restart Required |
|--------|----------|----------|------------------|
| **External Secrets Operator** | Static secrets, periodic sync | Polling (1h) | Yes |
| **Secrets Store CSI Driver** | File-based, watch rotation | inotify | No |
| **Vault Secrets Operator** | Vault-specific, dynamic | Automatic renewal | Optional |

## HashiCorp Vault Fundamentals

### Core Components

- **Secrets Engines**: KV v2 (static), Database (dynamic), AWS, PKI, SSH
- **Auth Methods**: Kubernetes, JWT/OIDC, AppRole, LDAP
- **Policies**: HCL-based access control (least privilege)
- **Leases**: TTL for secrets, auto-renewal, auto-revocation

### Static Secrets (KV v2)

```bash
# Create secret
vault kv put secret/myapp/config api_key=sk_live_EXAMPLE

# Read secret
vault kv get secret/myapp/config

# List versions
vault kv metadata get secret/myapp/config
```

### Dynamic Database Credentials

```bash
# Configure PostgreSQL
vault write database/config/postgres \
  plugin_name=postgresql-database-plugin \
  connection_url="postgresql://{{username}}:{{password}}@postgres:5432/mydb"

# Create role
vault write database/roles/app-role \
  db_name=postgres \
  creation_statements="CREATE ROLE \"{{name}}\"..." \
  default_ttl="1h"

# Generate credentials
vault read database/creds/app-role
```

For detailed Vault architecture, see `references/vault-architecture.md`.

## Kubernetes Integration

### External Secrets Operator (ESO)

Syncs secrets from 30+ providers to Kubernetes Secrets.

```yaml
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: vault-backend
spec:
  provider:
    vault:
      server: "https://vault.example.com"
      auth:
        kubernetes:
          role: "app-role"
```

```yaml
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: database-credentials
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: vault-backend
  target:
    name: db-credentials
  data:
  - secretKey: password
    remoteRef:
      key: secret/data/database/config
```

### Vault Secrets Operator (VSO)

Kubernetes-native Vault integration with automatic lease renewal.

```yaml
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultDynamicSecret
metadata:
  name: postgres-creds
spec:
  vaultAuthRef: vault-auth
  mount: database
  path: creds/app-role
  renewalPercent: 67  # Renew at 67% of TTL
  destination:
    name: dynamic-db-creds
```

For ESO vs CSI vs VSO comparison, see `references/kubernetes-integration.md`.

## Secret Rotation Patterns

### Pattern 1: Versioned Static Secrets (Blue/Green)

1. Create new secret version in Vault
2. Update staging environment
3. Monitor for errors (24-48 hours)
4. Gradual production rollout (10% → 50% → 100%)
5. Revoke old secret (after 7 days)

### Pattern 2: Dynamic Database Credentials

Vault auto-generates credentials with short TTL:
- App fetches credentials from Vault
- Vault automatically renews lease (at 67% of TTL)
- On expiration, Vault revokes access
- On renewal failure, app requests new credentials

### Pattern 3: TLS Certificate Rotation

Using cert-manager + Vault PKI:
- cert-manager requests certificate from Vault
- Automatically renews before expiration (default: 67% of duration)
- Updates Kubernetes Secret on renewal
- Optional pod restart (via Reloader)

For detailed rotation workflows, see `references/rotation-patterns.md`.

## Multi-Language Integration

### Python (hvac)

```python
import hvac

client = hvac.Client(url='https://vault.example.com')
client.auth.kubernetes(role='app-role', jwt=jwt)

# Fetch dynamic credentials
response = client.secrets.database.generate_credentials(name='postgres-role')
username = response['data']['username']
password = response['data']['password']
```

### Go (Vault API)

```go
import vault "github.com/hashicorp/vault/api"

client, _ := vault.NewClient(vault.DefaultConfig())
k8sAuth, _ := auth.NewKubernetesAuth("app-role")
client.Auth().Login(context.Background(), k8sAuth)

secret, _ := client.Logical().Read("database/creds/postgres-role")
```

### TypeScript (node-vault)

```typescript
import vault from 'node-vault';

const client = vault({ endpoint: 'https://vault.example.com' });
await client.kubernetesLogin({ role: 'app-role', jwt });

const response = await client.read('database/creds/postgres-role');
```

For complete examples, see `examples/dynamic-db-credentials/`.

## Secret Scanning

### Pre-Commit Hooks (Gitleaks)

```bash
# Install Gitleaks
brew install gitleaks

# Run on staged files
gitleaks protect --staged --verbose
```

Pre-commit hook prevents secrets from being committed.
For setup, see `examples/secret-scanning/pre-commit`.

### CI/CD Integration

```yaml
# GitHub Actions
- name: Run Gitleaks
  uses: gitleaks/gitleaks-action@v2
```

### Remediation Workflow

When a secret is leaked:
1. **Rotate immediately** (within 1 hour)
2. **Revoke at provider**
3. **Remove from Git history** (BFG Repo-Cleaner)
4. **Force push** (notify team)
5. **Audit access** (who had access during leak window)
6. **Document incident**

For detailed remediation, see `references/secret-scanning.md`.

## Zero-Knowledge Patterns

### Client-Side Encryption (E2EE)

User password → PBKDF2 → encryption key → encrypt secret → send to server

Server stores only encrypted blobs (cannot decrypt).

### Shamir's Secret Sharing

Split secret into N shares, require M to reconstruct (e.g., 3 of 5).

```bash
# Initialize Vault with Shamir shares
vault operator init -key-shares=5 -key-threshold=3

# Unseal requires 3 of 5 key shares
vault operator unseal <KEY_1>
vault operator unseal <KEY_2>
vault operator unseal <KEY_3>
```

For implementations, see `references/zero-knowledge.md`.

## Library Recommendations (2025)

### Secret Stores

| Library | Use Case | Trust Score |
|---------|----------|-------------|
| HashiCorp Vault | Enterprise, multi-cloud | High (73.3/100) |
| External Secrets Operator | Kubernetes integration | High (85.0/100) |
| AWS Secrets Manager | AWS workloads | High |
| GCP Secret Manager | GCP workloads | High |
| Azure Key Vault | Azure workloads | High |

### Secret Scanning

| Library | Use Case | T