incident-triage

What this skill does

# incident-triage

Rapid incident classification, severity assessment, and response coordination.

## Triggers


Alternate expressions and non-obvious activations (primary phrases are matched automatically from the skill description):

- "P0" / "P1" / "SEV1" / "SEV2" → severity-based incident triage
- "we got paged" → production incident response
- "war room" → incident coordination setup

## Purpose

This skill provides rapid incident response coordination by:
- Classifying incident type and severity
- Assembling response team
- Coordinating initial response actions
- Tracking timeline and status
- Facilitating communication
- Preparing post-incident review

## Behavior

When triggered, this skill:

1. **Gathers incident details**:
   - What is happening?
   - When did it start?
   - Who/what is affected?
   - What changed recently?

2. **Classifies severity**:
   - Assess customer impact
   - Determine scope
   - Assign severity level
   - Calculate business impact

3. **Assembles response team**:
   - Identify required responders
   - Notify on-call personnel
   - Establish incident commander

4. **Initiates response**:
   - Create incident channel/bridge
   - Start timeline documentation
   - Coordinate initial diagnosis

5. **Manages communication**:
   - Internal status updates
   - Customer communication (if needed)
   - Executive notifications (for high severity)

6. **Tracks resolution**:
   - Document actions taken
   - Track mitigation progress
   - Confirm resolution
   - Schedule post-incident review

## Severity Levels

### SEV1 / P0 - Critical

```yaml
sev1:
  name: Critical
  alias: [P0, SEV1, Critical]

  criteria:
    - Complete service outage
    - Data loss or corruption
    - Security breach
    - >50% customers affected
    - Revenue-impacting

  response:
    response_time: 15 minutes
    update_frequency: 15 minutes
    executive_notification: immediate
    customer_communication: within 30 minutes

  escalation:
    - incident_commander: required
    - engineering_manager: required
    - vp_engineering: within 30 minutes
    - cto: within 1 hour (if unresolved)

  target_resolution: 4 hours
```

### SEV2 / P1 - High

```yaml
sev2:
  name: High
  alias: [P1, SEV2, High]

  criteria:
    - Major feature unavailable
    - Significant degradation
    - 10-50% customers affected
    - Workaround exists but painful

  response:
    response_time: 30 minutes
    update_frequency: 30 minutes
    executive_notification: within 1 hour
    customer_communication: within 2 hours (if extended)

  escalation:
    - incident_commander: required
    - engineering_manager: within 1 hour

  target_resolution: 8 hours
```

### SEV3 / P2 - Medium

```yaml
sev3:
  name: Medium
  alias: [P2, SEV3, Medium]

  criteria:
    - Feature partially degraded
    - <10% customers affected
    - Workaround available
    - Non-critical path affected

  response:
    response_time: 2 hours
    update_frequency: 2 hours
    executive_notification: daily summary
    customer_communication: as needed

  escalation:
    - team_lead: within 4 hours

  target_resolution: 24 hours
```

### SEV4 / P3 - Low

```yaml
sev4:
  name: Low
  alias: [P3, SEV4, Low]

  criteria:
    - Minor issue
    - Cosmetic problem
    - Edge case affected
    - Easy workaround

  response:
    response_time: next business day
    update_frequency: daily
    executive_notification: weekly summary

  escalation: standard ticket flow

  target_resolution: 1 week
```

## Incident Response Flow

```
┌─────────────────────────────────────────────────────────────┐
│ 1. DETECTION & TRIAGE                                       │
│    • Alert received or issue reported                       │
│    • Gather initial details                                 │
│    • Classify severity                                      │
│    • Create incident record                                 │
│    • Time: <15 minutes                                      │
└─────────────────────────────────────────────────────────────┘
                          │
                          ▼
┌─────────────────────────────────────────────────────────────┐
│ 2. MOBILIZATION                                             │
│    • Page on-call responders                                │
│    • Establish incident commander                           │
│    • Create communication channel                           │
│    • Notify stakeholders per severity                       │
│    • Time: <5 minutes after triage                          │
└─────────────────────────────────────────────────────────────┘
                          │
                          ▼
┌─────────────────────────────────────────────────────────────┐
│ 3. INVESTIGATION                                            │
│    • Review recent changes                                  │
│    • Check monitoring/logs                                  │
│    • Identify affected components                           │
│    • Form hypothesis                                        │
│    • Time: ongoing, status updates per SLA                  │
└─────────────────────────────────────────────────────────────┘
                          │
                          ▼
┌─────────────────────────────────────────────────────────────┐
│ 4. MITIGATION                                               │
│    • Implement workaround if available                      │
│    • Rollback if change-related                             │
│    • Scale resources if capacity issue                      │
│    • Isolate affected components                            │
│    • Goal: Reduce customer impact                           │
└─────────────────────────────────────────────────────────────┘
                          │
                          ▼
┌─────────────────────────────────────────────────────────────┐
│ 5. RESOLUTION                                               │
│    • Implement permanent fix                                │
│    • Verify fix is effective                                │
│    • Monitor for recurrence                                 │
│    • Update status to resolved                              │
└─────────────────────────────────────────────────────────────┘
                          │
                          ▼
┌─────────────────────────────────────────────────────────────┐
│ 6. POST-INCIDENT                                            │
│    • Schedule post-incident review                          │
│    • Document timeline and actions                          │
│    • Identify root cause                                    │
│    • Create follow-up action items                          │
│    • Update runbooks/documentation                          │
└─────────────────────────────────────────────────────────────┘
```

## Incident Record Format

```markdown
# Incident Report: INC-2025-001234

## Summary

| Field | Value |
|-------|-------|
| Title | Database connection pool exhaustion |
| Severity | SEV1 (Critical) |
| Status | Resolved |
| Start Time | 2025-12-08 14:32 UTC |
| Detected | 2025-12-08 14:35 UTC |
| Resolved | 2025-12-08 15:47 UTC |
| Duration | 1h 15m |
| Impact | 100% of API requests failing |
| Customers Affected | ~45,000 |

## Incident Commander

**Name**: Sarah Chen
**Role**: Senior SRE

## Response Team

| Role | Name | Joined |
|------|------|--------|
| Incident Commander | Sarah Chen | 14:38 |
| Backend Lead | David Kim | 14:40 |
| DBA | Elena Rodriguez | 14:45 |
| Comms Lead | James Wilson | 14:50 |

## Impact Assessment

### Customer Impact
- **Scope**: All customers using web and mobile apps
- **Severity**: Complete service outage
- **Duration**: 1h 15m
- **Affected Features**: All authenticated features

### Business Impact
- **Revenue Loss**: Estimated $XX,XXX
- **SLA Breach**: Yes (99.9% monthly target affected)
- **Customer Complaints**: 127 support tickets

## Timeline

| Time (UTC) | Event |
|------------|-------|
| 14:32 | First customer reports o