regression-metrics

What this skill does

# regression-metrics

Track and analyze regression statistics, trends, and health indicators.

## Triggers


Alternate expressions and non-obvious activations (primary phrases are matched automatically from the skill description):

- "regression KPIs" → regression metric dashboard
- "flakiness score" → test stability metrics

## Purpose

This skill provides regression analytics by:
- Tracking regression occurrence rates
- Measuring time-to-detection and time-to-fix
- Analyzing regression patterns and hotspots
- Identifying high-risk areas
- Trending regression metrics over time
- Generating regression health dashboards

## Behavior

When triggered, this skill:

1. **Collects regression data**:
   - Parse regression test results
   - Load historical regression records
   - Gather bisect findings
   - Import baseline comparisons
   - Aggregate issue tracker data

2. **Calculates key metrics**:
   - Regression rate (per sprint/release)
   - Mean time to detect (MTTD)
   - Mean time to fix (MTTF)
   - Regression recurrence rate
   - Escape rate (production regressions)

3. **Identifies patterns**:
   - Common root causes
   - High-regression components
   - Time-of-day/sprint patterns
   - Correlation with code changes

4. **Analyzes trends**:
   - Regression rate over time
   - Detection speed improvements
   - Fix time trends
   - Quality trajectory

5. **Generates visualizations**:
   - Regression heatmaps
   - Trend charts
   - Burn-down tracking
   - Risk matrices

6. **Produces actionable insights**:
   - Prioritize high-risk areas
   - Recommend test improvements
   - Suggest process changes
   - Set quality goals

## Key Metrics

### Regression Rate

```yaml
regression_rate:
  description: Number of regressions per time period
  formula: regressions_detected / time_period
  units: regressions per sprint/week/release

  targets:
    excellent: "< 2 per sprint"
    good: "2-5 per sprint"
    acceptable: "5-10 per sprint"
    poor: "> 10 per sprint"

  calculation:
    count: new regressions introduced
    period: sprint, release, or month
    exclude: known issues, flaky tests
```

### Mean Time to Detect (MTTD)

```yaml
mttd:
  description: Average time from regression introduction to detection
  formula: sum(detection_time) / regression_count
  units: hours or days

  targets:
    excellent: "< 4 hours"
    good: "< 24 hours"
    acceptable: "< 7 days"
    poor: "> 7 days"

  calculation:
    detection_time: commit_time_to_failure_report
    includes: automated and manual detection
```

### Mean Time to Fix (MTTF)

```yaml
mttf:
  description: Average time from detection to fix deployment
  formula: sum(fix_time) / regression_count
  units: hours or days

  targets:
    critical: "< 4 hours"
    high: "< 24 hours"
    medium: "< 7 days"
    low: "< 30 days"

  calculation:
    fix_time: detection_to_fix_deployed
    severity_weighted: true
```

### Escape Rate

```yaml
escape_rate:
  description: Percentage of regressions reaching production
  formula: (production_regressions / total_regressions) * 100
  units: percentage

  targets:
    excellent: "< 5%"
    good: "5-10%"
    acceptable: "10-20%"
    poor: "> 20%"

  calculation:
    production_regressions: found by users/monitoring
    total_regressions: all detected including pre-release
```

### Recurrence Rate

```yaml
recurrence_rate:
  description: Percentage of regressions that recur after fix
  formula: (recurring_regressions / total_fixed) * 100
  units: percentage

  targets:
    excellent: "< 5%"
    good: "5-10%"
    acceptable: "10-15%"
    poor: "> 15%"

  indicates:
    - insufficient test coverage
    - lack of regression tests
    - poor fix quality
```

## Metrics Dashboard

```markdown
# Regression Metrics Dashboard

**Period**: Last 30 Days (2025-12-29 to 2026-01-28)
**Project**: User Service

## Executive Summary

| Metric | Current | Target | Status | Trend |
|--------|---------|--------|--------|-------|
| Regression Rate | 4.2/sprint | < 5 | ✅ Good | ↓ Improving |
| MTTD | 8.5 hours | < 24h | ✅ Good | ↓ Improving |
| MTTF | 18.7 hours | < 24h | ⚠️ Close | → Stable |
| Escape Rate | 12% | < 10% | ⚠️ Above Target | ↑ Worsening |
| Recurrence Rate | 7% | < 10% | ✅ Good | → Stable |

**Overall Health**: ⚠️ Good with Concerns
**Priority Focus**: Reduce production escapes

## Regression Trend (Last 6 Sprints)

```
Sprint 8:  ██████████ 10 regressions
Sprint 9:  ████████   8 regressions
Sprint 10: ██████     6 regressions
Sprint 11: █████      5 regressions
Sprint 12: ████       4 regressions
Sprint 13: ████       4 regressions
           ↓ -60% improvement since Sprint 8
```

**Analysis**: Significant improvement trend. Stabilizing around 4-5 per sprint.

## Detection Speed Trend

```
Week 1: 24h ████████████████████████
Week 2: 18h ██████████████████
Week 3: 12h ████████████
Week 4:  9h █████████
Week 5:  8h ████████
        ↓ -67% improvement in 5 weeks
```

**Analysis**: Automation improvements paying off. Most regressions now caught within hours.

## Component Heatmap

Regressions by component (last 30 days):

| Component | Regressions | Change | Risk Level |
|-----------|-------------|--------|------------|
| src/auth/ | 🔴🔴🔴 3 | +1 | High |
| src/api/ | 🟡🟡 2 | 0 | Medium |
| src/db/ | 🟡🟡 2 | -1 | Medium |
| src/user/ | 🟡 1 | -2 | Low |
| src/utils/ | 🟢 0 | 0 | Low |

**Hotspot Alert**: `src/auth/` showing increased regression rate

## Root Cause Analysis

| Root Cause | Count | % | Trend |
|------------|-------|---|-------|
| Missing test coverage | 5 | 42% | → |
| Integration not tested | 3 | 25% | ↑ |
| Edge case not considered | 2 | 17% | ↓ |
| Flaky test masking issue | 1 | 8% | → |
| Breaking dependency change | 1 | 8% | → |

**Insight**: 67% of regressions preventable with better coverage/integration testing

## Severity Distribution

| Severity | Count | MTTF | Status |
|----------|-------|------|--------|
| Critical | 1 | 3.2h | ✅ Fast response |
| High | 4 | 12.5h | ✅ Within target |
| Medium | 6 | 28.4h | ⚠️ Above target |
| Low | 1 | 72h | ✅ Acceptable |

## Time-to-Detection Analysis

```
Detection Method:
  Automated Tests: 75% (avg 4.2h detection)
  Manual Testing:  17% (avg 32h detection)
  Production:       8% (avg 96h detection)
```

**Insight**: Automation catching most issues early. Need to reduce production escapes.

## Time-to-Fix Analysis

```
Fix Duration by Severity:
  Critical: ▓▓▓ 3.2h (target: 4h) ✅
  High:     ▓▓▓▓▓▓ 12.5h (target: 24h) ✅
  Medium:   ▓▓▓▓▓▓▓▓▓▓▓▓▓▓ 28.4h (target: 24h) ⚠️
  Low:      ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ 72h ✅
```

**Issue**: Medium-severity regressions taking slightly longer than target

## Regression Recurrence

| Original Issue | Recurred | Reason |
|----------------|----------|--------|
| AUTH-101 | ✅ Yes | Missing regression test |
| API-205 | ❌ No | Regression test added |
| DB-089 | ❌ No | Regression test added |
| USER-145 | ❌ No | Regression test added |

**Recurrence Rate**: 25% (1 of 4) - One regression lacked test

## Production Escapes

Regressions that reached production:

| Issue | Severity | Detection | Impact | MTTD |
|-------|----------|-----------|--------|------|
| AUTH-203 | High | User report | 500 users | 12h |

**Analysis**: 1 escape this period. Auth module regression bypassed staging tests.

## Recommendations

### High Priority

1. **Add integration tests for auth flows**
   - Reason: 3 regressions in auth, 1 production escape
   - Impact: Reduce auth regressions by ~60%
   - Effort: 2 days

2. **Improve staging test coverage**
   - Reason: Production escape indicates gap
   - Impact: Reduce escape rate to <5%
   - Effort: 1 week

3. **Reduce medium-severity MTTF**
   - Reason: 28.4h vs 24h target
   - Impact: Faster user impact resolution
   - Effort: Process improvement

### Medium Priority

4. **Add regression tests for all fixes**
   - Reason: 25% recurrence rate on fixes without tests
   - Impact: Zero recurrence for tested fixes
   - Effort: Ongoing discipline

5. **Monitor auth mo