regression-bisect

What this skill does

# regression-bisect

Automatically identify the commit that introduced a regression using git bisect.

## Triggers


Alternate expressions and non-obvious activations (primary phrases are matched automatically from the skill description):

- "git bisect" → binary search for regression commit
- "when did this break" → regression commit discovery

## Purpose

This skill automates regression root cause analysis by:
- Using git bisect to binary search commit history
- Running automated tests at each bisect point
- Identifying the exact commit that introduced a failure
- Providing blame information and diff context
- Suggesting potential fixes based on the breaking change

## Behavior

When triggered, this skill:

1. **Validates prerequisites**:
   - Confirms reproducible failure exists
   - Verifies test automation is available
   - Ensures clean working directory
   - Identifies good and bad commits

2. **Executes git bisect**:
   - Starts bisect with known good/bad commits
   - Runs test suite at each bisect point
   - Marks commits as good/bad based on test results
   - Continues until breaking commit is found

3. **Analyzes breaking commit**:
   - Shows commit message and metadata
   - Displays full diff of the breaking change
   - Identifies changed files and functions
   - Maps to requirements/issues if available

4. **Provides context**:
   - Shows blame information for affected code
   - Lists related commits in the area
   - Identifies potential reviewers (commit authors)
   - Suggests rollback or fix strategies

5. **Documents findings**:
   - Creates regression analysis report
   - Links to issue tracker if available
   - Updates regression baseline with findings
   - Generates timeline of when regression was introduced

## Bisect Configuration

### Manual Bisect

```yaml
bisect_manual:
  mode: interactive
  good_commit: v1.2.0  # Known good state
  bad_commit: HEAD     # Known bad state
  test_command: "npm test -- auth.test.ts"

  workflow:
    - git bisect start
    - git bisect bad HEAD
    - git bisect good v1.2.0
    - git bisect run npm test -- auth.test.ts
```

### Automated Bisect

```yaml
bisect_automated:
  mode: automated
  detect_good: auto  # Use last passing CI build
  detect_bad: auto   # Use current failing state
  test_command: "npm test"
  timeout: 600       # 10 minutes per commit test

  auto_detection:
    - check_ci_history
    - find_last_green_build
    - set_as_good_commit
    - use_current_as_bad
```

### Bisect with Custom Test

```yaml
bisect_custom:
  mode: custom
  test_script: scripts/regression-test.sh
  success_exit_code: 0
  failure_exit_code: 1

  test_script_template: |
    #!/bin/bash
    # Custom regression test
    npm run build || exit 1
    npm test -- specific-test.ts || exit 1
    ./validate-output.sh || exit 1
    exit 0
```

## Bisect Report Format

```markdown
# Regression Bisect Report

**Date**: 2026-01-28
**Issue**: Authentication fails on login
**Analyzer**: regression-bisect skill

## Executive Summary

**Breaking Commit**: abc123def456
**Author**: John Developer <[email protected]>
**Date**: 2026-01-15 14:32:00
**Bisect Duration**: 8 commits tested, 12 minutes

**Root Cause**: Token validation logic changed to require `iss` claim

## Bisect Results

### Commit Range

- **Good**: v1.2.0 (2026-01-10) - 50 commits ago
- **Bad**: HEAD (2026-01-28) - Current state
- **Breaking**: abc123def456 (2026-01-15) - 25 commits into range

### Bisect Log

| Commit | Status | Test Result | Duration |
|--------|--------|-------------|----------|
| abc123 | ❌ Bad | auth.test.ts failed | 45s |
| def789 | ❌ Bad | auth.test.ts failed | 42s |
| ghi012 | ✅ Good | all tests passed | 48s |
| jkl345 | ✅ Good | all tests passed | 46s |
| mno678 | ❌ Bad | auth.test.ts failed | 44s |
| **pqr901** | **❌ Breaking** | **First failure** | **43s** |
| stu234 | ✅ Good | all tests passed | 47s |
| vwx567 | ✅ Good | all tests passed | 45s |

## Breaking Commit Analysis

### Commit Details

```
commit pqr901xyz
Author: John Developer <[email protected]>
Date:   Mon Jan 15 14:32:00 2026 +0000

    feat(auth): add JWT issuer validation

    - Require `iss` claim in JWT tokens
    - Validate issuer against whitelist
    - Update token generation to include issuer

    Closes #456
```

### Files Changed

| File | Changes | Impact |
|------|---------|--------|
| src/auth/validate-token.ts | +15/-5 | High - Core validation logic |
| src/auth/generate-token.ts | +8/-2 | Medium - Token generation |
| test/auth/token.test.ts | +25/-0 | Low - Test updates |

### Diff Analysis

```diff
diff --git a/src/auth/validate-token.ts b/src/auth/validate-token.ts
index abc123..def456 100644
--- a/src/auth/validate-token.ts
+++ b/src/auth/validate-token.ts
@@ -15,6 +15,11 @@ export function validateToken(token: string): TokenPayload {
     throw new Error('Invalid token signature');
   }

+  // NEW: Require issuer claim
+  if (!payload.iss) {
+    throw new Error('Token missing issuer claim');
+  }
+
   return payload;
 }
```

**Breaking Change**: The new validation requires `iss` claim, but existing tokens don't include it.

## Root Cause

The commit added mandatory `iss` (issuer) claim validation without:
1. Updating existing token generation (added in same commit but missed migration)
2. Providing backward compatibility for existing tokens
3. Coordinating with deployment to invalidate old sessions

**Impact**: All existing user sessions became invalid immediately.

## Affected Components

- **Authentication**: Token validation fails for all existing sessions
- **API Gateway**: 401 errors on all authenticated endpoints
- **User Experience**: Force logout of all active users

## Related Information

### Traceability

- **Issue**: #456 - "Add JWT issuer validation for security"
- **Requirement**: @.aiwg/requirements/use-cases/UC-AUTH-003-jwt-security.md
- **Review**: PR #789 - Merged without integration testing

### Similar Issues

Git history shows related changes:
- commit xyz789 (2026-01-10): Initial JWT implementation
- commit uvw456 (2026-01-12): Add JWT expiration validation
- commit pqr901 (2026-01-15): Add JWT issuer validation ← BREAKING

### Code Ownership

```
src/auth/validate-token.ts:
  85% John Developer <[email protected]>
  10% Jane Reviewer <[email protected]>
   5% Bob Maintainer <[email protected]>
```

## Recommendations

### Immediate (Hotfix)

- [ ] **Option A**: Revert commit pqr901 to restore service
- [ ] **Option B**: Make `iss` validation optional with feature flag
- [ ] **Option C**: Deploy token refresh that includes `iss` claim

**Recommended**: Option B - Feature flag allows gradual rollout

### Short-term (This Sprint)

- [ ] Add integration tests for auth changes
- [ ] Add backward compatibility tests for token validation
- [ ] Update deployment runbook for auth changes
- [ ] Add session migration logic for breaking changes

### Ongoing

- [ ] Require integration tests for auth PRs
- [ ] Add "breaking change" checklist to PR template
- [ ] Schedule auth regression suite in CI
- [ ] Document token format in API docs

## Fix Strategy

### Proposed Fix

```typescript
// src/auth/validate-token.ts
export function validateToken(token: string): TokenPayload {
  const payload = jwt.verify(token, SECRET_KEY);

  if (!payload.sub) {
    throw new Error('Invalid token signature');
  }

  // FIXED: Make issuer validation optional during migration
  if (featureFlags.requireIssuer && !payload.iss) {
    throw new Error('Token missing issuer claim');
  }

  return payload;
}
```

### Migration Path

1. Deploy fix with `requireIssuer: false`
2. Update token generation to include `iss`
3. Wait for all old tokens to expire (24 hours)
4. Enable `requireIssuer: true`
5. Remove feature flag after validation

## Timeline

```
2026-01-10: v1.2.0 released (working)
2026-01-15: Commit pqr901 merged (breaking change introduced)
2026-01-15: No CI failure (tests didn't catch integration issue)
2026-01-16: Deployed to production
2026-01-16: User r